Bug #1231

The problem with USB stick and msdosfs

Added by hasso almost 6 years ago. Updated almost 2 years ago.

Status:ClosedStart date:
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:-
Target version:-

Description

da0: <pqi IntelligentStick 0.00> Removable Direct Access SCSI-2 device
da0: 40.000MB/s transfers
da0: 963MB (1974271 512 byte sectors: 64H 32S/T 963C)

$ sudo dd if=/dev/zero of=/mnt/file bs=512k count=100
100+0 records in
100+0 records out
52428800 bytes transferred in 1.744845 secs (30047824 bytes/sec)
$ time sudo umount /mnt
real 4m7.152s <- ?!?!?!
user 0m0.008s
sys 0m0.047s
$

It doesn't happen with any other OS (tried Debian, MacOSX and Windows XP)
and there is no problem in DragonFly with the very same formatted to UFS
either.

History

#1 Updated by dillon almost 6 years ago

:It doesn't happen with any other OS (tried Debian, MacOSX and Windows XP)
:and there is no problem in DragonFly with the very same formatted to UFS
:either.
:
:--
:Hasso Tepper

I'm guessing it is a function of msdosfs, which doesn't try to
aggregate clusterable writes. Linux's block devices work differently,
it's easier for linux to cluster writes. Not really on my list,
non-native filesystems aren't really designed with performance
in mind.

-Matt
Matthew Dillon
<>

#2 Updated by hasso almost 6 years ago

50MB in 4 minutes isn't really "a performance problem" IMHO, but a serious
usability problem ;). And I doubt that the lack of clustering is the one
to blame here (at least not alone). And as I said - it isn't the problem
with every USB mass storage media, with other stuff I have access to show
reasonable performance with msdosfs.

#3 Updated by corecode almost 6 years ago

I've noticed that writes to msdosfs always used 4kb blocks. That's probably a clustering problem then.

cheers
simon

#4 Updated by jdc almost 6 years ago

Does the behaviour change if you use standard 512-byte blocks? People
seem to think that USB pen/flash drives behave exactly like hard disks
when it comes to their methodology of storage and block handling -- they
don't.

#5 Updated by corecode almost 6 years ago

Yes. Smaller writes are much worse than larger writes.

#6 Updated by jdc almost 6 years ago

The below may or may not be relevant to Hasso's situation, but I'm
chiming in regardless.

I did some testing of this nature on FreeBSD, using their new USB stack
in -CURRENT, specifically on USB flash drives. The old USB stack
performed horribly (and I've a feeling what's in DFBSD).

Here are the numbers:

http://lists.freebsd.org/pipermail/freebsd-current/2008-November/000271.html

Be sure to read the Points of Interest. Larger block sizes (above
128KB) performed equal to, or sometimes worse (!), than 128KB. Also
note the performance increase with 64KB blocks.

My point here is that yes, there's going to be a difference between
512-byte and 512-kilobyte blocks, but when it comes to USB flash drives,
there is in fact a "sweet spot".

#7 Updated by dillon almost 6 years ago

Insofar as msdosfs goes the BIO's are done in msdos 'clusters',
which I think are around 4K.

USB flash sticks are one and two-chip solutions with virtually
no buffering or smarts. Trying to chase performance on these things
is pointless.

-Matt
Matthew Dillon
<>

#8 Updated by tuxillo almost 2 years ago

  • Description updated (diff)
  • Status changed from New to Closed
  • Assignee deleted (0)

Hi,

msdos(5) is much faster now. I use to copy big files to my usb stick, and I don't get those slow transfers.

Cheers,
Antonio Huete

Also available in: Atom PDF