The problem with USB stick and msdosfs
da0: <pqi IntelligentStick 0.00> Removable Direct Access SCSI-2 device
da0: 40.000MB/s transfers
da0: 963MB (1974271 512 byte sectors: 64H 32S/T 963C)
$ sudo dd if=/dev/zero of=/mnt/file bs=512k count=100
100+0 records in
100+0 records out
52428800 bytes transferred in 1.744845 secs (30047824 bytes/sec)
$ time sudo umount /mnt
real 4m7.152s <- ?!?!?!
It doesn't happen with any other OS (tried Debian, MacOSX and Windows XP)
and there is no problem in DragonFly with the very same formatted to UFS
#1 Updated by dillon about 8 years ago
:It doesn't happen with any other OS (tried Debian, MacOSX and Windows XP)
:and there is no problem in DragonFly with the very same formatted to UFS
I'm guessing it is a function of msdosfs, which doesn't try to
aggregate clusterable writes. Linux's block devices work differently,
it's easier for linux to cluster writes. Not really on my list,
non-native filesystems aren't really designed with performance
#2 Updated by hasso about 8 years ago
50MB in 4 minutes isn't really "a performance problem" IMHO, but a serious
usability problem ;). And I doubt that the lack of clustering is the one
to blame here (at least not alone). And as I said - it isn't the problem
with every USB mass storage media, with other stuff I have access to show
reasonable performance with msdosfs.
#6 Updated by jdc about 8 years ago
The below may or may not be relevant to Hasso's situation, but I'm
chiming in regardless.
I did some testing of this nature on FreeBSD, using their new USB stack
in -CURRENT, specifically on USB flash drives. The old USB stack
performed horribly (and I've a feeling what's in DFBSD).
Here are the numbers:
Be sure to read the Points of Interest. Larger block sizes (above
128KB) performed equal to, or sometimes worse (!), than 128KB. Also
note the performance increase with 64KB blocks.
My point here is that yes, there's going to be a difference between
512-byte and 512-kilobyte blocks, but when it comes to USB flash drives,
there is in fact a "sweet spot".
#7 Updated by dillon about 8 years ago
Insofar as msdosfs goes the BIO's are done in msdos 'clusters',
which I think are around 4K.
USB flash sticks are one and two-chip solutions with virtually
no buffering or smarts. Trying to chase performance on these things