DragonFlyBSD

:On ufs,download a file from ftp,the max download speed is 11Mb/s
:Same file,same ftp,the max download speed is 4Mb/s
:Can I say that hammer fs is slower than ufs?

I have no idea why you would be seeing a difference, HAMMER is not
    slower then UFS reading a single file, certainly not that slow.

-Matt
                    Matthew Dillon 
                    <dillon@backplane.com>

OK,this is the screen

same net,same file,same ftpd,different ufs and hammer fs.

All the same if FreeBSD,the transfer speed is 11Mb/s
Maybe the difference reason is Ethernet driver difference of FB and DF?

Do you observe any difference if you run the following command
on DragonFly machine for each file?

$ dd if=src.tar.gz of=/dev/null

Also try changing the block size(bs=XXX, defaults to 512) to see if there's
any difference.
$ dd if=src.tar.gz of=/dev/null bs=4k

Cheers.

On Tue, Aug 12, 2008 at 7:54 AM, lhmwzy
<sinknull@crater.dragonflybsd.org> wrote:

lhmwzy <lhmwzy@gmail.com> added the comment:

All the same if FreeBSD,the transfer speed is 11Mb/s
Maybe the difference reason is Ethernet driver difference of FB and DF?

Output of dmesg and ifconfig on both client and server. And which
NICs are used to do the ftp testing. BTW, could you use netperf or
iperf to do some simple TCP stream tests?

Best Regards,
sephe

on DF:
dd if=src.tar.gz of=/dev/null
189612+1 records in
189612+1 records out
97081686 bytes transferred in 2.343634 secs (41423571 bytes/sec)

dd if=src.tar.gz of=/dev/null bs=4k
23701+1 records in
23701+1 records out
97081686 bytes transferred in 0.359248 secs (270235901 bytes/sec)

dd if=src.tar.gz of=/dev/null bs=8k
11850+1 records in
11850+1 records out
97081686 bytes transferred in 0.201687 secs (481348025 bytes/sec)

dd if=src.tar.gz of=/dev/null bs=16k
5925+1 records in
5925+1 records out
97081686 bytes transferred in 0.121297 secs (800362656 bytes/sec)

on FB7:

dd if=src.tar.gz of=/dev/null
189612+1 records in
189612+1 records out
97081686 bytes transferred in 0.386370 secs (251266144 bytes/sec)

dd if=src.tar.gz of=/dev/null bs=4k
23701+1 records in
23701+1 records out
97081686 bytes transferred in 0.217616 secs (446114485 bytes/sec)

dd if=src.tar.gz of=/dev/null bs=8k
11850+1 records in
11850+1 records out
97081686 bytes transferred in 0.068447 secs (1418350897 bytes/sec)

dd if=src.tar.gz of=/dev/null bs=16k
5925+1 records in
5925+1 records out
97081686 bytes transferred in 0.067074 secs (1447380483 bytes/sec)

Ok, I had to confirm something before asking you to do anything:
are you downloading a 100Mbyte from a DragonFly FTP server, and
talking about the difference in the transfer rate between when
the filesystem from which ftpd reads is UFS or HAMMER? (which is
what I assumed) If so, your result below lacks the comparison between
reading from DragonFly UFS and reading from DragonFly HAMMER. Instead,
you're comparing sequential read rate between DragonFly and FreeBSD7.

dmesg:
Copyright (c) 2003-2008 The DragonFly Project.
Copyright (c) 1992-2003 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
DragonFly 2.0.0-RELEASE #0: Thu Aug 7 10:16:03 CST 2008
lhm@hummer.org:/usr/obj/www/src/sys/LHMWZY
TSC clock: 2127890488 Hz, i8254 clock: 1193193 Hz
CPU: Intel(R) Xeon(R) CPU E7320 2.13GHz (2127.48-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0x6f8 Stepping = 8
Features=0xfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS>
Features2=0x2211<SSE3,DS_CPL,SSSE3,CX16>
real memory = 1073741824 (1048576K bytes)
avail memory = 1034481664 (1010236K bytes)
Preloaded elf kernel "/kernel" at 0xc0404000.
Preloaded elf module "/modules/acpi.ko" at 0xc0404288.
Pentium Pro MTRR support enabled
md0: Malloc disk
pcibios: BIOS version 2.10
Using $PIR table, 9 entries at 0xc00fdf30
ACPI: RSDP 0x0xf6c60/0x0014 (v 0 PTLTD )
ACPI: RSDT 0x0x3fefab68/0x0030 (v 1 PTLTD RSDT 0x06040000 LTP
0x00000000)
ACPI: FACP 0x0x3fefef14/0x0074 (v 1 INTEL 440BX 0x06040000 PTL
0x000F4240)
ACPI: DSDT 0x0x3fefab98/0x437C (v 1 PTLTD Custom 0x06040000 MSFT
0x0100000D)
ACPI: FACS 0x0x3fefffc0/0x0040
ACPI: APIC 0x0x3fefef88/0x0050 (v 1 PTLTD APIC 0x06040000 LTP
0x00000000)
ACPI: BOOT 0x0x3fefefd8/0x0028 (v 1 PTLTD $SBFTBL$ 0x06040000 LTP
0x00000001)
npx0: <math processor> on motherboard
npx0: INT 16 interface
Using XMM optimized bcopy/copyin/copyout
acpi0: <PTLTD RSDT> on motherboard
acpi0: Power Button (fixed)
Warning: ACPI is disabling APM's device. You can't run both
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
cpu0: <ACPI CPU> on acpi0
acpi_acad0: <AC Adapter> on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x64,0x60 irq 1 on acpi0
atkbd0: <AT Keyboard> flags 0x1 irq 1 on atkbdc0
kbd0 at atkbd0
acpi_cpu: throttling enabled, 8 steps (100% to 12.5%), currently 100.0%
psm0: failed to get data.
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: model IntelliMouse, device ID 3
isab0: <ACPI Generic ISA bridge> on acpi0
ppc0 port 0x378-0x37f irq 7 on acpi0
ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode
ppbus0: <Parallel port bus> on ppc0
legacypci0 on motherboard
pcib0: <Intel 82443BX (440 BX) host to PCI bridge> on legacypci0
pci0: <PCI bus> on pcib0
pcib1: <Intel 82443BX (440 BX) PCI-PCI (AGP) bridge> at device 1.0 on pci0
pci1: <PCI bus> on pcib1
isab1: <Intel 82371AB PCI to ISA bridge> at device 7.0 on pci0
isa0: <ISA bus> on isab1
pci0: <Intel PIIX4 ATA controller> at 7.1
pci0: <Intel 82371AB Power management controller> at 7.3
pci0: <VGA-compatible display device> at 15.0
mpt0: <LSILogic 1030 Ultra4 Adapter> port 0x1080-0x10ff mem
0xf4800000-0xf4800fff irq 9 at device 16.0 on pci0
lnc0: <AMD PCnet-PCI> port 0x1400-0x147f irq 11 at device 17.0 on pci0
lnc0: 16 receive buffers, 4 transmit buffers
lnc0: MAC address: 00:50:56:98:50:54
orm0: <Option ROMs> at iomem
0xc0000-0xc7fff,0xca000-0xcafff,0xdc000-0xdffff,0xe0000-0xe3fff on
isa0
pmtimer0 on isa0
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
ppc1: cannot reserve I/O port range
Waiting 5 seconds for SCSI devices to settle
da0 at mpt0 bus 0 target 0 lun 0
da0: <VMware Virtual disk 1.0> Fixed Direct Access SCSI-2 device
da0: 3.300MB/s transfers (3.300MHz DT, offset -765769400)
da0: 40960MB (83886080 512 byte sectors: 255H 63S/T 5221C)
da1 at mpt0 bus 0 target 1 lun 0
da1: <VMware Virtual disk 1.0> Fixed Direct Access SCSI-2 device
da1: 3.300MB/s transfers (3.300MHz DT, offset -765769400)
da1: 8192MB (16777216 512 byte sectors: 255H 63S/T 1044C)
da2 at mpt0 bus 0 target 2 lun 0
da2: <VMware Virtual disk 1.0> Fixed Direct Access SCSI-2 device
da2: 3.300MB/s transfers (3.300MHz DT, offset -765769400)
da2: 15360MB (31457280 512 byte sectors: 255H 63S/T 1958C)
Mounting root from ufs:/dev/da0s1a
Waiting (max 60 seconds) for system thread vnlru to stop...stopped
Waiting (max 60 seconds) for system thread bufdaemon_hw to stop...stopped
Waiting (max 60 seconds) for system thread bufdaemon to stop...stopped
Waiting (max 60 seconds) for system thread syncer to stop...stopped

syncing disks... 7
done
Uptime: 18m53s
Shutting down ACPI
Rebooting...
Copyright (c) 2003-2008 The DragonFly Project.
Copyright (c) 1992-2003 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
DragonFly 2.0-RELEASE #0: Mon Aug 11 14:15:12 CST 2008
lhm@hummer.org:/usr/obj/www/src/sys/LHMWZY
TSC clock: 2127853808 Hz, i8254 clock: 1193173 Hz
CPU: Intel(R) Xeon(R) CPU E7320 2.13GHz (2127.49-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0x6f8 Stepping = 8
Features=0xfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS>
Features2=0x2211<SSE3,DS_CPL,SSSE3,CX16>
real memory = 1073741824 (1048576K bytes)
avail memory = 1034481664 (1010236K bytes)
Preloaded elf kernel "/kernel" at 0xc0404000.
Preloaded elf module "/modules/acpi.ko" at 0xc0404288.
Pentium Pro MTRR support enabled
md0: Malloc disk
pcibios: BIOS version 2.10
Using $PIR table, 9 entries at 0xc00fdf30
ACPI: RSDP 0x0xf6c60/0x0014 (v 0 PTLTD )
ACPI: RSDT 0x0x3fefab68/0x0030 (v 1 PTLTD RSDT 0x06040000 LTP
0x00000000)
ACPI: FACP 0x0x3fefef14/0x0074 (v 1 INTEL 440BX 0x06040000 PTL
0x000F4240)
ACPI: DSDT 0x0x3fefab98/0x437C (v 1 PTLTD Custom 0x06040000 MSFT
0x0100000D)
ACPI: FACS 0x0x3fefffc0/0x0040
ACPI: APIC 0x0x3fefef88/0x0050 (v 1 PTLTD APIC 0x06040000 LTP
0x00000000)
ACPI: BOOT 0x0x3fefefd8/0x0028 (v 1 PTLTD $SBFTBL$ 0x06040000 LTP
0x00000001)
npx0: <math processor> on motherboard
npx0: INT 16 interface
Using XMM optimized bcopy/copyin/copyout
acpi0: <PTLTD RSDT> on motherboard
acpi0: Power Button (fixed)
Warning: ACPI is disabling APM's device. You can't run both
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
cpu0: <ACPI CPU> on acpi0
acpi_acad0: <AC Adapter> on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x64,0x60 irq 1 on acpi0
atkbd0: <AT Keyboard> flags 0x1 irq 1 on atkbdc0
kbd0 at atkbd0
acpi_cpu: throttling enabled, 8 steps (100% to 12.5%), currently 100.0%
psm0: failed to get data.
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: model IntelliMouse, device ID 3
isab0: <ACPI Generic ISA bridge> on acpi0
ppc0 port 0x378-0x37f irq 7 on acpi0
ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode
ppbus0: <Parallel port bus> on ppc0
legacypci0 on motherboard
pcib0: <Intel 82443BX (440 BX) host to PCI bridge> on legacypci0
pci0: <PCI bus> on pcib0
pcib1: <Intel 82443BX (440 BX) PCI-PCI (AGP) bridge> at device 1.0 on pci0
pci1: <PCI bus> on pcib1
isab1: <Intel 82371AB PCI to ISA bridge> at device 7.0 on pci0
isa0: <ISA bus> on isab1
pci0: <Intel PIIX4 ATA controller> at 7.1
pci0: <Intel 82371AB Power management controller> at 7.3
pci0: <VGA-compatible display device> at 15.0
mpt0: <LSILogic 1030 Ultra4 Adapter> port 0x1080-0x10ff mem
0xf4800000-0xf4800fff irq 9 at device 16.0 on pci0
lnc0: <AMD PCnet-PCI> port 0x1400-0x147f irq 11 at device 17.0 on pci0
lnc0: 16 receive buffers, 4 transmit buffers
lnc0: MAC address: 00:50:56:98:50:54
orm0: <Option ROMs> at iomem
0xc0000-0xc7fff,0xca000-0xcafff,0xdc000-0xdffff,0xe0000-0xe3fff on
isa0
pmtimer0 on isa0
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
ppc1: cannot reserve I/O port range
Waiting 5 seconds for SCSI devices to settle
da0 at mpt0 bus 0 target 0 lun 0
da0: <VMware Virtual disk 1.0> Fixed Direct Access SCSI-2 device
da0: 3.300MB/s transfers (3.300MHz DT, offset -765769400)
da0: 40960MB (83886080 512 byte sectors: 255H 63S/T 5221C)
da1 at mpt0 bus 0 target 1 lun 0
da1: <VMware Virtual disk 1.0> Fixed Direct Access SCSI-2 device
da1: 3.300MB/s transfers (3.300MHz DT, offset -765769400)
da1: 8192MB (16777216 512 byte sectors: 255H 63S/T 1044C)
da2 at mpt0 bus 0 target 2 lun 0
da2: <VMware Virtual disk 1.0> Fixed Direct Access SCSI-2 device
da2: 3.300MB/s transfers (3.300MHz DT, offset -765769400)
da2: 15360MB (31457280 512 byte sectors: 255H 63S/T 1958C)
Mounting root from ufs:/dev/da0s1a

Ok,there is maybe something wrong.
I download the same file from the same ftpd(windows ftpd).
So it is write speed,not read speed.
I do the test again:
the result as following:

ftp> put src.tar.gz
local: src.tar.gz remote: src.tar.gz
227 Entering Passive Mode (10,66,23,79,12,151)
150 Opening BINARY mode data connection for src.tar.gz.
100% |****************************************| 94806 KiB 11.21
MiB/s 00:00 ETA
226 Transfer complete.

ftp> get src.tar.gz
local: src.tar.gz remote: src.tar.gz
227 Entering Passive Mode (10,66,23,79,12,154)
150 Opening BINARY mode data connection for src.tar.gz (97081686 Bytes).
100% |****************************************| 94806 KiB 4.11
MiB/s 00:00 ETA
226 Transfer complete.
97081686 bytes received in 00:22 (4.11 MiB/s)

Can I say that the hammer fs read speed is normal,but write speed is abnormal?

:ftp> put src.tar.gz
:local: src.tar.gz remote: src.tar.gz
:227 Entering Passive Mode (10,66,23,79,12,151)
:150 Opening BINARY mode data connection for src.tar.gz.
:100% |****************************************| 94806 KiB 11.21
:MiB/s 00:00 ETA
:226 Transfer complete.
:
:ftp> get src.tar.gz
:local: src.tar.gz remote: src.tar.gz
:227 Entering Passive Mode (10,66,23,79,12,154)
:150 Opening BINARY mode data connection for src.tar.gz (97081686 Bytes).
:100% |****************************************| 94806 KiB 4.11
:MiB/s 00:00 ETA
:226 Transfer complete.
:97081686 bytes received in 00:22 (4.11 MiB/s)
:
:Can I say that the hammer fs read speed is normal,but write speed is abnormal?

This can't have anything to do with HAMMER at all.  A 97 MB file will
    fit completely in the buffer cache, reading or writing.

This has to do with some VMWare muff up, probably related to either
    the network driver or the disk driver.

Try posting the FreeBSD7 dmesg output too, maybe that will give us 
    a hint.

As far as fully cached reads with dd go... that's probably a pipe
    or bcopy optimization issue.  But, again, all bets are off under VMWare.

-Matt

dmesg for FB7:

Copyright (c) 1992-2008 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 7.0-STABLE #0: Wed Jun 11 07:31:11 CST 2008
lhm@freebsd.bxzx.com:/usr/obj/usr/src/sys/lhmwzy
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(R) CPU E7320 @ 2.13GHz (2129.71-MHz K8-class CPU)
Origin = "GenuineIntel" Id = 0x6f8 Stepping = 8
Features=0xfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS>
Features2=0x2211<SSE3,DS_CPL,SSSE3,CX16>
AMD Features=0x20100800<SYSCALL,NX,LM>
AMD Features2=0x1<LAHF>
usable memory = 2140442624 (2041 MB)
avail memory = 2065850368 (1970 MB)
ACPI APIC Table: <PTLTD APIC >
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 1
MADT: Forcing active-low polarity and level trigger for SCI
ioapic0 <Version 1.1> irqs 0-23 on motherboard
kbd1 at kbdmux0
acpi0: <PTLTD RSDT> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
Timecounter "ACPI-safe" frequency 3579545 Hz quality 850
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
cpu0: <ACPI CPU> on acpi0
acpi_throttle0: <ACPI CPU Throttling> on cpu0
cpu1: <ACPI CPU> on acpi0
acpi_throttle1: <ACPI CPU Throttling> on cpu1
acpi_throttle1: failed to attach P_CNT
device_attach: acpi_throttle1 attach returned 6
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
isab0: <PCI-ISA bridge> at device 7.0 on pci0
isa0: <ISA bus> on isab0
pci0: <mass storage, ATA> at device 7.1 (no driver attached)
pci0: <bridge> at device 7.3 (no driver attached)
vgapci0: <VGA-compatible display> port 0x1060-0x106f mem
0xf8000000-0xfbffffff,0xf4000000-0xf47fffff at device 15.0 on pci0
mpt0: <LSILogic 1030 Ultra4 Adapter> port 0x1080-0x10ff mem
0xf4810000-0xf4810fff irq 17 at device 16.0 on pci0
mpt0: [ITHREAD]
mpt0: MPI Version=1.2.0.0
em0: <Intel(R) PRO/1000 Network Connection 6.9.5> port 0x1070-0x1077
mem 0xf4820000-0xf483ffff,0xf4800000-0xf480ffff irq 18 at device 17.0
on pci0
em0: Memory Access and/or Bus Master bits were not set!
em0: [FILTER]
em0: Ethernet address: 00:0c:29:67:80:ec
acpi_acad0: <AC Adapter> on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: [ITHREAD]
psm0: model IntelliMouse, device ID 3
orm0: <ISA Option ROMs> at iomem
0xc0000-0xc7fff,0xca000-0xcafff,0xdc000-0xdffff,0xe0000-0xe3fff on
isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Timecounters tick every 1.000 msec
Waiting 5 seconds for SCSI devices to settle
da0 at mpt0 bus 0 target 0 lun 0
da0: <VMware Virtual disk 1.0> Fixed Direct Access SCSI-2 device
da0: 3.300MB/s transfers
da0: Command Queueing Enabled
da0: 81920MB (167772160 512 byte sectors: 255H 63S/T 10443C)
SMP: AP CPU #1 Launched!
Trying to mount root from ufs:/dev/da0s1a

DF and FB7 is under a same ESX 3.5.

The two configurations aren't even remotely similar.

You are comparing a FreeBSD SMP system running on 2 cpus against
    a DragonFly kernel running on 1.

The VMWare running FreeBSD is emulating an Intel ethernet adapter (EM0).
    The one running DragonFly is emulating LNC0.  Totally different network
    adapters.  EM0 is a much better adapter.

The disk configuration is also completely different.  If those VMWare
    disks are backed by plain files they could be fragmented on top of that.

If you want to compare the systems you need to equalize the
    configurations so you are comparing the same thing.  FreeBSD
    will still probably pipe the dd test better, but it won't be 2x.

My guess is that the network adapter differences are creating the
    biggest issue.  Get EM running on DragonFly too, instead of LNC.  Also
    either run FreeBSD UP or build a SMP DragonFly kernel.  Check the
    VMWare setup for those disks as well.

-Matt

You're right.
Sorry for annoyance.
TKS.

(closing bug report)