Bug #2577

virtio-blk iops performance is cpu limited on high end devices

Added by gjs278 about 3 years ago. Updated about 3 years ago.

Status:NewStart date:08/01/2013
Priority:NormalDue date:
Assignee:vsrinivas% Done:


Target version:-


Qemu 1.5.2 on Gentoo AMD64 kernel 3.10.4 host with an i7 980x processor at 4.2ghz

qemu-system-x86_64 -machine accel=kvm -cpu host -drive file=/dev/fioa3,if=virtio,cache=none,aio=native -balloon virtio -smp 6 -m 6144M

/dev/fioa3 is a 160gb slc fusion-io card

DragonFlyBSD 3.4.2-RELEASE is the guest OS

# /tmp/rr1 /dev/vbd0
Device /dev/vbd0 bufsize 512 limit 10.800GB nprocs 32
randrand 1.001s 24293 loops = 41.202uS/loop
randrand 1.002s 24384 loops = 41.072uS/loop
randrand 1.001s 24633 loops = 40.640uS/loop

# /tmp/rr1 /dev/vbd0 4096
Device /dev/vbd0 bufsize 4096 limit 10.800GB nprocs 32
randrand 1.001s 24333 loops = 41.119uS/loop
randrand 1.002s 24389 loops = 41.052uS/loop
randrand 1.001s 24367 loops = 41.093uS/loop

# /tmp/rr1 /dev/vbd0 16384
Device /dev/vbd0 bufsize 16384 limit 10.800GB nprocs 32
randrand 1.001s 21006 loops = 41.619uS/loop
randrand 1.002s 21167 loops = 41.348uS/loop
randrand 1.001s 20520 loops = 48.850uS/loop

cpu usage on the host nears 100% while /tmp/rr1 is running. at nprocs 32, the device should be capable of at least 100k iops. the same 25k limit is seen using an ssd array as well.


#1 Updated by vsrinivas about 3 years ago

  • Assignee set to vsrinivas

#2 Updated by gjs278 about 3 years ago

# sysctl machdep.cpu_idle_hlt=0

Raises cpu usage on the host OS to 100% and results in better /tmp/rr1 numbers of around 30k loops at the 512 and 4096 block size, compared to the previous results of 25k max under machdep.cpu_idle_hlt = 2

Also available in: Atom PDF