qcow2 Physical Size With Different preallocation Settings

qcow2 Physical Size With Different preallocation Settings

Get Social!

The qcow2 image format is the defacto image format for KVM/ QEMU virtual machines. The format provides various parameters that can be configured when creating the image, each with their benefits and drawbacks.

The below section describes the preallocation attribute and how it can effect the size and performance of a virtual machine.

Please see this blog post for more information on preallocation, and then continue on to the results!

The below tests are all performed on the same hardware and on a single hard disk that’s on it’s own dedicated bus with no other traffic. The disk itself is a mechanical Western Digital Green 2TB. I’ve done it on this rather than on an SSD so that the results are more dramatic so that we can understand how much IO performance makes a difference. The tests are also using the same size virtual disk image of 4GB, encryption is disabled, cluster_size is the default 65536 and lazy_refcounts are off unless otherwise specified.

Virtual Disk Creation Time

The first example shows how long it takes to create each virtual disk image and how much physical disk space is being used/ reserved for the image.

preallocation setting Time to create Physical size on disk
off 0.312s 196K
metadata 0.507s 844K
falloc 0.015s 4.0G
full 39.402s 4.0G

As you can see, it takes a huge amount of time to use the full allocation setting because the filesystem it’s being written to has to assign the full size of the file and write empty data to it (in our case around 4GB). The least is taken up with falloc and that’s because qemu-img uses the underlying filesystems fallocate function to allocate the disk space without having to write data to consume the full size.

You can download the bash script used for the above test Disk Test preallocation Disk Size Script.

Virtual Disk Performance

The next thing to consider is the performance of each virtual disk type. For this test each virtual disk is mounted and written to using dd. The performance hit here is when the virtual disk has to expand and allocate physical disk space for new data clusters and new metadata, with metadata creation being by far the biggest overhead.

preallocation setting Time to create MB/ s
off 184.23s 729kB/ s
metadata 85.87s 1.6MB/ s
falloc 100.77s 1.3MB/ s
full 84.31s 1.6MB/ s

You can immediately see that virtual disks with no preallocation take by far the longest to write to, and virtual disks with full preallocation are the quickest. Interestingly a preallocation value of metadata is a very close second to full which indicates much of the performance hit is down to assigning and managing metadata.

You can download the bash script used for the above test Disk Test preallocation Write Performance.





16-Mar-2016 at 3:08 pm

Ah yeah you really tested it, thank you for your investigation. This really shows how metadata impact performance.

I guess that’s only a short-term look on performance (fresh disk and first full write), but measuring a long-term effect is nearly impossible, that would depend on so many criteria like how many vm’s on what kind of storage, how big is the expected data-fragmentation in- and outside of the vm’s etc…

Right now i’ll definetively make sure i use metadata-preallocation, especially when reclaiming disk space ;)


22-Dec-2018 at 12:25 pm

I still don’t understand why falloc is slower than metadata. I thought falloc also write metadata to disk ?

On From another page on this site I found:
“preallocation=falloc – allocates space for the metadata and data but marks the blocks as unallocated. This will provision slower than metadata but quicker than full. Guest write performance will be much quicker than metadata and similar to full.”

Secondly, falloc makes a sparsed file, so one could say, it doesn’t use 4GB of space on disk, even though ls -s says, right?

Leave a Reply

Visit our advertisers

Quick Poll

Do you use ZFS on Linux?

Visit our advertisers