You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I was playing with zfs and found something interesting: there is big difference in writing to zfs file directly VS writing into same file mounted as filesystem / block device.
Writing 2Gb directly leads to 2.8Gb real writes to disk, 40% more writes, this is OK.
But writing 2Gb into same file mounted as loop device leads to 4.2Gb of real writes to disk.
What kind of magic is working here ?
Same happens to zvols, same with files mounted as filesystem - 4.2Gb.
But Qcow2 is an EXCEPTION (because it does not use intermediate block device?), writes to Qcow2 are 2.8Gb too, not 4.2Gb (same when mounted inside VM or directly using qemu-nbd).
Openzfs 2.1.11-1
UPDATE: tested with openzfs 2.2.3-1, same results
Having looked at this a bit closer:
Both cases you see the write inflation are when the write path goes through a loop device (mounting a filesystem from a file automagically sets one up). Looking at what losetup tells me about the loop devices created for mounting filesystems from files:
# losetup -l /dev/loop0
NAME SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE DIO LOG-SEC
/dev/loop0 0 0 0 512
# mount /tmp/testfile /mnt
# losetup -l /dev/loop0
NAME SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE DIO LOG-SEC
/dev/loop0 0 0 1 0 /tmp/testfile 0 512
(this is from mount /tmp/testfile /mnt) and if I interprete this correctly it tells us that the loop device uses SYNC I/O (DIO=0) and a block size of 512 bytes.
That could lead up to big writes being chopped up into smaller I/O that (as of them being SYNC) can not be coalesced by ZFS, which would explain the write amplification as of R/M/D cycles on data and/or metadata rewrites for the file.
As of your test with the qcow2 image not showing that write amplification I suspect that nbd uses a different write path that does not suffer from what losetup is.
I was playing with zfs and found something interesting: there is big difference in writing to zfs file directly VS writing into same file mounted as filesystem / block device.
Writing 2Gb directly leads to 2.8Gb real writes to disk, 40% more writes, this is OK.
But writing 2Gb into same file mounted as loop device leads to 4.2Gb of real writes to disk.
What kind of magic is working here ?
Same happens to zvols, same with files mounted as filesystem - 4.2Gb.
But Qcow2 is an EXCEPTION (because it does not use intermediate block device?), writes to Qcow2 are 2.8Gb too, not 4.2Gb (same when mounted inside VM or directly using qemu-nbd).
Openzfs 2.1.11-1
UPDATE: tested with openzfs 2.2.3-1, same results
Tests:
zpool create -d -o feature@async_destroy=enabled -o feature@empty_bpobj=enabled -o feature@lz4_compress=enabled -o ashift=12 -O compression=lz4 -O acltype=posixacl -O xattr=sa -O utf8only=on -O atime=off -O relatime=on -O recordsize=64k pool0 /dev/sda_lol #Nothing special here, just recordsize=64k for filesystems. zfs create pool0/test
The text was updated successfully, but these errors were encountered: