qemu/block
Jean-Louis Dupond 42a2890a76 qcow2: add discard-no-unref option
When we for example have a sparse qcow2 image and discard: unmap is enabled,
there can be a lot of fragmentation in the image after some time. Especially on VM's
that do a lot of writes/deletes.
This causes the qcow2 image to grow even over 110% of its virtual size,
because the free gaps in the image get too small to allocate new
continuous clusters. So it allocates new space at the end of the image.

Disabling discard is not an option, as discard is needed to keep the
incremental backup size as low as possible. Without discard, the
incremental backups would become large, as qemu thinks it's just dirty
blocks but it doesn't know the blocks are unneeded.
So we need to avoid fragmentation but also 'empty' the unneeded blocks in
the image to have a small incremental backup.

In addition, we also want to send the discards further down the stack, so
the underlying blocks are still discarded.

Therefor we introduce a new qcow2 option "discard-no-unref".
When setting this option to true, discards will no longer have the qcow2
driver relinquish cluster allocations. Other than that, the request is
handled as normal: All clusters in range are marked as zero, and, if
pass-discard-request is true, it is passed further down the stack.
The only difference is that the now-zero clusters are preallocated
instead of being unallocated.
This will avoid fragmentation on the qcow2 image.

Fixes: https://gitlab.com/qemu-project/qemu/-/issues/1621
Signed-off-by: Jean-Louis Dupond <jean-louis@dupond.be>
Message-Id: <20230605084523.34134-2-jean-louis@dupond.be>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
2023-06-05 13:15:42 +02:00
..
export aio: remove aio_disable_external() API 2023-05-30 17:37:26 +02:00
monitor block/monitor: Fix crash when executing HMP commit 2023-04-25 15:11:57 +02:00
accounting.c
aio_task.c
amend.c block: Mark BlockDriver callbacks for amend job GRAPH_RDLOCK 2023-05-10 14:16:54 +02:00
backup.c
blkdebug.c blkdebug: add missing coroutine_fn annotation 2023-04-25 13:17:28 +02:00
blkio.c block/blkio: use qemu_open() to support fd passing for virtio-blk 2023-06-01 11:08:21 -04:00
blklogwrites.c
blkreplay.c
blkverify.c block: Mark bdrv_recurse_can_replace() and callers GRAPH_RDLOCK 2023-05-10 14:16:54 +02:00
block-backend.c block: add blk_io_plug_call() API 2023-06-01 07:34:03 -04:00
block-copy.c
block-gen.h
block-ram-registrar.c
bochs.c
cloop.c
commit.c blockjob: Adhere to rate limit even when reentered early 2023-05-19 19:12:12 +02:00
copy-before-write.c copy-before-write: Fix open with child in iothread 2023-05-30 17:29:35 +02:00
copy-before-write.h
copy-on-read.c
copy-on-read.h
coroutines.h nbd: Mark nbd_co_do_establish_connection() and callers GRAPH_RDLOCK 2023-05-10 14:16:53 +02:00
create.c block: Call .bdrv_co_create(_opts) unlocked 2023-05-19 19:12:12 +02:00
crypto.c block: Call .bdrv_co_create(_opts) unlocked 2023-05-19 19:12:12 +02:00
crypto.h
curl.c aio: remove aio_disable_external() API 2023-05-30 17:37:26 +02:00
dirty-bitmap.c
dmg-bz2.c
dmg-lzfse.c
dmg.c
dmg.h
file-posix.c block/linux-aio: convert to blk_io_plug_call() API 2023-06-01 07:34:03 -04:00
file-win32.c thread-pool: avoid passing the pool parameter every time 2023-04-25 13:17:28 +02:00
filter-compress.c
gluster.c cutils: Adjust signature of parse_uint[_full] 2023-06-02 12:27:19 -05:00
graph-lock.c graph-lock: Disable locking for now 2023-05-19 19:16:53 +02:00
io.c block: Collapse padded I/O vecs exceeding IOV_MAX 2023-06-05 13:11:06 +02:00
io_uring.c block/io_uring: convert to blk_io_plug_call() API 2023-06-01 07:34:03 -04:00
iscsi-opts.c
iscsi.c aio: remove aio_disable_external() API 2023-05-30 17:37:26 +02:00
linux-aio.c block/linux-aio: convert to blk_io_plug_call() API 2023-06-01 07:34:03 -04:00
meson.build block: add blk_io_plug_call() API 2023-06-01 07:34:03 -04:00
mirror.c mirror: Hold main AioContext lock for calling bdrv_open_backing_file() 2023-05-30 17:21:23 +02:00
nbd.c nbd: Mark nbd_co_do_establish_connection() and callers GRAPH_RDLOCK 2023-05-10 14:16:53 +02:00
nfs.c cutils: Adjust signature of parse_uint[_full] 2023-06-02 12:27:19 -05:00
null.c
nvme.c block/nvme: convert to blk_io_plug_call() API 2023-06-01 07:34:03 -04:00
parallels-ext.c
parallels.c parallels: Incorrect condition in out-of-image check 2023-06-05 13:13:59 +02:00
parallels.h
plug.c block: add blk_io_plug_call() API 2023-06-01 07:34:03 -04:00
preallocate.c
progress_meter.c
qapi-sysemu.c block: Take main AioContext lock when calling bdrv_open() 2023-05-30 17:21:23 +02:00
qapi.c block: add accounting for zone append operation 2023-05-15 08:18:10 -04:00
qcow.c block: Call .bdrv_co_create(_opts) unlocked 2023-05-19 19:12:12 +02:00
qcow2-bitmap.c qcow2: mark various functions as coroutine_fn and GRAPH_RDLOCK 2023-04-25 13:17:28 +02:00
qcow2-cache.c
qcow2-cluster.c qcow2: add discard-no-unref option 2023-06-05 13:15:42 +02:00
qcow2-refcount.c qcow2: Don't call bdrv_getlength() in coroutine_fns 2023-05-10 14:16:53 +02:00
qcow2-snapshot.c qcow2: mark various functions as coroutine_fn and GRAPH_RDLOCK 2023-04-25 13:17:28 +02:00
qcow2-threads.c thread-pool: avoid passing the pool parameter every time 2023-04-25 13:17:28 +02:00
qcow2.c qcow2: add discard-no-unref option 2023-06-05 13:15:42 +02:00
qcow2.h qcow2: add discard-no-unref option 2023-06-05 13:15:42 +02:00
qed-check.c
qed-cluster.c
qed-l2-cache.c
qed-table.c
qed.c block: Call .bdrv_co_create(_opts) unlocked 2023-05-19 19:12:12 +02:00
qed.h
quorum.c block: Mark bdrv_recurse_can_replace() and callers GRAPH_RDLOCK 2023-05-10 14:16:54 +02:00
raw-format.c raw-format: Fix open with 'file' in iothread 2023-05-30 17:21:23 +02:00
rbd.c
replication.c
reqlist.c
snapshot-access.c
snapshot.c
ssh.c aio: remove aio_disable_external() API 2023-05-30 17:37:26 +02:00
stream.c blockjob: Adhere to rate limit even when reentered early 2023-05-19 19:12:12 +02:00
throttle-groups.c
throttle.c
trace-events block/io_uring: convert to blk_io_plug_call() API 2023-06-01 07:34:03 -04:00
trace.h
vdi.c block: Call .bdrv_co_create(_opts) unlocked 2023-05-19 19:12:12 +02:00
vhdx-endian.c
vhdx-log.c
vhdx.c block: Call .bdrv_co_create(_opts) unlocked 2023-05-19 19:12:12 +02:00
vhdx.h
vmdk.c block: Call .bdrv_co_create(_opts) unlocked 2023-05-19 19:12:12 +02:00
vpc.c block: Call .bdrv_co_create(_opts) unlocked 2023-05-19 19:12:12 +02:00
vvfat.c vvfat: mark various functions as coroutine_fn 2023-04-25 13:17:28 +02:00
win32-aio.c aio: remove aio_disable_external() API 2023-05-30 17:37:26 +02:00
write-threshold.c