Adjustable max compressed extent size

[ErrorNoInternet]

I wonder if it would be possible to allow the user to tweak the maximum size of a compressed extent (which is currently hardcoded to 128 KiB) on a per-file/subvolume basis? This would be useful to:

• increase compression ratios
• decrease fragmentation on large, write-once-read-many files
• possibly make LZ4 worthy again
• decrease write amplification on flash storage with lower compressed extent sizes

On ZFS this is possible with recordsize (per-dataset), on bcachefs with --encoded_extent_max (per filesystem; set during mkfs time, per-subvolume/file support is planned), and on F2FS with the compress_log_size mount option (or per-file with f2fs_io set_coption <algo> <4KiB * 2^this> <file>).

[kdave]

Yeah, it would be good. I used work on that in my early years, there's a project idea https://archive.kernel.org/oldwiki/btrfs.wiki.kernel.org/index.php/Project_ideas.html#Compression_enhancements and I maybe still have some of the code. This needs format change due to encoding of the flexible length.

With more CPU power we have nowadays the compression chunks could be larger which saves metadata and naturally enables better compression ratios, namely for ZSTD.

How it could be configured is an separate issue, but in general possible to do per-file/per-subvolume/per-fs.

[adam900710]

I believe the main blockage is just how to persistently save such config value.

We also have qgroup auto-reap waiting for such guide line, and to me they are really the same requirement, no on-disk format change at all, just a runtime behavior change.

So no matter if it's some PERSISTENT key change or superblock flags change, just let us conclude on a proper way to save such value.

[Forza-tng]

Would an xattr be suitable since we already use the xattr btrfs.compression? Maybe we could extend it and use btrfs.compression_extsize?

Today we can do

> setfattr -n btrfs.compression -v zstd file
> getfattr -n btrfs.compression file
# file: file
btrfs.compression="zstd"

It could be extended to

> setfattr -n btrfs.compression -v zstd file
> setfattr -n btrfs.compression_extsize -v 512 file
> getfattr -n btrfs.compression file
# file: file
btrfs.compression="zstd"

> getfattr -n btrfs.compression_extsize file
# file: file
btrfs.compression_extsize="512"

Of course we could use btrfs property set as well.