Generic raster image format recompression #3510
jonsneyers
started this conversation in
JPEG XL specification extensions
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
We have
jbrd
to represent JPEG bitstream reconstruction data that can be used to do bit-exact recompression of JPEG files based on combining the quantized DCT coefficients (and basic image metadata) contained in the JXL payload with the remaining bitstream details contained in thejbrd
box. This works mostly because the amount of remaining bitstream details is quite small since JPEG entropy coding has limited degrees of freedom.For PNG it would be harder to do something similar since DEFLATE entropy coding has a large amount of encoder freedom, which would necessitate a lot of "remaining bitstream details" to achieve bit-exact recompression, possibly too much to be effective.
However, there are lots of other image formats that effectively store basically uncompressed images or that use simple entropy coding that is deterministic or almost deterministic: PPM, BMP, TIFF, PSD, XCF, etc. For these, we could define a generic
ibrd
(image bitstream reconstruction data) box that works in a similar way asjbrd
but using the decoded sample values from the JXL payload instead of the quantized DCT coefficients.Basically what is needed for this is a simple description language that can be used to reconstruct a bitstream. The bitstream would be reconstructed using a sequence of operations of one of these forms:
Such a generic recompression method of course requires an encoder that can properly parse the input format and encode it in a correct way, but the spec only describes the decoder side and that can be agnostic of the actual syntax of the reconstructed format: all it has to do is Brotli decoding and outputting image buffers.
Complex formats like BMP, TIFF, PSD, XCF allow various ways to encode the actual image data, and we will likely not be able to reconstruct bitstreams in all those ways: e.g. TIFF has options for uncompressed, LZW, and DEFLATE ("ZIP"); uncompressed and LZW have deterministic encoding but DEFLATE doesn't; BMP is often just uncompressed but there are also BMP files which use palette colors (which jxl only has as an internal coding tool, not as an exposed image type) or that contain a JPEG or PNG payload. So when implementing an encoder for such formats, it will have to be taken into account that there will be cases that will succeed and cases that will fail. It could be still possible to do a form of non-bitexact recompression, using a different encoding method for the same data, that allows reconstructing a different BMP/TIFF/PSD/XCF file than the input file but with the same pixel data and also preserving all the metadata from the input file (only the pixel buffers themselves would be encoded differently).
Since several formats do allow DEFLATE compression and the implementations existing in the wild probably only use one specific DEFLATE encoder (e.g. libz in a specific configuration), it could be worthwhile to allow this as a buffer encoding scheme too, at the cost of introducing a dependency to libz (or doing a partial reimplementation of its encoder).
Obviously this is all a nontrivial project, and to make sure the specification of it is sufficiently generic, we would need representative sets of images in various formats we want to cover and an implementation of a recompressing encoder for these formats. The nice thing is that from a libjxl perspective, it does not require any new API, it can simply be added as a higher-level layer that internally uses the libjxl API to get decoded buffers and metadata boxes.
I would suggest that if there's an interest in this, we first start by developing a software implementation before finalizing the syntax for such an
ibrd
box and formalizing it officially in the spec. It could go either in 18181-2 (as a new Annex after the one that specifies JPEG bitstream reconstruction) or in a new part on its own.Beta Was this translation helpful? Give feedback.
All reactions