cjxl fails on "MotionPhoto" jpegs

[Quackdoc]

Describe the bug
MotionPhotos are a cursed file where a video appended to a jpeg file. cjxl will currently fail to transcode the image with lossless bitstream reconstruction data with the error "JPEG bitstream reconstruction data could not be created. Possibly there is too much tail data."

To Reproduce
cjxl filein.jpeg fileout.jxl
File provided

Expected behavior
A) CJXL could discard the data and throw a message to the user. While discarding this data isn't technically lossless as far as the entire image file is concerned. This extra data doesn't pertain to the actual jpeg itself needed for proper reconstruction of the "image". CJXL should likely by default exit with an "Error" to bail out of any scripts unless an extra flag is passed.

B) CJXL could append the same video file to the end of the JXL file in the same manor as a motionphoto. This would retain "true" losslessness of the entire file and not just the jpeg data.

Screenshots
Not Aplicable

Environment

• OS: Linux
• cjxl/djxl version string: cjxl v0.10.2 e148959 [AVX2,SSE4,SSE2]

Additional context
MotionPhotos are cursed, but there are a good number of them out in the wild and failing to transcode these with lossless reconstruction is a pain point for at least some users

Exiftool can identify multiple types of motionphotos "Motion Photo" is the most common, However some "Micro Videos" exist too. exiftool can find the appropriate offset for those using exiftool "$FILENAME" | grep "Micro Video Offset" | sed -E 's/^[^:]+: ([0-9]+)$/\1/'

Almost all motion photos seem to be mp4 based and have a ftypmp42 signature to them. Identification based on this seems reliable.

to extract the video from a MotionPhoto exiftool -b -EmbeddedVideoFile motionphoto.jpg > out.mp4
to extract the JPEG data from a motionphoto (replaces the current file, creates a new "original file") exiftool -b -trailer:all= motionphoto.jpeg

Example Image

[jonsneyers]

More generally it would be good if we have a better way to treat "semantically meaningful" tail data when transcoding JPEG files. Besides these "MotionPhoto" cases, there is also UltraHDR (which has a gain map in the tail data) and MPF in general (Multi Picture Format) like some examples I've seen where the main image has fake bokeh applied to the background but the original image with a sharp background is still there in the tail data; also just a concatenation of JPEG files (which is something e.g. ffmpeg can ingest) could be seen as an example of this.

Currently tail data is stored in the jbrd and just treated like any other "unknown" data, like unknown APP markers or padding bits: it simply gets Brotli-compressed and in that way it becomes "hidden" in the JXL file, in the sense that the only way to retrieve such info is to apply the JPEG bitstream reconstruction procedure first.

It would be cleaner if we would define some kind of recursive structure in the file format where there is an optional box, name TBD but for the purpose of discussion we could call it tail, which contains the "tail data" of the main image. Then if the tail data is a movie, it could simply be copied into such a tail box, but if the tail data is another JPEG bitstream, then JXL's JPEG bitstream recompression could be applied recursively to that tail JPEG, and the tail box of the main image would then contain another JXL file. This could be nested arbitrarily deep, e.g. if you would concatenate three JPEGs then the third one would be in the tail box of the JXL file corresponding to the second one, which would be in the tail box of the first one.

Maybe instead of tail it could be called next or something. Such a structure would not allow seeking (it's a linked list, not a random access data structure), but for sequential access it would be fine. It would also make it straightforward to strip the tail data while still allowing reconstruction of the main image.

This method could also be used to extend the JXL file format to allow representing sequences of images that are coded completely independently (i.e. not as frames within a single JXL codestream), which can be useful since multi-frame single-codestream JXL has the limitation that all frames need to have the same dimensions and colorspace.

I am turning this issue into a discussion since this is not just a libjxl issue but changing anything here would require extending the spec (ISO/IEC 18181-2 in this case).