Add JPEG XL Open/Read support via libjxl #7848

olokelo · 2024-03-02T00:38:31Z

This PR enables opening and reading JPEG XL images and animations.
Supported image modes are: RGB, RGBA, RGBa, L, LA, La.
A relatively recent libjxl version is needed to compile Pillow with libjxl support.
The main changes are the addition of _jxl.c and JxlImagePlugin.py.
I'm also the author of jxlpy so this PR was influenced by the work of contributors there. This PR is also largely based on WebPImagePlugin.py which had similar implementation.

Why?
JPEG XL has recently seen increased adoption especially in Apple ecosystem. A lot of users are requesting Pillow support for JPEG XL as their products use Pillow and need to be able to handle jxl files.

I'm open to suggestions and comments. I understand such change would need a lot of testing and probably changes. After all Pillow would need to become somewhat dependent on libjxl. Creating documentation will not be a big problem however I decided to wait for feedback from Pillow core developers.

for more information, see https://pre-commit.ci

src/PIL/features.py

Tests/test_jxl_leaks.py

Tests/test_file_jxl_metadata.py

src/PIL/JxlImagePlugin.py

brunoais · 2024-03-02T14:36:33Z

May you please commit the images as LFS?

olokelo · 2024-03-02T18:43:02Z

May you please commit the images as LFS?

Do you mean those .jxl files under Tests/images I've committed? I'm not really sure how to do that.

brunoais · 2024-03-03T08:30:48Z

~~The explanation is outlined here: https://docs.github.com/en/repositories/working-with-files/managing-large-files/configuring-git-large-file-storage~~
Nevermind. I just noticed they are not putting the test images as lfs:

They already setup lfs and they only use for other larger stuff. Let it stay as you did.

Removed jxl feature

for more information, see https://pre-commit.ci

olokelo · 2024-03-11T12:04:24Z

Mac OS builds were failing because clang complained about goto labels being declared before variables in scope.
I also merged @radarhere's commits that remove the jxl feature which was causing troubles before.
Now it should be fine. Almost all checks pass except codecov.

radarhere · 2024-03-19T07:26:15Z

src/PIL/JxlImagePlugin.py

+        elif n_frames > 0:
+            self.n_frames = n_frames
+            self._tps_dur_secs = tps_num / tps_denom
+        # TODO: handle libjxl timecods


Suggested change

# TODO: handle libjxl timecods

# TODO: handle libjxl time codes

?

hugovk · 2024-03-19T08:53:37Z

src/PIL/__init__.py

@@ -47,6 +47,7 @@
    "IptcImagePlugin",
    "JpegImagePlugin",
    "Jpeg2KImagePlugin",
+    "JxlImagePlugin",


Naming things: like other plugins, shall we use the name of the format rather than the extension?

Suggested change

"JxlImagePlugin",

"JpegXlImagePlugin",

That's a good point. Not sure why I decided to go with "jxl" from the beginning.
Would you want me to change all the references to jxl (including tests, function/class names, feature.jxl) or only plugin and .c extension filename?

I think let's just use jxl for the filename extension (so the Tests/images/jxl/) dir is fine too) and of course the MIME type, and change the rest.

Tests/test_file_jxl_metadata.py

hugovk · 2024-03-19T08:55:57Z

Tests/test_jxl_leaks.py

+    # TODO: lower the limit, I'm not sure what is correct limit
+    # since I have libjxl debug system-wide
+    mem_limit = 16 * 1024  # kb


How can we find out the correct limit?

I tested it on release version of libjxl and I've set it to 6 megabytes which seem to work well, that's also similar to memory usage of djxl for decoding hopper.jxl.
I think libjxl memory usage could depend also on number of decoding threads used. By default we use all available cores.

Tests/test_jxl_leaks.py

src/_jxl.c

src/PIL/JxlImagePlugin.py

hugovk · 2024-03-19T09:13:03Z

src/PIL/JxlImagePlugin.py

+        # jpeg xl does some weird shenanigans when storing exif
+        # it omits first 6 bytes of tiff header but adds 4 byte offset instead
+        if len(exif) <= 4:
+            return None


Can we add a test case for this and _getexit?

hugovk · 2024-03-19T09:13:24Z

src/PIL/JxlImagePlugin.py

+
+    def seek(self, frame):
+        if not self._seek_check(frame):
+            return


And a test for this?

There's already test_file_jxl_animated.py test which also does seeking.

The return isn't covered: https://app.codecov.io/gh/python-pillow/Pillow/pull/7848/blob/src/PIL/JxlImagePlugin.py

hugovk · 2024-03-19T09:24:07Z

I've merged in main to include #7827 to fix a lot of the warnings I'm seeing on macOS 14.4 Sonoma with CommandLineTools installed.

And this is failing to build for me with:

      src/_jxl.c:174:12: error: incompatible integer to pointer conversion returning 'int' from a function with result type 'void *' [-Wint-conversion]
          return true;
                 ^~~~
      /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/include/stdbool.h:21:14: note: expanded from macro 'true'
      #define true 1
                   ^

Full log: log2.txt

olokelo · 2024-03-19T12:26:29Z

I've merged in main to include #7827 to fix a lot of the warnings I'm seeing on macOS 14.4 Sonoma with CommandLineTools installed.

And this is failing to build for me with:
      src/_jxl.c:174:12: error: incompatible integer to pointer conversion returning 'int' from a function with result type 'void *' [-Wint-conversion]
          return true;
                 ^~~~
      /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/15.0.0/include/stdbool.h:21:14: note: expanded from macro 'true'
      #define true 1
                   ^
Full log: log2.txt

Thanks for reporting. The _jxl_decoder_count_frames function was declared to return void * instead of bool. That was an obvious mistake and I'm even not sure why it worked under gcc.

src/PIL/JxlImagePlugin.py

hugovk · 2024-03-19T13:45:25Z

src/PIL/JxlImagePlugin.py

+    if is_jxl and not SUPPORTED:
+        return "image file could not be identified because JXL support not installed"


Exception instead of return string?

Should we raise SyntaxError like here in Jpeg2K plugin?
I based my work on WebP plugin which just returns string.

for more information, see https://pre-commit.ci

develar · 2024-04-23T19:11:27Z

It would be excellent to see some progress in this area. I am currently working on introducing JPEG XL support to https://immich.app for previews and thumbnails (as Python is used for ML). However, an additional plugin pillow-jpegxl-plugin complicates the deployment process.

bgorissen · 2024-04-24T16:05:11Z

Could you still add support for 16 bit single channel files? Such images are commonly used in fluorescent microscopy, and storing them in jxl instead of tiff or png would lead to substantial storage savings. Pillow is used by various relevant software including imageio and CellProfiler.

In _pil_jxl_get_mode you would have to add:

    if(bi->bits_per_sample == 16 && bi->num_color_channels == 1 && bi->alpha_bits == 0 && !bi->alpha_premultiplied) {
        return "I;16";
    }

and in _jxl_decoder_new, the line

if (decp->pixel_format.data_type != JXL_TYPE_UINT8) {

should be changed into:

if (decp->pixel_format.data_type != JXL_TYPE_UINT8 && decp->pixel_format.data_type != JXL_TYPE_UINT16) {

aclark4life · 2024-04-24T16:08:13Z

Any relation-to or overlap-with #1888 here? 🤔

bgorissen · 2024-04-24T16:40:46Z

Any relation-to or overlap-with #1888 here? 🤔

Not quite, fluorescent microscopy images are often stored in a single channel while #1888 is about multichannel files. Pillow already supports single channel images (as #1888 points out), so it makes sense that this JPEG XL reader supports them too.

olokelo · 2024-04-27T09:36:34Z

It would be excellent to see some progress in this area. I am currently working on introducing JPEG XL support to https://immich.app for previews and thumbnails (as Python is used for ML). However, an additional plugin pillow-jpegxl-plugin complicates the deployment process.

@develar Sounds awesome, thank you for working on JPEG XL support in Immich. I presume that merging this PR is now up to Pillow core developers since all checks have passed. However by now we would probably need to resolve some merge conflicts.

Could you still add support for 16 bit single channel files? Such images are commonly used in fluorescent microscopy, and storing them in jxl instead of tiff or png would lead to substantial storage savings.

@bgorissen Yes, of course it's possible. Thank you for detailed description of changes. Could you provide some image(s) to test this mode?

j99ca · 2024-05-07T17:43:01Z

@olokelo I 2nd @bgorissen request around 16 bit single channel support. I work with imaging systems that produce 16 bit single channel images, like thermal and depth, and the JXL format provides some significant cost savings storage-wise compared to our 16 bit single channel PNG format we are currently using. I would love to see the JXL ecosystem mature and this plugin would go a long way towards our adoption of the format

bgorissen · 2024-05-08T13:52:29Z

@olokelo I've attached a 16-bits grayscale jxl file (zipped because Github does not support jxl). You could crop it to create a smaller test file. The file was converted from a public tiff image from the dataset cpg0011 (Laber et al., 2023), available from the Cell Painting Gallery on the Registry of Open Data on AWS (https://registry.opendata.aws/cellpainting-gallery/), and released under CC0 1.0 Universal (which puts it in the public domain).

subcutaneous__D14__45adb24f-23b7-43f8-9612-0a7ecaa22ada__BR00101077__r02c03f01p01-ch1sk1fk1fl1.zip

olokelo and others added 2 commits March 2, 2024 00:56

add tests

8e0c5db

[pre-commit.ci] auto fixes from pre-commit.com hooks

a57ebea

for more information, see https://pre-commit.ci

radarhere reviewed Mar 2, 2024

View reviewed changes

src/PIL/features.py Outdated Show resolved Hide resolved

radarhere reviewed Mar 2, 2024

View reviewed changes

Tests/test_jxl_leaks.py Outdated Show resolved Hide resolved

radarhere reviewed Mar 2, 2024

View reviewed changes

Tests/test_file_jxl_metadata.py Outdated Show resolved Hide resolved

radarhere reviewed Mar 2, 2024

View reviewed changes

src/PIL/JxlImagePlugin.py Outdated Show resolved Hide resolved

minor fixes, linting corrections

23fb57d

olokelo mentioned this pull request Mar 2, 2024

JPEG XL support #4247

Open

radarhere added 2 commits March 6, 2024 20:54

Added type hints

2eb5987

Removed feature

37b58f3

radarhere mentioned this pull request Mar 6, 2024

Removed jxl feature olokelo/Pillow#1

Merged

olokelo and others added 3 commits March 11, 2024 10:48

Merge pull request #1 from radarhere/jxl-support2

eeaecb4

Removed jxl feature

fix goto labels for clang

24b63ad

[pre-commit.ci] auto fixes from pre-commit.com hooks

0b50410

for more information, see https://pre-commit.ci

radarhere added the JPEG label Mar 11, 2024

radarhere reviewed Mar 19, 2024

View reviewed changes

hugovk reviewed Mar 19, 2024

View reviewed changes

Merge branch 'main' into jxl-support2

f403672

olokelo added 2 commits March 19, 2024 13:00

modify leak test

f6086d4

fix _jxl_decoder_count_frames

5320450

hugovk reviewed Mar 19, 2024

View reviewed changes

olokelo added 3 commits March 19, 2024 19:22

minor plugin code tweaks

1b049ab

add type hints

6048520

rename jxl -> jpegxl

8fa280f

olokelo and others added 8 commits March 19, 2024 22:25

add test case for seeking to the same frame

58c37bf

flip cases in metadata test

48bbc2e

[pre-commit.ci] auto fixes from pre-commit.com hooks

443a352

for more information, see https://pre-commit.ci

fix some type hinting mistakes

62c58c2

[pre-commit.ci] auto fixes from pre-commit.com hooks

8cab1c1

for more information, see https://pre-commit.ci

change Optional to python 3.10+ syntax

e5003ff

add more metadata test cases

fa5bfac

[pre-commit.ci] auto fixes from pre-commit.com hooks

0b71605

for more information, see https://pre-commit.ci

develar mentioned this pull request Apr 24, 2024

feat(server): thumbnail and preview in JPEG XL format immich-app/immich#9056

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add JPEG XL Open/Read support via libjxl #7848

Add JPEG XL Open/Read support via libjxl #7848

olokelo commented Mar 2, 2024 •

edited by radarhere

brunoais commented Mar 2, 2024 •

edited

olokelo commented Mar 2, 2024

brunoais commented Mar 3, 2024

olokelo commented Mar 11, 2024

radarhere Mar 19, 2024

hugovk Mar 19, 2024

olokelo Mar 19, 2024

hugovk Mar 19, 2024

hugovk Mar 19, 2024

olokelo Mar 19, 2024

hugovk Mar 19, 2024

hugovk Mar 19, 2024

olokelo Mar 19, 2024

hugovk Mar 19, 2024

hugovk commented Mar 19, 2024

olokelo commented Mar 19, 2024

hugovk Mar 19, 2024

olokelo Mar 19, 2024 •

edited

develar commented Apr 23, 2024 •

edited

bgorissen commented Apr 24, 2024

aclark4life commented Apr 24, 2024

bgorissen commented Apr 24, 2024 •

edited

olokelo commented Apr 27, 2024

j99ca commented May 7, 2024

bgorissen commented May 8, 2024 •

edited

	# TODO: handle libjxl timecods
	# TODO: handle libjxl time codes

		if is_jxl and not SUPPORTED:
		return "image file could not be identified because JXL support not installed"

Add JPEG XL Open/Read support via libjxl #7848

Are you sure you want to change the base?

Add JPEG XL Open/Read support via libjxl #7848

Conversation

olokelo commented Mar 2, 2024 • edited by radarhere

brunoais commented Mar 2, 2024 • edited

olokelo commented Mar 2, 2024

brunoais commented Mar 3, 2024

olokelo commented Mar 11, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hugovk commented Mar 19, 2024

olokelo commented Mar 19, 2024

Choose a reason for hiding this comment

olokelo Mar 19, 2024 • edited

Choose a reason for hiding this comment

develar commented Apr 23, 2024 • edited

bgorissen commented Apr 24, 2024

aclark4life commented Apr 24, 2024

bgorissen commented Apr 24, 2024 • edited

olokelo commented Apr 27, 2024

j99ca commented May 7, 2024

bgorissen commented May 8, 2024 • edited

olokelo commented Mar 2, 2024 •

edited by radarhere

brunoais commented Mar 2, 2024 •

edited

olokelo Mar 19, 2024 •

edited

develar commented Apr 23, 2024 •

edited

bgorissen commented Apr 24, 2024 •

edited

bgorissen commented May 8, 2024 •

edited