Rawler processing of fuji images inconsistent

[dividebysandwich]

Hi,

I have integrated rawler into imagepipe (instead of rawloader) to make use of the better camera support in rawler. I am not using the raw development in rawler because imagepipe is a lot faster. This works great for most images for most cameras, but I ran into several issues with Fuji RAFs:

1. in raf.rs, in get_xtrans_cfa, the comment states that the values are stored in reverse order. I found this to not be true, and had to reverse R and B channels again to get proper color decoding for uncompressed 14LE images from an X-T5

2. I am experiencing incorrect colors for the X-T20 and X-T3, both compressed and uncompressed. The thing is, with rawloader the uncompressed ones had correct colors. I checked and they all use the same function 14le_unpacked, which is identical to the original version in rawloader, so I suspect it must be a step after the decode. Is there a known issue with that, or am I getting some side effect of integrating rawler into imagepipe?

If you want to check out what I did in imagepipe, please see pedrocr/imagepipe@5f9efa4

I have attached two files, one is compressed, one uncompressed, and they both exhibit the same color issue as the image is shifted to pink. See: https://lightningview.app/examples.zip

[cytrinox]

If I convert both files to DNG with dnglab and open them with darktable, everything looks fine. I shoot a lot with my X-T5 and never hit any problems.
When it comes to channel shifts in RAF files, in most cases I hit them the root cause was non-matching CFA info after cropping the image. Maybe you crop somewhere the image but don't shift the CFA?

@hanatos Maybe you can verify it, too - as vkdt is afaik the only GUI that uses rawler to directly render raw files.

[hanatos]

i can confirm these load fine through rawler and the extracted CFA is correct here. i did not understand the comment in 1. re R and B reversal, i just read the metadata as provided by rawler.

[dividebysandwich]

I am pretty sure there's some processing steps in dnglab itself after rawler that change things. What I am doing is to load the .RAF into a RawImage using rawler::decode_file(), and then process that through imagepipe. Since this works for Nikon, Sony, Canon, etc. raws, I know that this isn't fundamentally broken, especially since rawler is based on rawloader and still has a lot in common. What I did have to do is to re-add the exif rotation handling because that is inexplicably removed and replaced with a todo comment in rawimage.rs and a default of Orientation::Normal. I ended up doing that in imagepipe though.

I did check the CFA loading from the metadata and that appears to be working just fine. Forcing this to a manually defined CFA does not change anything.

@cytrinox Hmm it might be related to cropping and not properly shifting CFA, I will try to investigate that.

@hanatos: I don't really know what to tell you other than the sequence in rawloader works and the sequence in rawler is reversed, and the comment even states that "somehow" the sequence appears to be reverted. My changing R and B around fixed things for me.

This is the first image, which exhibits the color issue.

This is the second one, after fixing the RGB sequence:

This is the second image using the current dnglab master (again, note, I am talking about loading the image into a RawImage using rawler, not the whole dnglab chain or even the other processing steps in rawler):

Since both exif rotation and cropped_cfa() are essentially deleted, could it be that these steps are simply not done by rawler anymore, and that this is the cause of my problems when I use RawImage?

[hanatos]

i use `image.camera.cfa.color_at()` to find the colour of a pixel. i do all the rotation/cropping myself, if any.

…

On Wed, Mar 13, 2024 at 2:12 PM dividebysandwich ***@***.***> wrote: I am pretty sure there's some processing steps in dnglab itself after rawler that change things. What I am doing is to load the .RAF into a RawImage using rawler::decode_file(), and then process that through imagepipe. Since this works for Nikon, Sony, Canon, etc. raws, I know that this isn't fundamentally broken, especially since rawler is based on rawloader and still has a lot in common. What I did have to do is to re-add the exif rotation handling because that is inexplicably removed and replaced with a todo comment in rawimage.rs and a default of Orientation::Normal. I ended up doing that in imagepipe though. I did check the CFA loading from the metadata and that appears to be working just fine. Forcing this to a manually defined CFA does not change anything. @cytrinox <https://github.com/cytrinox> Hmm it might be related to cropping and not properly shifting CFA, I will try to investigate that. @hanatos <https://github.com/hanatos>: I don't really know what to tell you other than the sequence in rawloader works and the sequence in rawler is reversed, and the comment even states that "somehow" the sequence appears to be reverted. My changing R and B around fixed things for me. This is the first image, which exhibits the color issue. image.png (view on web) <https://github.com/dnglab/dnglab/assets/23048489/7ff42c2e-3190-4191-87c4-5ba42fc299c6> This is the second one, after fixing the RGB sequence: image.png (view on web) <https://github.com/dnglab/dnglab/assets/23048489/2b6f3c2d-db88-4044-86eb-e2f0430c1463> This is the second image using the current dnglab master (again, note, I am talking about loading the image into a RawImage using rawler, not the whole dnglab chain or even the other processing steps in rawler): image.png (view on web) <https://github.com/dnglab/dnglab/assets/23048489/84396c29-f42c-46ed-90d5-8b33496703c7> Since both exif rotation and cropped_cfa() are essentially deleted, could it be that these steps are simply not done by rawler anymore, and that this is the cause of my problems when I use RawImage? — Reply to this email directly, view it on GitHub <#415 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMAKKLMPNBUD3VN7TKBPADYYBGCXAVCNFSM6AAAAABESOWML6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOJUGM3TKNZYHA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[dividebysandwich]

Okay, after some more debugging I found the following:

1. The color issues are indeed caused by the crop x/y shift. For now I've just tried to always start at 0/0 which fixes the colors.
2. The fun thing is, now the colors on the X-T5 image (not the X-T20 image) have R and B reversed again, so now I have reverse the mapping back to what is in dnglab rawler main:
   0 => 'R',
   1 => 'G',
   2 => 'B',

So it seems to me that rawler is not doing the CFA shift for the rawimage, but it's performed somewhere down the line. I haven't had the time to trace everything through yet. Either way, it appears that using rawler outside of dnglab isn't really supported anymore since things have shifted around a lot. I would love to use JUST rawler/dnglab instead of shoehorning it into imagepipe, but unfortunately the performance is quite a lot worse.
For now I'm happy that the images load with correct colors, I'll investigate a proper solution for the crop for the RawImage later.