Extra channel types

[jonsneyers]

Part 1 (core codestream) defines the following extra channel types:

name	value	meaning
kAlpha	0	Alpha transparency, where 0 means fully transparent
kDepth	1	Depth map. Higher values mean farther away from the camera.
kSpotColour	2	Spot colour channel; red, green, blue indicate its colour and solidity in [0, 1] indicates the overall blending factor, with 0 corresponding to fully translucent (invisible) and 1 corresponding to fully opaque.
kSelectionMask	3	Selection mask, which indicates a (fuzzy) region of interest, for example for image manipulation purposes. Pixels with value zero do not belong to the selection, pixels with the maximum value do belong to the selection.
kBlack	4	The K channel of a CMYK image. If present, a CMYK ICC profile is also present, and the RGB samples are to be interpreted as CMY, where 0 denotes full ink.
kCFA	5	Channel used to represent Colour Filter Array data (Bayer mosaic)
kThermal	6	Infrared thermography image. Higher values mean warmer temperature.
kNonOptional	15	The decoder indicates it cannot safely interpret the semantics of this extra channel.
kOptional	16	Extra channel that can be safely ignored.

Meanwhile, other types of extra channels have been suggested. While we left room for identifiers 7-14 to extend the spec later if needed, I don't think it is wise to update Part 1 too often. We also defined the kNonOptional and kOptional identifiers as a generic way to add new extra channels, the idea being that the actual type of the extra channel would be stored as a string in the extra channel name (according to some kind of naming convention that could potentially also include parameters in the name string), while a naive decoder would know if it can simply ignore the contents of such a channel or not.

We can use this thread to discuss what kind of additional extra channel types would be useful enough (i.e. not extremely application-specific) to have a standardized naming convention for. When we have a list, we could eventually initiate a new part of the specification (say, 18181-5) to formalize it.

Also, some of the existing extra channel types are currently underspecified: in particular, kDepth, kCFA and kThermal only have a vague meaning and it could be useful to define more exactly how to interpret them, if needed with additional metadata that is encoded either via the extra channel name or via an image header extension.

Some of the potential types of extra channels that could be useful:

• The various kinds of material property maps used in textures for 3D rendering, such as normal maps, height maps, metalicness, smoothness, occlusion, emission, etc.
• The various spectral bands of multi-spectral imaging. We already have the main color image for visible light and kThermal that can be interpreted as part of the infrared, but there is more needed to cover what is used in satellite imagery and other kinds of scientific applications.
• The OGV channels of CMYKOGV.
• Medical imaging: the various types of data used in radiography etc.

It only makes sense to standardize such extra channel types if there's an actual need/desire for interoperability. Typically the approach currently used by applications that require such extra channels is to define their own container formats which store the relevant metadata, and then to use generic payload codecs to encode each channel separately as a grayscale image (or possibly encoding up to 3 or 4 channels at a time). That of course also works, but there are advantages to doing it the other way around and using the JXL file format as the outer container and putting any application-specific metadata in an application-specific ISOBMF box:

• You get a file that can be opened/previewed by any software that supports JXL; of course naive viewers will not be able to show all the information present in the image in a meaningful/useable way, but it's better than nothing.
• Compression-wise, if you have e.g. 7 channels that are to some extent correlated, it is (at least in principle) possible to get a better compression density if they can get encoded together in a single jxl codestream than if they are encoded separately.
• While there will always be a need to have specialized applications to properly interpret and use such 'exotic' extra channels, at least it becomes possible in principle to have a 'generic viewer' that simply allows you to view each channel as a grayscale (or color-coded heatmap) image, together with a description (that could be just the extra channel name). This is not possible if each application uses its own specific container which requires different parsers for each new application.

Anyway, the definition and adoption of such new extra channel types will mostly depend on the applications that actually use them, so please point them to this discussion thread.