Documentation for the two photon series

[h-mayorquin]

In the documentation for the two photon series conerning data:

Binary data representing images across frames. If data are stored in an external file, this should be an empty 3D array.

Data Type: numeric
Dimensions: [[‘frame’, ‘x’, ‘y’], [‘frame’, ‘x’, ‘y’, ‘z’]]
Shape: [[None, None, None], [None, None, None, None]]
Name: data

https://nwb-schema.readthedocs.io/en/latest/format.html#twophotonseries

I have two questions:

1. Does the z value stand for the number of channels? I feel that it could be mistaken for depth which is a common connotation of the variable (alt-ought because we handle that in the schema with different imaging planes that's unlikely). Maybe we could clarify though?

2. The x and y stand for the rows and columns of the matrix, right? This is an ambiguity of imaging libraries that maybe we could avoid (there is a different convention in imaging processing and photography for x and y). To illustrate this, consider the following:

 ^
 |
 |
 Y
 |
 |
0/0---X--->

Where x could stand for columns. Other options instead of rows and columns are width and height which would refer to the image instead of the data array.

[CodyCBakerPhD]

1. Does the z value stand for the number of channels? I feel that it could be mistaken for depth which is a common connotation of the variable (alt-ought because we handle that in the schema with different imaging planes that's unlikely). Maybe we could clarify though?

Apparently this does actually stand for a z depth dimension, which can occur when multiple ImagingPlanes are so close together that they can be represented as a 3D volume. If the ImagingPlans are distinct, then you would have separate series for each plane.

Also, multiple color channels (a common mistake for a 4th dimension of a video-like series) would be added as separate PhotonSeries per color channel (usually only two), but those series would share ImagingPlanes.

2. The x and y stand for the rows and columns of the matrix, right? This is an ambiguity of imaging libraries that maybe we could avoid (there is a different convention in imaging processing and photography for x and y). To illustrate this, consider the following:

x and y stand for columns and rows of a the image matrix, hence requiring a transposition on writing to NWB. I agree this is ambiguous and might be nice to add clarification somewhere in the documentation. My personal preference is to refer to the dimensions of the image itself as width by height, since a user may not have any notion of the programmatic data representation and is merely used to viewing it with third-party tools.

[h-mayorquin]

Interesting. So, this is how it was intended.

Any source on this ? It would be good to leave a document, source or the statement of who you did discuss with to track provenance here on this issue.

[h-mayorquin]

Also, within the model, we have the following association:

TwoPhotonSeries -> ImagingPlane -> OpticalChannel (list of them).

So, if we have TwoPhotonSeries1 and TwoPhotonSeries2 that correspond to the two channels of the same series. How do we disambiguate which optical channel in the ImagingPlane belongs to which?

TwoPhotonSeries1 - > ImagingPlane -> OpticalChannel (which?)
TwoPhotonSeries2 -> ImagingPlane ->  OpticalChannel (which?)

[h-mayorquin]

So, the following lines is where the actual definition is written:

nwb-schema/core/nwb.ophys.yaml

Lines 19 to 24 in 761a0d7

	dims:
	- - width|height
	- - width|height|depth
	shape:
	- - 2
	- - 3