Move Scippneutron's elem_unit and elem_dtype to Scipp?

[nvaytet]

Can we move Scippneutron's elem_unit and elem_dtype to Scipp?
They are quite useful.

See also #3029

[SimonHeybrock]

I was actually wondering if unit and dtype should just resolve to the elem-unit and elem-dtype?

[jl-wynen]

Pretty sure the dtype shouldn't. Otherwise, code that checks dtypes also has to check for data.bins is None in many places.

[SimonHeybrock]

Pretty sure the dtype shouldn't. Otherwise, code that checks dtypes also has to check for data.bins is None in many places.

I can't think of many examples where we are doing a check for the exotic dtypes you get from binned data, can you? And for those that do, maybe a more dedicated method or property would be a viable alternative?

[jl-wynen]

My argument is the other way around. Using dtype == float would be true for binned float data.

[SimonHeybrock]

My argument is the other way around. Using dtype == float would be true for binned float data.

Isn't that what we want, in most situations?

But I suppose the question is: If binned data is not described by the dtype (and not by the shape), which property describes it? Is it a third property (the bins property?), or must it necessarily either be part of dtype or shape? A good summary was recently written by @jpivarski in scikit-hep/ragged#6 (see bullet points in the middle of the original post).

I think it is worth thinking about this again. While it is true that we make binned data part of the dtype, this is not necessarily done consistently, since we have found it more than inconvenient in several places (e.g., astype, and user code adding elem_dtype, triggering this discussion). The bins property avoids making it part of the shape, but one could argue that it mostly replaces a "dim of unknown length". Scipp currently uses the bins property to perform actions along the internal bin dimensions:

# da.sizes == {'x': 2, 'y':3}
da = da.bins.sum()

but one could just as well write

# da.sizes == {'x': 2, 'y':3, 'event':None}
da = da.sum('event')

Now, it is unfortunately not as easy as that, since we almost always use a DataArray as the content, i.e., we also have da.bins.coords['x'] in addition to da.coords['x'], so it is not clear if thinking about this purely in terms of the shape is adequate.