Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support .zarr testing #51

Open
yarikoptic opened this issue Apr 21, 2023 · 0 comments
Open

Support .zarr testing #51

yarikoptic opened this issue Apr 21, 2023 · 0 comments

Comments

@yarikoptic
Copy link
Member

ATM we have /mnt/backup/dandi/dandizarrs/tools/jsonschema-check-zattrs which produced http://www.oneukrainian.com/tmp/dandizarrs-jsonschema-checks.out . That script avoids using fuse since just needs to access a single file. But in the long run I hope that more thorough validation of zarrs would come about (ref: ome/ome-zarr-py#142 (comment) and there on).

But following complications I see immediately which would need to be considered/addressed to make it feasible to validate using current datalad-fuse based solution:

  • currently we do not have zarr subdatasets installed within dandisets/* dandisets (like 000108). They all just live in their original ../dandizarrs location. So we cannot just easily use dandisets-fuse/ location we have since those subdatasets/locations would be empty
  • That dandizarrs itself is not even a datalad dataset, so we cannot even use datalad-fuse on that location (Support plain directory (not super-dataset) with datasets under it datalad/datalad-fuse#35) . After that issue addressed I guess we could have implemented some adhoc jumping to another fused location upon getting into not installed submodule but it is "suboptimal" since assumes the correspondence of the git commit for submodule and repo under dandizars/ which most likely would be ok but not necessarily.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant