Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

datalad-registry-search CLI/API function #316

Open
yarikoptic opened this issue Feb 26, 2024 · 4 comments · May be fixed by #343
Open

datalad-registry-search CLI/API function #316

yarikoptic opened this issue Feb 26, 2024 · 4 comments · May be fixed by #343
Assignees
@candleindark

Comments

@yarikoptic
Copy link
Member

pending

and likely could be done in tandem with that.

Attn @fangq .
@candleindark
Copy link
Collaborator

@yarikoptic We have pagination enabled in the API. How do you want to handle the situation in which the search result contains a large number of URLs? Do you want the CLI just spit out all to them at once or in pages?

@yarikoptic
Copy link
Member Author

spit out as soon as page received, go to next page - spit out and so on.

@candleindark
Copy link
Collaborator

@yarikoptic I assume that you want a the output to the terminal to be a list of datasets as JSON representation similar to our web API, e.g.

[
    {
      "url": "https://github.com/OpenNeuroDatasets-JSONLD/ds001297.git",
      "id": 31126,
      "ds_id": "2e429bfe-8862-11e8-98ed-0242ac120010",
      "head_describe": "00001-4-g2c4c2f7",
      "annex_key_count": 0,
      "annexed_files_in_wt_count": 495,
      "annexed_files_in_wt_size": 10593648758,
      "last_update_dt": "2024-01-16T17:47:49.227045+00:00",
      "git_objects_kb": 1045,
      "processed": true,
      "last_chk_dt": "2024-03-20T09:56:22.797062+00:00"
    },
    {
      "url": "https://github.com/OpenNeuroDatasets-JSONLD/ds000233.git",
      "id": 30727,
      "ds_id": "aa38f5a2-89f6-11e8-abf3-0242ac120004",
      "head_describe": "1.0.1-4-g3a10cc7",
      "annex_key_count": 0,
      "annexed_files_in_wt_count": 362,
      "annexed_files_in_wt_size": 5090155130,
      "last_update_dt": "2024-01-16T17:41:23.926921+00:00",
      "git_objects_kb": 922,
      "processed": true,
      "last_chk_dt": "2024-03-20T09:25:21.326719+00:00"
    }
]

I can yield a per page result using get_status_dict such as https://github.com/datalad/datalad-extension-template/blob/793c5543b8f1385e007ceb2b8cd1db667b9d38e6/datalad_helloworld/hello_cmd.py#L62-L76. However, each yield will add some "boilerplate" surrounding the result in each page. Does Datalad provide any mechanism for an extension to yield outputs devoid of the "boilerplate" so that page results can be yield to the terminal without separations.

Screenshot 2024-03-20 at 4 11 28 PM

@yarikoptic
Copy link
Member Author

yes,

I believe it is custom_result_renderer -- see uses https://github.com/search?q=repo%3Adatalad%2Fdatalad%20custom_result_renderer&type=code

@candleindark candleindark linked a pull request Mar 22, 2024 that will close this issue
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@candleindark candleindark

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Search in CLI candleindark/datalad-registry
2 participants
@yarikoptic @candleindark