Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve perf for long WMO list search with ArgoIndex #334

Open
gmaze opened this issue Feb 23, 2024 · 1 comment
Open

Improve perf for long WMO list search with ArgoIndex #334

gmaze opened this issue Feb 23, 2024 · 1 comment
Labels
internals Internal machinery performance stale No activity over the last 90 days

Comments

@gmaze
Copy link
Member

gmaze commented Feb 23, 2024

The modular method for search in ArgoIndex pyarrow and pandas backends is not an optimal design for large requests.

For instance the search_wmo methods:

are based on a simple loop through the list of WMOs to search.
This could be optimised with apply methods for instance. But, in this cas could we keep the module design as it is ? (with a the construction of a binary mask that is then applied by the run method).

@gmaze gmaze added performance internals Internal machinery labels Feb 23, 2024
Copy link

This issue was marked as staled automatically because it has not seen any activity in 90 days

@github-actions github-actions bot added the stale No activity over the last 90 days label May 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
internals Internal machinery performance stale No activity over the last 90 days
Projects
None yet
Development

No branches or pull requests

1 participant