Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update data fetcher gdac/erddap to retrieve BGC variables (oxygen) #202

Draft
wants to merge 54 commits into
base: master
Choose a base branch
from

Conversation

RaphaelBajon
Copy link

Trying to implement BGC Argo floats on argopy using the erddap server. The first goal of this request was to get oxygen variable. Yet it would be better if we could get all the variables of the BGC Argo floats (not only oxygen), loading them directly with argopy (really faster) once the new variables are implemented in the argopy source code.

@gmaze gmaze added enhancement New feature or request backends argo-BGC About biogeochemical variables labels Apr 5, 2022
@gmaze gmaze added this to the Go from alpha to beta milestone Apr 5, 2022
@gmaze
Copy link
Member

gmaze commented Apr 5, 2022

Thanks @RaphaelBajon for this work !
This should benefits a lot of people 👍🏻

…ew days with BGC manual, to know the most important variables using BGC floats). To be implemented bgc-standard/bgc-expert mode. An issue is pushed if the number of bgc requested variables is too high (server error).
@gmaze
Copy link
Member

gmaze commented Apr 5, 2022

About CI tests

Every time you push new commits, CI tests are triggered here: https://github.com/euroargodev/argopy/pull/202/checks
All tests must pass ✅
And you should make sure that new features are tested.

Fetching BGC data from the erddap is already coded in tests, but skipped.
All you have to do to include tests is to fix the name of the BGC data set id from the erddap server here:

DSEXISTS_bgc = erddap_ds_exists(ds="ArgoFloats-bio")

@gmaze gmaze changed the title Erddapdoxy Update data fetcher erddap to retrieve BGC variables (oxygen) Apr 6, 2022
@gmaze gmaze marked this pull request as draft April 6, 2022 06:53
@gmaze
Copy link
Member

gmaze commented Apr 12, 2022

I don't understand why CI tests with BGC dataset are successful with the standard user mode, only expert mode should possibly work 😨
There is possibly something wrong and misleading in the design of the tests, I'll check this asap

@RaphaelBajon
Copy link
Author

RaphaelBajon commented May 19, 2022

'PARAMATER_DATA_MODE' is not well implemented when requesting it from the erddap server. Each biological variables must have one mode 'A', 'R' or 'D' looking at the BGC documentation (for example, Section 4.2.3 or in the Argo Manual Section 2.6.5 PARAMETER DATA MODE)
We must wait from an update of the data on the erddap server before continuing implementing the different BGC variables.

RaphaelBajon and others added 2 commits May 30, 2022 16:00
to keep track of PARAMETER_DATA_MODE values, must add the dimension, converting profiles to points
@github-actions
Copy link

This pull request was marked as staled automatically because it has not seen any activity in 90 days

@github-actions github-actions bot added the stale No activity over the last 90 days label Aug 29, 2022
@gmaze gmaze removed the stale No activity over the last 90 days label Sep 23, 2022
@github-actions
Copy link

This pull request was marked as staled automatically because it has not seen any activity in 90 days

@github-actions github-actions bot added the stale No activity over the last 90 days label Dec 22, 2022
@SBS-EREHM
Copy link

SBS-EREHM commented Apr 5, 2023

Hello all. Status bgc support? No errors for dataset='phy'

argopy.set_options(src='gdac', dataset='bgc', mode='expert')
ArgoSet = ArgoDataFetcher().region(region_date)
df = ArgoSet.to_dataframe()

returns error in fetchers.py:556

ValueError: Errors happened with all URLs, this could be due to an internal impossibility to read returned content.

@RaphaelBajon
Copy link
Author

Hey @gmaze,
I have made the different updates on my branch and merge the remote branch. Only 1 old pytest from test_ref_region was failing. Now, I am not able to run the pytests since I have merged the remote changes (issue accessing the data I think).
We can discuss tomorrow if you have free time for the last commits and unit tests. Zoom will be great (available all day except 14h00-15h00).

pytest -ra -v -s --durations=10 test_fetchers_facade_data.py test_fetchers_data_erddap.py test_fetchers_data_gdac.py test_xarray.py.

@gmaze
Copy link
Member

gmaze commented May 9, 2023

Hi @RaphaelBajon
Awesome !
I can't today, what about Wed. 10th in the morning ?

@gmaze gmaze self-requested a review May 16, 2023 13:34
@gmaze gmaze marked this pull request as ready for review May 16, 2023 13:34
@github-actions
Copy link

This pull request was marked as staled automatically because it has not seen any activity in 90 days

@github-actions github-actions bot added the stale No activity over the last 90 days label Aug 16, 2023
@gmaze
Copy link
Member

gmaze commented Apr 12, 2024

Hi @RaphaelBajon !
I close this PR since it has been superseded by work done in https://argopy.readthedocs.io/en/latest/whats-new.html#v0-1-14-29-sep-2023
I'll keep the branch though, because some of your work is still relevant for more BGC support
Thanks a lot for your work, we'll find a way to acknowledge this in a future release !
gm

@gmaze gmaze closed this Apr 12, 2024
@gmaze gmaze reopened this Apr 12, 2024
@gmaze
Copy link
Member

gmaze commented Apr 12, 2024

let's keep it as a draft !

@gmaze gmaze marked this pull request as draft April 12, 2024 10:46
@gmaze gmaze removed their request for review April 12, 2024 10:46
@github-actions github-actions bot removed the stale No activity over the last 90 days label Apr 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
argo-BGC About biogeochemical variables backends enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants