[BUG] load_forecastingdata with wind_4_seconds_dataset fails

[benHeid]

load_forecastingdata("wind_4_seconds_dataset", return_type="pd_multiindex_hier") fails with KeyError: '4_seconds'
To Reproduce

from sktime.datasets import load_forecastingdata
load_forecastingdata("solar_4_seconds_dataset", return_type="pd_multiindex_hier")

Expected behavior
Should return the time series instead of failing.

[fkiraly]

I can confirm this on windows, python 3.11, current main.

It seems to me the data source has changed, and is no longer adhering to the original specification?

This is the failure cause:

Who is maintaining the specification or the module currently?

My guess is, @achieveordie, @hazrulakmal, @yarnabrina? Perhaps @ciaran-g?

[fkiraly]

Plus, why did the tests not catch this - any ideas, @yarnabrina?

[yarnabrina]

We skip datasets folder in both new and old CI, if I remember correctly. Unless there's a change in that folder, those are not run in regular CI.

That being said, the CRON job did not fail so may be we have to check if this is covered or not.

https://github.com/sktime/sktime/actions/workflows/test_datasets.yml

[fkiraly]

Hm, looks like there is no test that would actually attempt downloads from forecastingdata? We should at least make spot checks.

[achieveordie]

I was not involved in this part of the codebase. It doesn't seem like there's any change in the dataset (the last change appears to be in 2020) and the previous code changes were made by @hazrulakmal almost a year ago.

I suspect that frequency='4_seconds' was never incorporated in the code or tests and was found by @benHeid when he manually called this dataset.

[achieveordie]

Hm, looks like there is no test that would actually attempt downloads from forecastingdata? We should at least make spot checks.

The closest test we have seems to be test_load_forecastingdata() from test_datadownload.py but that only checks for a "UnitTest" file.

There's also a test called test_check_link_downloadable() from the same file but it only checks whether the link is active. It is also marked as expected to fail in #5462.

[achieveordie]

The easiest solution is to add "4_seconds": "4s" in freq_map although I'm not sure if there are more such datasets that might warrant a more programmatic approach (dynamically creating freq_map based on a regex).