How to harvest Data Fair catalog?

[croesus]

I'm the product manager for https://opennetzero.org - a search engine for net-zero datasets. Our platform uses CKAN. We'd like to be able to harvest https://data.ademe.fr/ (which uses Data Fair) and add it to our index but can't find an appropriate endpoint. The documentation mentions harvesting via the API but doesn't give any details how to do this.

Is there a DCAT endpoint published by Data Fair? If not, how would you suggest we index the catalogue? Thanks!

[nicolas-bonnel]

Hi,

You can access documentation here : https://data.ademe.fr/openapi-viewer/?url=https://data.ademe.fr/data-fair/api/v1/api-docs.json&proxy=false , but documentation does not replace host and there are CORS errors.

The request for the catalog is : https://data.ademe.fr/data-fair/api/v1/datasets?size=20&page=1&owner=organization:g1pKfMqaE&publicationSites=data-fair-portals:efWMeL1ZP&visibility=public

You can increase size and get the whole catalog with one request.

The portal https://data.ademe.fr/ use the API, you can toggle the dev console in your browser and see HTTP requests when you browse the catalog.

[croesus]

Thank you! We'll investigate further.

Indexing a proprietary API, even a simple one, requires work specific to that platform and will delay it being included in our catalog. Please would you consider publishing a DCAT/DCAT-AP catalog as a feature of Data Fair? DCAT-AP has been adopted by the EU as the data catalog standard for public sector data portals and it makes it easier to build additional services (like ours) on top.

Thanks again for your response, and your work in helping organisations manage and publish their data.

[nicolas-bonnel]

Yes, publishing a DCAT catalog is on our roadmap.
We'd like to be compatible with France national data portal and it makes senses to be harvestable by CKAN.

However I don't have a date for this feature.