Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update releases JSON format #51

Closed
marcwrobel opened this issue Dec 19, 2022 · 10 comments
Closed

Update releases JSON format #51

marcwrobel opened this issue Dec 19, 2022 · 10 comments
Labels
enhancement New feature or request

Comments

@marcwrobel
Copy link
Member

Currently the JSON format for releases in /releases is :

{
  "<version_1>: "<date_1>",
  "<version_2>: "<date_2>",
...
  "<version_n>: "<date_n>"
}

This format is not extensible : only a date can be provided. Using a format like the following one could make it extensible (e.g. open to new properties that could eventually used for more automation):

{
  "versions": [
    {
      "number": "<version_1>"
      "date": "<date_1>"
    },
    {
      "number": "<version_2>"
      "date": "<date_2>"
    },
...
    {
      "number": "<version_n>"
      "date": "<date_n>"
    }
  ]
}

This will make it possible to add more properties at product level and at version level.

I do not particularly have "extensions" ideas for the time being, but I have the feeling we should not wait to do that. The more we will add automation scripts, the more work it will be if one day we want to change the format.

@marcwrobel
Copy link
Member Author

This issue is linked to https://github.com/endoflife-date/releases.json, but that may be a bit difficult to implement such formats.

@marcwrobel
Copy link
Member Author

Having versions grouped under their cycle would be best if possible. I would make possible for instance to automate the retrieval of EOL date (for example for https://learn.microsoft.com/en-us/lifecycle/products/?terms=Azure%20DevOps).

@captn3m0
Copy link
Member

Been wanting to do this (See Step 1 on this discussion for eg), so big +1 for creating this.

I like the proposed schema, can be improved with additional fields (such as link for eg) over time.

One important extension is to support EOL and development dates. For example, for several manual scrapes, like:

  • COS
  • GKE
  • Palo Alto Pages
  • EKS
  • HAProxy
  • ROS

we scrape pages that already include the EOL data. This is data that can automatically be patched (if upstream changes) using latest.py.

-1 for release cycle grouping, since it can get quite confusing. We have releaseCycle labels that latest.py uses, and it still has to make some guesses.

I do not particularly have "extensions" ideas for the time being, but I have the feeling we should not wait to do that. The more we will add automation scripts, the more work it will be if one day we want to change the format.

+1

(Puts long-term hat on). I think of this as an ingredient dataset to our main repo. Ideally, this repo by itself will be helpful to other projects by itself as well. So ideally, this would closely match the releases.json format by itself, with any additional bits getting patched on top by the main repo. But a core-idea is for this to be a repo containing all versions, which we aren't accounting for in our releases.json schema (yet). So maybe something to patch there as well? We already have a supportedReleases key, having another releases key (same as versions in your proposal above) won't make much difference - it's gonna be optional anyway.

Since this would be a breaking change for latest.py, maybe we can patch latest.py first to support both old and new schema. That lets us incrementally change the automation here over a few separate PRs.

@marcwrobel
Copy link
Member Author

We could also automatically create/update purls using repology (endoflife-date/endoflife.date#1778 (comment)).

@captn3m0
Copy link
Member

Yes, that is definitely needed. Need to write the script for that. Unsure about keeping the generated PURLs separate, or overwriting the md files (like how we do with latest.py). Probably going with latter.

@marcwrobel
Copy link
Member Author

marcwrobel commented Dec 20, 2022

Yes, overwriting is better IMHO. Toy may need a new property for that though (repology_id ?).

@marcwrobel
Copy link
Member Author

Another improvement: we could also indicate the origin of the data (npmjs, maven, git...).

@captn3m0
Copy link
Member

We can provide a list of PURLs for the data-source. pkg:rubygems/bundler points to https://rubygems.org/gems/bundler, but is more standardized.

Another use-case for our frontend would be to directly link to relevant releases from the latest column.

@captn3m0
Copy link
Member

captn3m0 commented Jan 4, 2023

use-case for DotNet to track EOL: endoflife-date/endoflife.date#2234

marcwrobel added a commit that referenced this issue May 13, 2023
Create a common function to list and load product configurations for generic scripts. Generic scripts are scripts that handle multiple product based on some identifier (URL, coordinates...).

This make them more simple to read and maintain, while making the way they work much more consistent.

This is only the first step in that process, the end goal being:

- providing utilities to simplify the maintenance of existing script / development of new scripts (while keeping those script independent),
- being able to change globaly how things work (such as #51)
marcwrobel added a commit that referenced this issue May 13, 2023
Create a common function to list and load product configurations for generic scripts. Generic scripts are scripts that handle multiple product based on some identifier (URL, coordinates...).

This make them more simple to read and maintain, while making the way they work much more consistent.

This is only the first step in that process, the end goal being:

- providing utilities to simplify the maintenance of existing scripts / development of new scripts (while keeping those script independent),
- being able to change globaly how things work (such as #51)
marcwrobel added a commit that referenced this issue May 13, 2023
Create a common function to list and load product configurations for generic scripts. Generic scripts are scripts that handle multiple product based on some identifier (URL, coordinates...).

This make them more simple to read and maintain, while making the way they work much more consistent.

This is only the first step in that process, the end goal being:

- providing utilities to simplify the maintenance of existing scripts / development of new scripts (while keeping those script independent),
- being able to change globaly how things work (such as #51)
marcwrobel added a commit that referenced this issue May 20, 2023
Create a common function to list and load product configurations for generic scripts. Generic scripts are scripts that handle multiple product based on some identifier (URL, coordinates...).

This make them more simple to read and maintain, while making the way they work much more consistent.

This is only the first step in that process, the end goal being:

- providing utilities to simplify the maintenance of existing scripts / development of new scripts (while keeping those script independent),
- being able to change globaly how things work (such as #51)
@captn3m0 captn3m0 pinned this issue Jul 25, 2023
@marcwrobel
Copy link
Member Author

This should have been closed with #274. Release-level data will be supported after #297 is merger.

Other ideas, such as supporting product-level data (for PURLs) or data origin will be handled in separate issues if there is some need.

filiptronicek added a commit to gitpod-io/workspace-images that referenced this issue Feb 11, 2024
Since the end of life version data format changed (see endoflife-date/release-data#51), we need to adapt for our autofix to keep working.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants