Move dictionary to list releases outside app.py and use a json file instead

[shiv-tyagi]

This moves the dictionary we were using to list branches for builds outside of app.py. Major reasons for doing this are:

Configurations like these should not remain with the main application itself.
There is no point in versioning this information with git. This configuration keeps changing very regularly. Whenever we do releases, want to delist a branch, or make any other changes, this config changes.
We can update this file at regular intervals and reload the server to automatically sync with releases at firmware.ardupilot.org.
For this, I have made changes to the app to remove the dictionary from app.py and load it from remotes.json instead. We also validate this file against remotes.schema.json to ensure the structure of the file is correct and valid. I am working on a Python script as well to autoupdate this file on the server. I will be making a separate PR for this.

This also opens doors for allowing builds for different remotes in the future. The remotes.json file is structured in such a way that we could add entries for various releases/branches/commits/tags for a remote, as well as different remotes themselves. The logic to handle different remotes can be added in the future. This is basically the first step in that direction.

I have attempted to remove all references to branches in the application and renamed them to versions/releases, as there can be multiple releases on the same branch (e.g., copter stable-4.5.1 and stable-4.5.2), and a user should be able to choose whichever stable version they want to build.

After merging, this should go to the beta server first for thorough testing. We can then put this on the main server together with the next changes coming up.

[peterbarker]

I think nonimating all the releases here is too much overhead for the people doing releases.

Globbing tag names would be a better move,

A hybrid model of mixing those things into a static list should work.

[shiv-tyagi]

@peterbarker I think of something similar. Instead of globbing tag names, I prefer scraping the directories at firmware.ardupilot.org instead. Because, even if we figure out the release names through tag names, we have to find the features.txt for those releases at firmware.ardupilot.org.
Like if the release is 4.8.0 beta (which is the latest beta let's say), the artifacts are found under /beta directory on firmware server not /beta-4.8.0 or something. I recently saw that the artifacts for copter beta-4.3.8 were put under /beta-4.3 directory. This inconsistency makes it difficult to map tag names to artifacts directory.

But if we do the opposite, i.e., scrape firmware.ardupilot.org, then it is easy to figure out the version number or git hash from there because the artifacts directory already contains git-version.txt and other files.

And the algorithm of how do we poll the releases can be changed anytime in future. That's decoupled from the application. All it needs is this json file. We can generate that json file whatever way we want.

[shiv-tyagi]

I studied how we do the tags for releases and was finally able to write a script to do the releases updation using tag names. Thanks PeterB for the suggestion.

[shiv-tyagi]

Update: Also added a new endpoint /refresh_remotes to reload the remotes.json file on the go. To trigger this reload the POST request should contain a token in the json payload to make sure the requestor is authorised to trigger the refresh. This token is matched with the CBS_REMOTES_RELOAD_TOKEN environment variable.