You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
ATM each test and its results on all the different environments are co-located together in the data-*-js modules
This poses 2 problems:
prevents automated updates of the results when tests start passing/failing
prevents generating compat tables for a completely different set of environments
The later of these 2 points I've already have a (draft) PR for (#1791), but the flaw in that PR to allow passing in custom results is that the approach breaks down in linking the custom results to a test, if the name of a test would ever change. I'm not sure how often this happens, but with the current setup that would not be an issue, but once we'd allow providing external results, it would become problematic as it would cause the externally recorded/stored results to get disconnected from the tests in data-*.js.
As for the first point: as the results are stored in an object inside a module, unless you somehow modify the sourcecode of the module through code, there's no way afaics to automate the updating of the results, making it manual labour
So, I would propose the following:
rename the data-.js files to test-.js
externalize the results from these test-.js files to plain json files, using the results-.json naming convention
use the name of (sub)test to link the test (from test-.js) to the results in results-.js
optional: allow overriding the displayed testname in the generated HTMl by providing the optional displayName in addition to the name on a (sub)test in test-*.js
This achieves the following goals:
a testrunner could now be made to automatically modify the results-*.json file, as to allow to make updating the Compat Table less manual
forces the name of a (sub)test to not get changed without thinking about it
prepares the codebase for the next step, which is to allow providing alternative environments and results from the command line, which in turn allows creating custom compat tables
Eventually this repo could then even be spilt in a package to create Compat Tables in general and a repo for the kangas.github.io gh pages (as suggested by @ljharb in #1791)
Thoughts?
The text was updated successfully, but these errors were encountered:
ATM each test and its results on all the different environments are co-located together in the data-*-js modules
This poses 2 problems:
The later of these 2 points I've already have a (draft) PR for (#1791), but the flaw in that PR to allow passing in custom results is that the approach breaks down in linking the custom results to a test, if the name of a test would ever change. I'm not sure how often this happens, but with the current setup that would not be an issue, but once we'd allow providing external results, it would become problematic as it would cause the externally recorded/stored results to get disconnected from the tests in data-*.js.
As for the first point: as the results are stored in an object inside a module, unless you somehow modify the sourcecode of the module through code, there's no way afaics to automate the updating of the results, making it manual labour
So, I would propose the following:
displayName
in addition to thename
on a (sub)test in test-*.jsThis achieves the following goals:
Eventually this repo could then even be spilt in a package to create Compat Tables in general and a repo for the kangas.github.io gh pages (as suggested by @ljharb in #1791)
Thoughts?
The text was updated successfully, but these errors were encountered: