feat(web): custom infrastructure for @web/test-runner use 🏃

[jahorton]

Try as I might, I couldn't find any reporters for @web/test-runner designed for use with TeamCity. Fortunately, they've got pretty good documentation for their reporter API... so I decided to roll our own. As work went on and I remembered some issues we've had with browser-level automated tests in the past, I also added a couple more features to enhance our build logs and surface data that wasn't easily accessible otherwise.

Obviously, I worked on these after actually migrating many of our automated tests to @web/test-runner. That said... it'd be simpler and more time-efficient to review such PRs with the changes near-finalized out of the gate, so I've hoisted this earlier in the commit chain.

So, the new modules:

1. test-runner-TC-reporter
   • This formats test results in the format that TeamCity natively parses, allowing us to have nice reports on the tests in build reports. I've also added some checks here to add a small, human-readable summary after the TC-formatting section is complete on which test-files, platforms, etc had failures and why (when available), providing a central point of reference than requiring a search through the build logs to find a lead for test-related build failures.
   • If there were no errors, no "failure report" section appears.
2. test-runner-rename-browser
   • The BrowserLaunchers support a name field, but don't take custom names when initializing them - at least as far as I can see. Fortunately, they're mutable after construction, so the workaround is pretty simple.
3. test-runner-stability-reporter.js
   • After some research and code-inspection into our preferred browser-engine launching setup, I found a way to "spy" on calls used to manage browser-engine state in order to surface otherwise-silent errors that can result sometimes when running tests.
   • As I managed to find a scenario in which the test-runner failed against WebKit without providing any relevant information, I felt this important.

Also, rather than require pre-compilation... I just used JSDoc comments to achieve pseudo-TypeScript. That can be changed if desired easily enough. I'm also not sure this is the best place to leave the definitions, so I'd like to address concerns on code location and the JS/TS question at the same time. As it is, this implementation is enough to proceed with further work, so I decided to move on to questions with clearer answers.

See related CI build logs for #11404 (Test - Language Modeling Layer) to inspect and verify the output and usefulness of these reporters.

@keymanapp-test-bot skip

[keymanapp-test-bot]

User Test Results

Test specification and instructions

User tests are not required

Test Artifacts

• Android
  • Keyman for Android apk
  • FirstVoices for Android 16.0 apk
  • FirstVoices Keyboards for Android apk
  • KeyboardHarness apk
  • Keyman for Android 16.0 apk
  • KMSample1 apk
  • KMSample2 apk
• Developer
  • Keyman Developer
  • Compiler Regression Tests
  • kmcomp.zip
• iOS
  • Keyman for iOS (simulator image)
  • FirstVoices Keyboards for iOS (simulator image)
  • TestFlight internal PR build version - 18.0.30 (0.11403.10982)
• Keyboards
  • Test Keyboards
• Web
  • KeymanWeb Test Home
• Windows
  • Keyman for Windows
  • FirstVoices Keyboards for Windows
  • Text Editor (32 bit)
  • Text Editor (64 bit)

[jahorton]

I happened to notice that Web's child build scripts weren't outputting their headless-testing results into TC-friendly format even when run by CI. It was a simple fix, and it's kind of related anyway, so I threw the one-liner fix in here.

[jahorton]

From the latest build of #11404:

Here's confirmation that the custom TC-reporter functions as intended.

[mcdurdin]

LGTM, assuming that TODO does not need to be addressed first

[keyman-server]

Changes in this pull request will be available for download in Keyman version 18.0.41-alpha