JS nested tests v2

[aSemy]

A continuation of #2957 - full credits to @benkuly for the original PR!

This PR updates #2957 with the latest changes from master. I've tested this locally by publishing Kotest to MavenLocal and checking it in a project, and it seems to work.

Slack thread: https://kotlinlang.slack.com/archives/CT0G9SD7Z/p1709829959811199

Fix #3141

Details

The problem: I believe that JS Test Frameworks first run all contexts to discover them, and then execute the tests. This is in contrast to Kotest, which will execute tests as they are discovered, and this prevents JS Test Frameworks from discovering tests that are defined later.

Workaround

As a workaround Kotest will concatenate the contexts with an arrow:

Discussions

Discussions have mostly been taking place in #kotest in the Kotlin Slack. If you'd like to get involved, feel free to join us on Slack, or to leave a comment here on GitHub!

• https://slack-chats.kotlinlang.org/t/16648645/i-want-to-update-kaml-to-support-kotlin-js-but-kaml-uses-nes#00c9b227-a833-41f3-a3d8-1999db132ca9
• https://slack-chats.kotlinlang.org/t/16660941/https-github-com-kotest-kotest-pull-3913-js-nested-tests-upd#d6f48507-fd5d-4feb-8577-f414c6293eeb
• https://slack-chats.kotlinlang.org/t/16716623/are-there-any-reasons-why-nested-js-ist-just-ignored-for-exa#d39493cd-6725-432d-ab07-ec8739425235
• https://slack-chats.kotlinlang.org/t/16919751/u54cm9pgs-and-i-have-been-investigating-improving-kotest-sup#5c0d04d8-9a00-459c-bd1b-9403fb8c3c33

[OliverO2]

The engine changes in this PR would break #3805.

[OliverO2]

Not a review at this time, but results from experimenting with the code.

Thank you for getting it to this stage! For browser-based JS, results look good already. Also works partially on Node.js and Wasm, yet some challenges remain.

How I tested

• settings.gradle.kts: Added mavenLocal() to repositories.
• kotest-tests/kotest-tests-js/build.gradle.kts: used appropriate version numbers for mavenLocal publication:
  • id("io.kotest.multiplatform") version "5.9.0-LOCAL"
  • kotestCompilerPluginVersion.set("5.9.0-LOCAL")
• kotest-tests/kotest-tests-js/src: Renamed directory jsTest to jsHostedTest. (jsHosted is the common source set for JS and JS-hosted Wasm. In the future, we'll also see Kotlin supporting standalone Wasm runtimes without a JS API.)
• kotest-framework/kotest-framework-engine/src/commonMain/kotlin/io/kotest/engine/test/TestCaseExecutor.kt: Added @file:OptIn(ExperimentalStdlibApi::class).
• gradlew publishToMavenLocal
• gradlew -p kotest-tests/kotest-tests-js --continue jsBrowserTest jsNodeTest wasmJsBrowserTest wasmJsNodeTest

Observations

• jsBrowserTest: performs as expected
• jsNodeTest: performs partially
  • No test report due to Node JS tests do not report failures correctly #3329 (unrelated to this one, scheduled to be fixed soon).
  • Multiple occurrences of

    done() called multiple times

• wasmJsBrowserTest: performs partially
  • TimeoutTest, 4 occurrences:

    Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

• wasmJsNodeTest: performs partially
  • Some tests seem to be missing (e.g. NestedTests/level-1-context).
  • Reports a total of 48 tests (the other invocations report 52).

Suggestions before proceeding with a review

• Fix remaining issues.
• Make JS functionality CI-testable, e.g.:
• Consider integrating kotest-tests/kotest-tests-js into kotest-framework/kotest-framework-multiplatform-plugin-gradle/test-project
  • test-project can be auto-tested with the correct library and compiler plugin version via gradlew kotest-framework:kotest-framework-multiplatform-plugin-gradle:test.
  • test-project can also be used for manually configured local tests by temporarily adding the following properties to kotest-framework/kotest-framework-multiplatform-plugin-gradle/test-project/gradle.properties:

    org.gradle.jvmargs=-Xmx2048m -XX:MaxMetaspaceSize=512m
    
    kotlinVersion=1.9.23
    kotestVersion=5.9.0-LOCAL
    useNewNativeMemoryModel=true
    

• Add an apiDump for CI testing to succeed.