The Javascript test runners don't support promises at the outer layer.

Eg in mocha,

describe("foo") {  <---- does not support promises
   test("bar") { <--- supports promises
    }
}

Therefore we cannot use coroutines anywhere but the final scope. Since kotest supports coroutines at every layer, the only way to make it work in JS land is to support flat tests only. Unless you can find a JS test runner that supports promises at every level of nesting.

The alternative would be to change kotest to not support coroutines anywhere but the final scope, and that would be a) a huge breaking change and b) reduce the functionality for anyone in JVM land.

It looks like https://github.com/substack/tape might supported nested async tests, so we could explore that for Kotest 6.0

@sksamuel I can't find anything in Tape's documentation about nested async tests (that also have the top levels async). Would you mind pointing something to me? I am not against trying to make an implementation.

Also, I understand that it would indeed be a huge breaking change. However, I am not sure to understand the interest of having the top levels being async too.
The only thing I can see would be to use asynchronous retrieval from a database to load metadata to build test cases. Is that it?

Aside from whether it's a good idea or not to have async parent tests, changing it now would break thousands of installs.

In tape it looks like a top level test comes with a callback t which can then fire more tests https://github.com/substack/tape#ttestname-opts-cb

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Very much a hack, but I got this working which suits my needs, and should be easy to migrate over if/when nested JS test support is added: FlatSpec.kt

Obligatory disclaimer: use with caution. No guarantee it'll work in future releases, as using FunSpec in this way is almost certainly not supported. That being said though, feel free to use it, just leave the block comment for any future travelers who may be (mis)fortunate enough to stumble upon it :)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Maybe using kotlin.test could help with dynamic test registration on JS?

https://discuss.kotlinlang.org/t/kotlin-test-js-specifying-test-without-annotation/11280

piacenti Jan '19

I found a way to create the tests without the annotation and they work.
Very quick and dirty:

import kotlin.test.Test
import kotlin.test.assertTrue

@JsModule("kotlin-test")
@JsNonModule
external val kTest: dynamic

class TestJs {

  @Test
  fun blah() {
    kTest.kotlin.test.suite("package test", false) {
      kTest.kotlin.test.suite("suit test", false) {
        kTest.kotlin.test.test("test", false) {
          assertTrue(true)
        }
      }
    }
  }
}

But not perfect yet since karma seems to have the counts a bit messed up. I had 4 test before adding this code and then it went to 6 tests out of 4.

HeadlessChrome 72.0.3617 (Windows 10.0.0): Executed 0 of 4 SUCCESS (0 secs / 0 secs)
HeadlessChrome 72.0.3617 (Windows 10.0.0): Executed 1 of 4 SUCCESS (0 secs / 0.005 secs)
HeadlessChrome 72.0.3617 (Windows 10.0.0): Executed 2 of 4 SUCCESS (0 secs / 0.006 secs)
HeadlessChrome 72.0.3617 (Windows 10.0.0): Executed 3 of 4 SUCCESS (0 secs / 0.009 secs)
HeadlessChrome 72.0.3617 (Windows 10.0.0): Executed 4 of 4 SUCCESS (0 secs / 0.01 secs)
HeadlessChrome 72.0.3617 (Windows 10.0.0): Executed 5 of 4 SUCCESS (0 secs / 0.01 secs)
HeadlessChrome 72.0.3617 (Windows 10.0.0): Executed 6 of 4 SUCCESS (0 secs / 0.01 secs)
HeadlessChrome 72.0.3617 (Windows 10.0.0): Executed 6 of 4 SUCCESS (0.105 secs / 0.01 secs)

Making the test class an object gets rid of the additional duplicate tests generated by the code above

I've fleshed this out and created a working example on Kotlin 1.8.10, JS IR browser

import kotlin.test.Test
import kotlin.test.assertTrue

@JsModule("kotlin-test")
@JsNonModule
external val kTest: dynamic

class TestJs {

  // some dummy service that we'll test
  class UsernameValidator {
    fun isValid(username: String?) = username!!.isNotBlank()
  }

  @Test
  fun passwordValidationTests() {
    // create an outer context
    context("Given a password validator") {
      val usernameValidator = UsernameValidator()

      // nest some inner contexts + tests
      context("When username is very long") {
        val username = "very long username ".repeat(100)
        test("Then expect validation passes") {
          assertTrue { usernameValidator.isValid(username) }
        }
      }
      context("When username is not blank") {
        val username = "foo bar"
        test("Then expect validation passes") {
          assertTrue { usernameValidator.isValid(username) }
        }
      }

      // tests + contexts can be dynamically added
      mapOf(
        "blank" to "   ",
        "null" to null,
      ).forEach { (failureReason, username) ->
        context("When username is $failureReason") {
          test("Then expect validation fails") {
            assertTrue { usernameValidator.isValid(username) }
          }
        }
      }
    }
  }
}

/** Test context - for nesting tests */
fun context(name: String, ignored: Boolean = false, contextBlock: () -> Unit) {
  kTest.kotlin.test.suite(name, ignored) {
    contextBlock()
  }
}

/** Execute a test */
fun test(name: String, ignored: Boolean = false, testBlock: () -> Unit) {
  kTest.kotlin.test.test(name, ignored) {
    testBlock()
  }
}

Example result:

So the tests aren't nested, but kotlin.test will prepend the contexts to the test name.

Is there any plan to support this anytime soon? I'm porting a JVM library to KMP and running into this issue when adding a js target. I'd hate to move away from kotest and there are hundreds of tests that are affected.

It's going to be added when we can build on top of compiler plugins stably. I am hoping that will be after Kotlin 2.0 is released.