Support testing coroutines that outlive the test

[dkhalanskyjb]

In general, coroutines should finish their execution at some point, as otherwise, they may stick around, wasting memory, CPU time, and possibly hogging some other resources. For this reason, when a singular piece of work is being tested, it's typically a mistake to have coroutines that don't finish by the end of the test. For this reason, we report unfinished coroutines as an error.

However, sometimes, there are coroutines that are expected to live throughout the lifetime of the program; for example, if the program is processing data from some source, like a sensor, there's no reason to ever stop the data flow from the sensor.

In order to test code that uses such long-running coroutines, it's required to somehow exempt them from being reported at the end.

For now, the workaround is to create a scope for the background tasks and cancel it at the end of the test:

@Test
fun testFoo() = runTest {
    // create a new scope that contains the test framework facilities,
    // but with an unrelated job
    val backgroundScope = CoroutineScope(coroutineContext + Job())
    try {
        // start a never-ending coroutine
        backgroundScope.launch {
            while (true) {
                yield()
            }
        }
        // the actual test here
    } finally {
        // cancel the background work
        backgroundScope.cancel()
    }
}

This is a significant pain point, judging by the activity in the issues regarding it: #1531, #3283, https://youtrack.jetbrains.com/issue/KT-52274, #3323.

Some options:

• If Customizable coroutine behaviour on scope termination #1065 gets implemented, then it will be possible to avoid doing anything else: it will suffice to mark the long-running coroutines for cancellation on scope termination. In production, their scope will just not be terminated, and in testing, on termination of the mocked scope, the long-running coroutines will be canceled. Pros: general, orthogonal. Cons: is not completely clean semantically, as cancellation happening in the test is not what happens in production.
• Communicate the expected usage patterns better. This will still be clunky, but at least not confusing. The path of least resistance.
• Some test-specific API. For example, TestScope could provide access to some child scope in which all tasks are ignored when checking for leaks. Such API can directly reflect the intended purpose, which is good, but there are issues when trying to introduce the concept of background work. In essence, we don't want to run the background work to completion when doing advanceUntilIdle, but in that case, finally blocks in background tasks are not guaranteed to be entered during the test. This needs careful thought.

Some proposed approaches that don't seem viable:

• Cancel children of TestScope by default. This is not how scopes typically work and violates the principle of least surprise.
• More generally, making checking for leaked coroutines optional. The majority of coroutines in a program perform a specific one-time task and need to finish, and even if some coroutines are long-running, the rest will still need to be checked for termination. Moreover, taking this approach prevents us from providing a more suitable flexible behavior in the future due to backward compatibility.