Feature request: nested test case

[aherlihy]

Hi, I would like to make a feature request to allow for nested test cases, or at least some way to separate tests by namespace (like "context" or "describe" in JS testing frameworks like mocha). My problem is that I'm composing test cases programmatically so as of right now, only 1 level of nesting means I have to bunch a series of asserts into one test case when they can fail independently. These tests are naturally very hierarchical and I have relatively computationally expensive set-up for those series of test cases so I would prefer not to have to rerun the entire set up for each test.

As with Mocha, I would not expect to be able to mutate state defined in an outer test case from an inner test case - I would expect that to fail or to have undefined behavior.

If you believe I can accomplish what I want to do without nesting, then I'd be very appreciative of any suggestions! I will do my best to explain my use case in more detail below.

I'm testing a compiler and I have a set of directories representing a program to test + input state + an expected output. In the MUnit test class, I'm using a Fixture to iterate over the directories and construct a context for each directory. Then in the test itself, I loop over the queries that need to be run, but because nested isn't supported I can't have each query fail independently. However since the context is created in a fixture, I can't move test down a layer because you can't access fixures outside of "test" statements. Further, the compiler itself has various pluggable layers, so I would love a way to nest the tests per layer of the compiler. What I'm doing right now is forcing everything to be at the top level but naming the classes so they are "nested" by in the name, but that ends up with ridiculously long hierarchical test names which doesn't let me take advantage of the nice nesting UI of IntelliJ.

So instead of these lines I would love to be able to write (and nest further, but simplified for this example):

   directories.foreach(testdir => {
      test(testdir) {
        val g = graph()
        g.queries.map((hint, query) => {
          test(hint) { // these can fail independently
            assertTrue(query.solve())
          }
      }

It would be great to be able to have fixtures for each level, but I would be happy even with simply a nested test namespace alone.
Thank you!