Generator tests run on the wrong instance, causing setUp to be ineffective and other issues

[sirosen]

This issue is a cleaned-up version of #80 , meant to capture the current context in nose2. It is an intentional duplicate to help make it easier to dive straight into this work, without getting tripped up on any of the older discussion.

There are several issues with respect to generator tests running on the wrong instance. Here's a clean reproduction:

$ cat test_gen.py
import unittest


class ExampleTest(unittest.TestCase):
    def setUp(self):
        print(f"setup on {id(self)}")

    def test_foo(self):
        def do_foo():
            print(f"run on {id(self)}")
        yield (do_foo,)
        yield (do_foo,)
        yield (do_foo,)

$ nose2 -v
test_foo:1
 (test_gen.ExampleTest) ... setup on 139890517293904
run on 139890517293328
ok
test_foo:2
 (test_gen.ExampleTest) ... setup on 139890517294720
run on 139890517293328
ok
test_foo:3
 (test_gen.ExampleTest) ... setup on 139890517294768
run on 139890517293328
ok

----------------------------------------------------------------------
Ran 3 tests in 0.001s

OK

The setup runs on different instances each time, which is supposed to happen. But then the test invocations happen on the same instance (and not one of the ones which got setup), which is not supposed to happen.

---

I'm going to try to make time to work on this after moving to python3-only, but I've also labelled it as 'help wanted'. If anyone wants to help try to understand and rework the generator test code, please feel free.

[sirosen]

A brief note on this for future readers:

I did some work on this last week and found that the fundamental problem is that in order for a test generator to run and produce a test (to add to the suite), the class it's attached to must be instantiated. That results in some confusing semantics because it needs to happen at test collection time.

My ideal fix would probably be to deprecate and remove generator tests in favor of parametrized tests. Ultimately, the fact that parametrization can happen at import-time means that it's a more sustainable strategy. However, that seems pretty harsh after so many years of the feature being there. Possibly this could be handled outside of collection time, via subtests... Needs more thought.