Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pytest does not show addClassCleanup errors #11728

Closed
4 tasks done
LukasJerabek opened this issue Dec 20, 2023 · 2 comments · Fixed by #12250
Closed
4 tasks done

Pytest does not show addClassCleanup errors #11728

LukasJerabek opened this issue Dec 20, 2023 · 2 comments · Fixed by #12250
Labels
plugin: unittest related to the unittest integration builtin plugin

Comments

@LukasJerabek
Copy link

LukasJerabek commented Dec 20, 2023 •
edited

  • a detailed description of the bug or problem you are having
    Below in minimal example is a code that if I run with python, I get
Running test one
.E
======================================================================
ERROR: tearDownClass (__main__.SomeTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tst.py", line 11, in clean_up_class
    raise AttributeError
AttributeError

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (errors=1)

which is fine, but if I run it with pytest I get:

==================================================================================== test session starts =====================================================================================
platform linux -- Python 3.8.18, pytest-7.4.3, pluggy-1.3.0
rootdir: /home/vagrant
collected 1 item                                                                                                                                                                             

tst.py .                                                                                                                                                                               [100%]

===================================================================================== 1 passed in 0.01s ======================================================================================

Without any sign of that an error has happend. Therefore it fail silently without anyone knowing something wrong happend.

Expected: Pytest shows error as well.

  • output of pip list from the virtual environment you are using
Package        Version
-------------- -------
exceptiongroup 1.2.0
iniconfig      2.0.0
packaging      23.2
pip            23.3.2
pluggy         1.3.0
pytest         7.4.3
setuptools     56.0.0
tomli          2.0.1

  • pytest and operating system versions
    pytest 7.4.3
    Centos7

  • minimal example if possible

import unittest


class SomeTest(unittest.TestCase):
    @classmethod
    def setUpClass(cls):
        cls.addClassCleanup(cls.clean_up_class)
        
    @classmethod
    def clean_up_class(cls):
        raise AttributeError

    def test_one(self):
        print("Running test one")
        self.assertTrue(True)

if __name__ == '__main__':
    unittest.main()
@bluetech bluetech added the plugin: unittest related to the unittest integration builtin plugin label Jan 4, 2024
@LukasJerabek
Copy link
Author

@bluetech hi, may I ask how is this going? I think it is rather inconvenient not being able to see certain errors.

@millerdev
Copy link
Contributor

This workaround can be dropped in a plugin module or conftest.py

import pytest


@pytest.hookimpl
def pytest_configure():
    patch_unittest_TestCase_doClassCleanup()


def patch_unittest_TestCase_doClassCleanup():
    """Raise/print errors caught during class cleanup

    pytest ignores `TestCase.tearDown_exceptions`, which causes them to
    pass silently.
    """

    @classmethod
    def doClassCleanupAndRaiseLastError(cls):
        doClassCleanups()
        errors = cls.tearDown_exceptions
        if errors:
            if len(errors) > 1:
                num = len(errors)
                for n, (exc_type, exc, tb) in enumerate(errors[:-1], start=1):
                    print(f"\nclass cleanup error ({n} of {num}):", file=sys.stderr)
                    print_exception(exc_type, exc, tb)
            raise errors[-1][1]

    import sys
    from traceback import print_exception
    from unittest.case import TestCase
    doClassCleanups = TestCase.doClassCleanups
    TestCase.doClassCleanups = doClassCleanupAndRaiseLastError

@millerdev millerdev mentioned this issue Apr 26, 2024
Merged
nicoddemus added a commit that referenced this issue Apr 27, 2024
@millerdev @nicoddemus
unittest: report class cleanup exceptions (#12250)
7e7503c 
Fixes #11728

---------

Co-authored-by: Bruno Oliveira <bruno@soliv.dev>
@stanislavlevin stanislavlevin mentioned this issue May 6, 2024
Closed
stanislavlevin added a commit to stanislavlevin/django-rest-framework that referenced this issue May 6, 2024
@stanislavlevin
tests: Check urlpatterns after cleanups
b96af1f 
According to docs:
https://docs.python.org/3/library/unittest.html#unittest.TestCase.addClassCleanup

> Add a function to be called after tearDownClass() to cleanup resources
  used during the test class. Functions will be called in reverse order to
  the order they are added (LIFO).

This was revealed with recent change in pytest (`8.2.0`):
> pytest-dev/pytest#11728: For unittest-based tests, exceptions during
  class cleanup (as raised by functions registered with
  TestCase.addClassCleanup) are now reported instead of silently failing.

`check_urlpatterns` is called before `cleanup_url_patterns` and fails
(problem was hidden by `pytest < 8.2.0`).

`doClassCleanups` can be used instead to check after-cleanup state:

https://docs.python.org/3/library/unittest.html#unittest.TestCase.doClassCleanups

> This method is called unconditionally after tearDownClass(), or after
  setUpClass() if setUpClass() raises an exception.

  It is responsible for calling all the cleanup functions added by
  addClassCleanup(). If you need cleanup functions to be called prior to
  tearDownClass() then you can call doClassCleanups() yourself.

Fixes: encode#9399
Signed-off-by: Stanislav Levin <slev@altlinux.org>
@stanislavlevin stanislavlevin mentioned this issue May 6, 2024
Merged
auvipy pushed a commit to encode/django-rest-framework that referenced this issue May 7, 2024
@stanislavlevin
tests: Check urlpatterns after cleanups (#9400)
36d5c0e 
According to docs:
https://docs.python.org/3/library/unittest.html#unittest.TestCase.addClassCleanup

> Add a function to be called after tearDownClass() to cleanup resources
  used during the test class. Functions will be called in reverse order to
  the order they are added (LIFO).

This was revealed with recent change in pytest (`8.2.0`):
> pytest-dev/pytest#11728: For unittest-based tests, exceptions during
  class cleanup (as raised by functions registered with
  TestCase.addClassCleanup) are now reported instead of silently failing.

`check_urlpatterns` is called before `cleanup_url_patterns` and fails
(problem was hidden by `pytest < 8.2.0`).

`doClassCleanups` can be used instead to check after-cleanup state:

https://docs.python.org/3/library/unittest.html#unittest.TestCase.doClassCleanups

> This method is called unconditionally after tearDownClass(), or after
  setUpClass() if setUpClass() raises an exception.

  It is responsible for calling all the cleanup functions added by
  addClassCleanup(). If you need cleanup functions to be called prior to
  tearDownClass() then you can call doClassCleanups() yourself.

Fixes: #9399

Signed-off-by: Stanislav Levin <slev@altlinux.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
plugin: unittest related to the unittest integration builtin plugin
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Report class cleanup exceptions millerdev/pytest
3 participants
@millerdev @bluetech @LukasJerabek