Test failure

[fabaff]

For Nixpkgs we see an error for Python 3.11 and Python 3.12 with pytest 7.4.4 and mypy-plugins 3.0.0.

============================= test session starts ==============================
platform linux -- Python 3.11.7, pytest-7.4.4, pluggy-1.4.0
rootdir: /build/source
configfile: pyproject.toml
testpaths: test-data
plugins: mypy-plugins-3.0.0
collected 20 items                                                             

test-data/test-etree.yml ....F.............                              [ 90%]
test-data/test_objectify.yml ..

=================================== FAILURES ===================================
__________________________ etree_element_iterchildren __________________________
/build/source/test-data/test-etree.yml:37: 
E   pytest_mypy_plugins.utils.TypecheckAssertionError: Invalid output: 
E   Actual:
E     main:3: note: Revealed type is "def (tag: Union[Union[builtins.str, builtins.bytes, lxml.etree.QName], typing.Collection[Union[builtins.str, builtins.bytes, lxml.etree.QName, typing.Collection[...], ...]], Any, None] =, *tags: Union[builtins.str, builtins.bytes, lxml.etree.QName, typing.Collection[...], ...], *, reversed: builtins.bool =) -> typing.Iterator[lxml.etree._Element]" (diff)
E     ...
E   Expected:
E     main:3: note: Revealed type is "def (tag: Union[Union[builtins.str, builtins.bytes, lxml.etree.QName], typing.Collection[Union[builtins.str, builtins.bytes, lxml.etree.QName, typing.Collection[...], ...]], Any, None] =, *tags: Union[builtins.str, builtins.bytes, lxml.etree.QName, typing.Collection[...], ...], reversed: builtins.bool =) -> typing.Iterator[lxml.etree._Element]" (diff)
E     ...
E   Alignment of first line difference:
E     E: ...llection[...], ...], reversed: builtins.bool =) -> typing.Iterator[lx...
E     A: ...llection[...], ...], *, reversed: builtins.bool =) -> typing.Iterator...
E                                ^
=========================== short test summary info ============================
FAILED test-data/test-etree.yml::etree_element_iterchildren - 
======================== 1 failed, 19 passed in 21.93s =========================

[scoder]

The "actual" signature is correct, the "expected" one is probably just due to incomplete type interpretation of older tool versions. Not sure how to handle this. We might have to disable the test to avoid a dependency on the latest test tools. Or just require upgrading.

[Dreamsorcerer]

If the test is comparing the actual output strings from mypy, then surely the version of mypy needs to be pinned with the project. The error messages could change in any release.