2.5.0: pytest is failing in 8 units because unexpected hash values #8099

kloczek · 2023-11-14T00:08:09Z

Initial Checks

I confirm that I'm using Pydantic V2

Description

Looks like 8 units are failing because unexpected hash values.

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

python3 -sBm build -w --no-isolation
because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
install .whl file in </install/prefix> using 'installer` module
run pytest with $PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>
build is performed in env which is cut off from access to the public network (pytest is executed with -m "not network")

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pydantic-2.5.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pydantic-2.5.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network' --ignore tests/test_docs.py
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.8.18, pytest-7.4.3, pluggy-1.3.0
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=True warmup_iterations=100000)
rootdir: /home/tkloczko/rpmbuild/BUILD/pydantic-2.5.0
configfile: pyproject.toml
testpaths: tests
plugins: hypothesis-6.84.3, benchmark-4.0.0, mock-3.12.0, Faker-20.0.0
collected 4260 items

tests/test_abc.py .s                                                                                                                                                                  [  0%]
tests/test_aliases.py ............................................                                                                                                                    [  1%]
tests/test_annotated.py .................s..........                                                                                                                                  [  1%]
tests/test_assert_in_validators.py .                                                                                                                                                  [  1%]
tests/test_callable.py ....                                                                                                                                                           [  1%]
tests/test_color.py ..........................................................................................                                                                        [  3%]
tests/test_computed_fields.py ................s...sss....x...                                                                                                                         [  4%]
tests/test_config.py ......................................sss........................                                                                                                [  6%]
tests/test_construction.py ............................................                                                                                                               [  7%]
tests/test_create_model.py .........................................                                                                                                                  [  8%]
tests/test_dataclasses.py ........................................s............................................ssssss.................s.........................................ssss. [ 11%]
.......................                                                                                                                                                               [ 12%]
tests/test_datetime.py .............................................................................................................................................................. [ 16%]
.................................                                                                                                                                                     [ 16%]
tests/test_decorators.py ......                                                                                                                                                       [ 17%]
tests/test_deprecated.py ..............................................                                                                                                               [ 18%]
tests/test_deprecated_validate_arguments.py .....................                                                                                                                     [ 18%]
tests/test_discriminated_union.py .....................sssss..............................                                                                                            [ 19%]
tests/test_edge_cases.py ................ss..............................................................................................s....................................ssss..s [ 23%]
ss............                                                                                                                                                                        [ 23%]
tests/test_errors.py ...                                                                                                                                                              [ 23%]
tests/test_exports.py .....................................................................................................................................                           [ 27%]
tests/test_fastapi_json_schema.py ..                                                                                                                                                  [ 27%]
tests/test_fields.py .s.....                                                                                                                                                          [ 27%]
tests/test_forward_ref.py .......................sss.s..............                                                                                                                  [ 28%]
tests/test_generics.py ..................................s.........s............................................sx.......sss...........                                               [ 30%]
tests/test_internal.py .........s.                                                                                                                                                    [ 31%]
tests/test_json.py ......................................................                                                                                                             [ 32%]
tests/test_json_schema.py ........................................................................................................................................................... [ 36%]
........................................................................................s................................................................x........................... [ 40%]
....................................                                                                                                                                                  [ 41%]
tests/test_main.py .................................................................................................sssssssssssssssssssssssss.....................s.................. [ 44%]
................................................................                                                                                                                      [ 46%]
tests/test_migration.py ............................................................................................................................................................. [ 50%]
.......................................................................                                                                                                               [ 51%]
tests/test_model_signature.py .............ss                                                                                                                                         [ 52%]
tests/test_model_validator.py ........                                                                                                                                                [ 52%]
tests/test_networks.py .........................................................................................................................................................x.... [ 56%]
....................................................................s.                                                                                                                [ 57%]
tests/test_networks_ipaddress.py .................................................................................................................................................... [ 61%]
...............                                                                                                                                                                       [ 61%]
tests/test_parse.py ...........                                                                                                                                                       [ 61%]
tests/test_pickle.py .....................                                                                                                                                            [ 62%]
tests/test_plugins.py ............                                                                                                                                                    [ 62%]
tests/test_private_attributes.py ........................                                                                                                                             [ 63%]
tests/test_rich_repr.py ..                                                                                                                                                            [ 63%]
tests/test_root_model.py .........................................................................                                                                                    [ 64%]
tests/test_serialize.py ........................................................................s...                                                                                  [ 66%]
tests/test_strict.py ......                                                                                                                                                           [ 66%]
tests/test_structural_pattern_matching.py s                                                                                                                                           [ 66%]
tests/test_tools.py .........                                                                                                                                                         [ 67%]
tests/test_type_adapter.py .................s..x........................                                                                                                              [ 68%]
tests/test_type_alias_type.py .......x...                                                                                                                                             [ 68%]
tests/test_types.py ................................................s................................................................................................................ [ 72%]
.....................................................................................................................x......ss....................................................... [ 76%]
..................................................................................................................................................................................... [ 80%]
..................................................................................................................................................................................... [ 84%]
......................................................                                                                                                                                [ 86%]
tests/test_types_namedtuple.py .........                                                                                                                                              [ 86%]
tests/test_types_payment_card_number.py ..............................................                                                                                                [ 87%]
tests/test_types_typeddict.py ..sss.sss.s.s.s.s.s.s.sss..sss.sss.s.sss..........s.s.s.s.s.s......                                                                                     [ 89%]
tests/test_typing.py ...s.......                                                                                                                                                      [ 89%]
tests/test_utils.py ..................sssssssssssss.s.............................................s.................................................................................. [ 93%]
...........                                                                                                                                                                           [ 93%]
tests/test_v1.py ..                                                                                                                                                                   [ 93%]
tests/test_validate_call.py ................s....................                                                                                                                     [ 94%]
tests/test_validators.py .................................................................................s.s........................................................................ [ 97%]
........                                                                                                                                                                              [ 98%]
tests/test_validators_dataclass.py ........                                                                                                                                           [ 98%]
tests/test_version.py ......                                                                                                                                                          [ 98%]
tests/test_warnings.py .....                                                                                                                                                          [ 98%]
tests/benchmarks/test_fastapi_startup_generics.py .                                                                                                                                   [ 98%]
tests/benchmarks/test_fastapi_startup_simple.py .                                                                                                                                     [ 98%]
tests/benchmarks/test_north_star.py EEEEEEEE                                                                                                                                          [ 98%]
tests/mypy/test_mypy.py sssssssssssssssssssssssssssssssssssssssssssssssssss                                                                                                           [ 99%]
tests/plugin/test_plugin.py s                                                                                                                                                         [100%]

========================================================================================== ERRORS ===========================================================================================
______________________________________________________________________ ERROR at setup of test_north_star_validate_json ______________________________________________________________________

    @pytest.fixture(scope='module')
    def north_star_data_bytes():
>       return _north_star_data_bytes()

tests/benchmarks/test_north_star.py:72:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def _north_star_data_bytes() -> bytes:
        from .generate_north_star_data import person_data

        needs_generating = not _NORTH_STAR_DATA_PATH.exists()
        if needs_generating:
            data = json.dumps(person_data(length=1000)).encode()
            _NORTH_STAR_DATA_PATH.write_bytes(data)
        else:
            data = _NORTH_STAR_DATA_PATH.read_bytes()

        # To make benchmarks a stable metric, validate the MD5 hash of the
        # existing generated data. If the data is deliberately changed,
        # update _EXPECTED_NORTH_STAR_DATA_MD5 above.
        #
        # NB updating Faker will almost certainly change the benchmark data.
        data_md5 = md5(data).hexdigest()
        if data_md5 != _EXPECTED_NORTH_STAR_DATA_MD5:
            if needs_generating:
>               raise ValueError(
                    f'Expected hash {_EXPECTED_NORTH_STAR_DATA_MD5} for north star data, but generated {data_md5}'
                )
E               ValueError: Expected hash 4d30ce33e301fd00c656f95e736c7785 for north star data, but generated 05eedf1b1cf50ab4687045cb32d36a37

tests/benchmarks/test_north_star.py:93: ValueError
__________________________________________________________________ ERROR at setup of test_north_star_validate_json_strict ___________________________________________________________________

    @pytest.fixture(scope='module')
    def north_star_data_bytes():
>       return _north_star_data_bytes()

tests/benchmarks/test_north_star.py:72:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def _north_star_data_bytes() -> bytes:
        from .generate_north_star_data import person_data

        needs_generating = not _NORTH_STAR_DATA_PATH.exists()
        if needs_generating:
            data = json.dumps(person_data(length=1000)).encode()
            _NORTH_STAR_DATA_PATH.write_bytes(data)
        else:
            data = _NORTH_STAR_DATA_PATH.read_bytes()

        # To make benchmarks a stable metric, validate the MD5 hash of the
        # existing generated data. If the data is deliberately changed,
        # update _EXPECTED_NORTH_STAR_DATA_MD5 above.
        #
        # NB updating Faker will almost certainly change the benchmark data.
        data_md5 = md5(data).hexdigest()
        if data_md5 != _EXPECTED_NORTH_STAR_DATA_MD5:
            if needs_generating:
>               raise ValueError(
                    f'Expected hash {_EXPECTED_NORTH_STAR_DATA_MD5} for north star data, but generated {data_md5}'
                )
E               ValueError: Expected hash 4d30ce33e301fd00c656f95e736c7785 for north star data, but generated 05eedf1b1cf50ab4687045cb32d36a37

tests/benchmarks/test_north_star.py:93: ValueError
________________________________________________________________________ ERROR at setup of test_north_star_dump_json ________________________________________________________________________

    @pytest.fixture(scope='module')
    def north_star_data_bytes():
>       return _north_star_data_bytes()

tests/benchmarks/test_north_star.py:72:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def _north_star_data_bytes() -> bytes:
        from .generate_north_star_data import person_data

        needs_generating = not _NORTH_STAR_DATA_PATH.exists()
        if needs_generating:
            data = json.dumps(person_data(length=1000)).encode()
            _NORTH_STAR_DATA_PATH.write_bytes(data)
        else:
            data = _NORTH_STAR_DATA_PATH.read_bytes()

        # To make benchmarks a stable metric, validate the MD5 hash of the
        # existing generated data. If the data is deliberately changed,
        # update _EXPECTED_NORTH_STAR_DATA_MD5 above.
        #
        # NB updating Faker will almost certainly change the benchmark data.
        data_md5 = md5(data).hexdigest()
        if data_md5 != _EXPECTED_NORTH_STAR_DATA_MD5:
            if needs_generating:
>               raise ValueError(
                    f'Expected hash {_EXPECTED_NORTH_STAR_DATA_MD5} for north star data, but generated {data_md5}'
                )
E               ValueError: Expected hash 4d30ce33e301fd00c656f95e736c7785 for north star data, but generated 05eedf1b1cf50ab4687045cb32d36a37

tests/benchmarks/test_north_star.py:93: ValueError
_____________________________________________________________________ ERROR at setup of test_north_star_validate_python _____________________________________________________________________

    @pytest.fixture(scope='module')
    def north_star_data_bytes():
>       return _north_star_data_bytes()

tests/benchmarks/test_north_star.py:72:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def _north_star_data_bytes() -> bytes:
        from .generate_north_star_data import person_data

        needs_generating = not _NORTH_STAR_DATA_PATH.exists()
        if needs_generating:
            data = json.dumps(person_data(length=1000)).encode()
            _NORTH_STAR_DATA_PATH.write_bytes(data)
        else:
            data = _NORTH_STAR_DATA_PATH.read_bytes()

        # To make benchmarks a stable metric, validate the MD5 hash of the
        # existing generated data. If the data is deliberately changed,
        # update _EXPECTED_NORTH_STAR_DATA_MD5 above.
        #
        # NB updating Faker will almost certainly change the benchmark data.
        data_md5 = md5(data).hexdigest()
        if data_md5 != _EXPECTED_NORTH_STAR_DATA_MD5:
            if needs_generating:
>               raise ValueError(
                    f'Expected hash {_EXPECTED_NORTH_STAR_DATA_MD5} for north star data, but generated {data_md5}'
                )
E               ValueError: Expected hash 4d30ce33e301fd00c656f95e736c7785 for north star data, but generated 05eedf1b1cf50ab4687045cb32d36a37

tests/benchmarks/test_north_star.py:93: ValueError
_________________________________________________________________ ERROR at setup of test_north_star_validate_python_strict __________________________________________________________________

    @pytest.fixture(scope='module')
    def north_star_data_bytes():
>       return _north_star_data_bytes()

tests/benchmarks/test_north_star.py:72:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def _north_star_data_bytes() -> bytes:
        from .generate_north_star_data import person_data

        needs_generating = not _NORTH_STAR_DATA_PATH.exists()
        if needs_generating:
            data = json.dumps(person_data(length=1000)).encode()
            _NORTH_STAR_DATA_PATH.write_bytes(data)
        else:
            data = _NORTH_STAR_DATA_PATH.read_bytes()

        # To make benchmarks a stable metric, validate the MD5 hash of the
        # existing generated data. If the data is deliberately changed,
        # update _EXPECTED_NORTH_STAR_DATA_MD5 above.
        #
        # NB updating Faker will almost certainly change the benchmark data.
        data_md5 = md5(data).hexdigest()
        if data_md5 != _EXPECTED_NORTH_STAR_DATA_MD5:
            if needs_generating:
>               raise ValueError(
                    f'Expected hash {_EXPECTED_NORTH_STAR_DATA_MD5} for north star data, but generated {data_md5}'
                )
E               ValueError: Expected hash 4d30ce33e301fd00c656f95e736c7785 for north star data, but generated 05eedf1b1cf50ab4687045cb32d36a37

tests/benchmarks/test_north_star.py:93: ValueError
_______________________________________________________________________ ERROR at setup of test_north_star_dump_python _______________________________________________________________________

    @pytest.fixture(scope='module')
    def north_star_data_bytes():
>       return _north_star_data_bytes()

tests/benchmarks/test_north_star.py:72:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def _north_star_data_bytes() -> bytes:
        from .generate_north_star_data import person_data

        needs_generating = not _NORTH_STAR_DATA_PATH.exists()
        if needs_generating:
            data = json.dumps(person_data(length=1000)).encode()
            _NORTH_STAR_DATA_PATH.write_bytes(data)
        else:
            data = _NORTH_STAR_DATA_PATH.read_bytes()

        # To make benchmarks a stable metric, validate the MD5 hash of the
        # existing generated data. If the data is deliberately changed,
        # update _EXPECTED_NORTH_STAR_DATA_MD5 above.
        #
        # NB updating Faker will almost certainly change the benchmark data.
        data_md5 = md5(data).hexdigest()
        if data_md5 != _EXPECTED_NORTH_STAR_DATA_MD5:
            if needs_generating:
>               raise ValueError(
                    f'Expected hash {_EXPECTED_NORTH_STAR_DATA_MD5} for north star data, but generated {data_md5}'
                )
E               ValueError: Expected hash 4d30ce33e301fd00c656f95e736c7785 for north star data, but generated 05eedf1b1cf50ab4687045cb32d36a37

tests/benchmarks/test_north_star.py:93: ValueError
_______________________________________________________________________ ERROR at setup of test_north_star_json_loads ________________________________________________________________________

    @pytest.fixture(scope='module')
    def north_star_data_bytes():
>       return _north_star_data_bytes()

tests/benchmarks/test_north_star.py:72:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def _north_star_data_bytes() -> bytes:
        from .generate_north_star_data import person_data

        needs_generating = not _NORTH_STAR_DATA_PATH.exists()
        if needs_generating:
            data = json.dumps(person_data(length=1000)).encode()
            _NORTH_STAR_DATA_PATH.write_bytes(data)
        else:
            data = _NORTH_STAR_DATA_PATH.read_bytes()

        # To make benchmarks a stable metric, validate the MD5 hash of the
        # existing generated data. If the data is deliberately changed,
        # update _EXPECTED_NORTH_STAR_DATA_MD5 above.
        #
        # NB updating Faker will almost certainly change the benchmark data.
        data_md5 = md5(data).hexdigest()
        if data_md5 != _EXPECTED_NORTH_STAR_DATA_MD5:
            if needs_generating:
>               raise ValueError(
                    f'Expected hash {_EXPECTED_NORTH_STAR_DATA_MD5} for north star data, but generated {data_md5}'
                )
E               ValueError: Expected hash 4d30ce33e301fd00c656f95e736c7785 for north star data, but generated 05eedf1b1cf50ab4687045cb32d36a37

tests/benchmarks/test_north_star.py:93: ValueError
_______________________________________________________________________ ERROR at setup of test_north_star_json_dumps ________________________________________________________________________

    @pytest.fixture(scope='module')
    def north_star_data_bytes():
>       return _north_star_data_bytes()

tests/benchmarks/test_north_star.py:72:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

    def _north_star_data_bytes() -> bytes:
        from .generate_north_star_data import person_data

        needs_generating = not _NORTH_STAR_DATA_PATH.exists()
        if needs_generating:
            data = json.dumps(person_data(length=1000)).encode()
            _NORTH_STAR_DATA_PATH.write_bytes(data)
        else:
            data = _NORTH_STAR_DATA_PATH.read_bytes()

        # To make benchmarks a stable metric, validate the MD5 hash of the
        # existing generated data. If the data is deliberately changed,
        # update _EXPECTED_NORTH_STAR_DATA_MD5 above.
        #
        # NB updating Faker will almost certainly change the benchmark data.
        data_md5 = md5(data).hexdigest()
        if data_md5 != _EXPECTED_NORTH_STAR_DATA_MD5:
            if needs_generating:
>               raise ValueError(
                    f'Expected hash {_EXPECTED_NORTH_STAR_DATA_MD5} for north star data, but generated {data_md5}'
                )
E               ValueError: Expected hash 4d30ce33e301fd00c656f95e736c7785 for north star data, but generated 05eedf1b1cf50ab4687045cb32d36a37

tests/benchmarks/test_north_star.py:93: ValueError
================================================================================== short test summary info ==================================================================================
SKIPPED [1] tests/test_abc.py:14: error value different on older versions
SKIPPED [1] tests/test_annotated.py:152: repr different on older versions
SKIPPED [1] tests/test_computed_fields.py:423: @computed_field @classmethod @property only works in 3.9+
SKIPPED [3] tests/test_computed_fields.py:516: error message is different on older versions
SKIPPED [1] tests/test_config.py:526: different on older versions
SKIPPED [1] tests/test_config.py:542: requires backport pre 3.11, fully tested in pydantic core
SKIPPED [1] tests/test_config.py:570: different on older versions
SKIPPED [1] tests/test_dataclasses.py:617: InitVar not supported in python 3.7
SKIPPED [1] tests/test_dataclasses.py:1635: kw_only is not available in python < 3.10
SKIPPED [1] tests/test_dataclasses.py:1648: kw_only is not available in python < 3.10
SKIPPED [4] tests/test_dataclasses.py:1691: kw_only is not available in python < 3.10
SKIPPED [1] tests/test_dataclasses.py:1968: stdlib dataclasses don't support Pydantic fields
SKIPPED [2] tests/test_dataclasses.py:2307: slots are only supported for dataclasses in Python >= 3.10
SKIPPED [2] tests/test_dataclasses.py:2327: slots are only supported for dataclasses in Python >= 3.10
SKIPPED [5] tests/test_discriminated_union.py:407: https://github.com/python/cpython/issues/103592
SKIPPED [2] tests/test_edge_cases.py:298: PEP585 generics only supported for python 3.9 and above
SKIPPED [1] tests/test_edge_cases.py:1510: difference in __name__ between versions
SKIPPED [3] tests/test_edge_cases.py:2300: error message different on older versions
SKIPPED [1] tests/test_edge_cases.py:2376: cannot use list.__class_getitem__ before 3.9
SKIPPED [3] tests/test_edge_cases.py:2477: cannot parametrize types before 3.9
SKIPPED [1] tests/test_fields.py:23: No error is thrown for `InitVar` for Python 3.8+
SKIPPED [1] tests/test_forward_ref.py:627: PEP585 generics only supported for python 3.9 and above
SKIPPED [1] tests/test_forward_ref.py:651: PEP585 generics only supported for python 3.9 and above
SKIPPED [1] tests/test_forward_ref.py:679: needs 3.9 or newer
SKIPPED [1] tests/test_forward_ref.py:714: needs 3.10 or newer
SKIPPED [1] tests/test_generics.py:782: repr different on older versions
SKIPPED [1] tests/test_generics.py:1035:
Depends on similar issue in CPython itself: https://github.com/python/cpython/issues/86483
Documented and skipped for possible fix later.
SKIPPED [1] tests/test_generics.py:2200: memray not enabled
SKIPPED [1] tests/test_generics.py:2496: requires python 3.11 or higher
SKIPPED [1] tests/test_generics.py:2532: requires python 3.11 or higher
SKIPPED [1] tests/test_generics.py:2563: Multiple inheritance with NamedTuple and the corresponding type annotations aren't supported before Python 3.11
SKIPPED [1] tests/test_internal.py:185: could not import 'devtools': No module named 'devtools'
SKIPPED [1] tests/test_json_schema.py:2247: ListEnum doesn't work in 3.8
SKIPPED [21] tests/test_main.py:1354: not implemented
SKIPPED [1] tests/test_main.py:1416: not implemented
SKIPPED [1] tests/test_main.py:1440: not implemented
SKIPPED [1] tests/test_main.py:1456: not implemented
SKIPPED [1] tests/test_main.py:1483: not implemented
SKIPPED [1] tests/test_main.py:1844: need 3.10 version
SKIPPED [1] tests/test_model_signature.py:172: repr different on older versions
SKIPPED [1] tests/test_model_signature.py:185: repr different on older versions
SKIPPED [1] tests/test_networks.py:822: email_validator is installed
SKIPPED [1] tests/test_serialize.py:1082: @computed_field @classmethod @property only works in 3.9+
SKIPPED [1] tests/test_structural_pattern_matching.py:6: requires python 3.10 or higher
SKIPPED [1] tests/test_type_adapter.py:83: ForwardRef doesn't accept module as a parameter in Python < 3.9
SKIPPED [1] tests/test_types.py:871: different output
SKIPPED [1] tests/test_types.py:1847: email_validator is installed
SKIPPED [1] tests/test_types.py:1855: email_validator is installed
SKIPPED [2] tests/test_types_typeddict.py:89: typing.TypedDict does not support all pydantic features in Python < 3.12
SKIPPED [1] tests/test_types_typeddict.py:89: Required and NotRequired are not available from <module 'typing' from '/usr/lib64/python3.8/typing.py'>
SKIPPED [2] tests/test_types_typeddict.py:109: typing.TypedDict does not support all pydantic features in Python < 3.12
SKIPPED [1] tests/test_types_typeddict.py:109: Required and NotRequired are not available from <module 'typing' from '/usr/lib64/python3.8/typing.py'>
SKIPPED [1] tests/test_types_typeddict.py:125: typing.TypedDict does not support all pydantic features in Python < 3.12
SKIPPED [1] tests/test_types_typeddict.py:145: typing.TypedDict does not support all pydantic features in Python < 3.12
SKIPPED [1] tests/test_types_typeddict.py:170: typing.TypedDict does not support all pydantic features in Python < 3.12
SKIPPED [1] tests/test_types_typeddict.py:184: typing.TypedDict does not support all pydantic features in Python < 3.12
SKIPPED [1] tests/test_types_typeddict.py:213: typing.TypedDict does not support all pydantic features in Python < 3.12
SKIPPED [1] tests/test_types_typeddict.py:305: typing.TypedDict does not support all pydantic features in Python < 3.12
SKIPPED [2] tests/test_types_typeddict.py:316: typing.TypedDict does not support all pydantic features in Python < 3.12
SKIPPED [1] tests/test_types_typeddict.py:316: Required and NotRequired are not available from <module 'typing' from '/usr/lib64/python3.8/typing.py'>
SKIPPED [2] tests/test_types_typeddict.py:377: typing.TypedDict does not support all pydantic features in Python < 3.12
SKIPPED [1] tests/test_types_typeddict.py:377: Required and NotRequired are not available from <module 'typing' from '/usr/lib64/python3.8/typing.py'>
SKIPPED [2] tests/test_types_typeddict.py:406: typing.TypedDict does not support all pydantic features in Python < 3.12
SKIPPED [1] tests/test_types_typeddict.py:406: Required and NotRequired are not available from <module 'typing' from '/usr/lib64/python3.8/typing.py'>
SKIPPED [1] tests/test_types_typeddict.py:441: typing.TypedDict does not support all pydantic features in Python < 3.12
SKIPPED [3] tests/test_types_typeddict.py:470: typing.TypedDict does not support all pydantic features in Python < 3.12
SKIPPED [1] tests/test_types_typeddict.py:714: typing.TypedDict does not support all pydantic features in Python < 3.12
SKIPPED [1] tests/test_types_typeddict.py:738: typing.TypedDict does not support all pydantic features in Python < 3.12
SKIPPED [1] tests/test_types_typeddict.py:749: typing.TypedDict does not support all pydantic features in Python < 3.12
SKIPPED [1] tests/test_types_typeddict.py:803: typing.TypedDict does not support all pydantic features in Python < 3.12
SKIPPED [1] tests/test_types_typeddict.py:822: typing.TypedDict does not support all pydantic features in Python < 3.12
SKIPPED [1] tests/test_types_typeddict.py:846: typing.TypedDict does not support all pydantic features in Python < 3.12
SKIPPED [1] tests/test_typing.py:73: not supported in this python version
SKIPPED [13] tests/test_utils.py:87: requires python 3.10 or higher
SKIPPED [1] tests/test_utils.py:118: generic aliases are not available in python < 3.9
SKIPPED [1] tests/test_utils.py:309: devtools not installed
SKIPPED [1] tests/test_validate_call.py:379: testing >= 3.9 behaviour only
SKIPPED [1] tests/test_validators.py:1605: https://github.com/python/cpython/issues/103592
SKIPPED [1] tests/test_validators.py:1646: https://github.com/python/cpython/issues/103592
SKIPPED [40] tests/mypy/test_mypy.py:158: Test only with "--test-mypy" flag
SKIPPED [2] tests/mypy/test_mypy.py:158: Need to handle some more things for mypy >=1.1.1
SKIPPED [1] tests/mypy/test_mypy.py:208: Test only with "--test-mypy" flag
SKIPPED [3] tests/mypy/test_mypy.py:231: Test only with "--test-mypy" flag
SKIPPED [1] tests/mypy/test_mypy.py:241: Test only with "--test-mypy" flag
SKIPPED [1] tests/mypy/test_mypy.py:253: Test only with "--test-mypy" flag
SKIPPED [3] tests/mypy/test_mypy.py:260: Test only with "--test-mypy" flag
SKIPPED [1] tests/plugin/test_plugin.py:8: Test only with `TEST_PLUGIN` env var set.
XFAIL tests/test_computed_fields.py::test_multiple_references_to_schema[make_typed_dict] - computed fields do not work with TypedDict yet. See https://github.com/pydantic/pydantic-core/issues/657
XFAIL tests/test_generics.py::test_generic_model_as_parameter_to_generic_type_alias - Generic models are not type aliases
XFAIL tests/test_json_schema.py::test_get_pydantic_core_schema_calls - We are calling __get_pydantic_json_schema__ too many times. The second time we analyze a model we get the CoreSchema from __pydantic_core_schema__. But then we proceed to append to the metadata json schema functions.
XFAIL tests/test_networks.py::test_mongodsn_default_ports[mongodb+srv://user:pass@localhost/app-mongodb+srv://user:pass@localhost/app] - This case is not supported. Check https://github.com/pydantic/pydantic/pull/7116 for more details.
XFAIL tests/test_type_adapter.py::test_validate_json_strict - Need to fix this in https://github.com/pydantic/pydantic/pull/5944
XFAIL tests/test_type_alias_type.py::test_field - description is currently dropped
XFAIL tests/test_types.py::test_invalid_schema_constraints[kwargs0-int] - int cannot be used with pattern but we do not currently validate that at schema build time
ERROR tests/benchmarks/test_north_star.py::test_north_star_validate_json - ValueError: Expected hash 4d30ce33e301fd00c656f95e736c7785 for north star data, but generated 05eedf1b1cf50ab4687045cb32d36a37
ERROR tests/benchmarks/test_north_star.py::test_north_star_validate_json_strict - ValueError: Expected hash 4d30ce33e301fd00c656f95e736c7785 for north star data, but generated 05eedf1b1cf50ab4687045cb32d36a37
ERROR tests/benchmarks/test_north_star.py::test_north_star_dump_json - ValueError: Expected hash 4d30ce33e301fd00c656f95e736c7785 for north star data, but generated 05eedf1b1cf50ab4687045cb32d36a37
ERROR tests/benchmarks/test_north_star.py::test_north_star_validate_python - ValueError: Expected hash 4d30ce33e301fd00c656f95e736c7785 for north star data, but generated 05eedf1b1cf50ab4687045cb32d36a37
ERROR tests/benchmarks/test_north_star.py::test_north_star_validate_python_strict - ValueError: Expected hash 4d30ce33e301fd00c656f95e736c7785 for north star data, but generated 05eedf1b1cf50ab4687045cb32d36a37
ERROR tests/benchmarks/test_north_star.py::test_north_star_dump_python - ValueError: Expected hash 4d30ce33e301fd00c656f95e736c7785 for north star data, but generated 05eedf1b1cf50ab4687045cb32d36a37
ERROR tests/benchmarks/test_north_star.py::test_north_star_json_loads - ValueError: Expected hash 4d30ce33e301fd00c656f95e736c7785 for north star data, but generated 05eedf1b1cf50ab4687045cb32d36a37
ERROR tests/benchmarks/test_north_star.py::test_north_star_json_dumps - ValueError: Expected hash 4d30ce33e301fd00c656f95e736c7785 for north star data, but generated 05eedf1b1cf50ab4687045cb32d36a37
================================================================== 4059 passed, 186 skipped, 7 xfailed, 8 errors in 22.01s ==================================================================

Python, Pydantic & OS Version

Here is list of installed modules in build env

Package                 Version
----------------------- ----------
annotated-types         0.6.0
attrs                   23.1.0
build                   1.0.3
cloudpickle             3.0.0
cppclean                0.13
dirty-equals            0.7.0
distro                  1.8.0
dnf                     4.18.1
dnspython               2.4.2
editables               0.5
email-validator         2.1.0
exceptiongroup          1.1.3
Faker                   20.0.0
gpg                     1.23.0
hatch-fancy-pypi-readme 23.1.0
hatchling               1.18.0
hypothesis              6.84.3
idna                    3.4
importlib-metadata      6.8.0
iniconfig               2.0.0
installer               0.7.0
libdnf                  0.72.0
mypy                    1.6.0
mypy-extensions         1.0.0
packaging               23.2
pathspec                0.11.2
pluggy                  1.3.0
py-cpuinfo              9.0.0
pydantic_core           2.14.1
pyproject_hooks         1.0.0
pytest                  7.4.3
pytest-benchmark        4.0.0
pytest-mock             3.12.0
python-dateutil         2.8.2
pytz                    2023.3
six                     1.16.0
sortedcontainers        2.4.0
tomli                   2.0.1
trove-classifiers       2023.11.13
typing_extensions       4.8.0
wheel                   0.41.3
zipp                    3.17.0

The text was updated successfully, but these errors were encountered:

davidhewitt · 2023-11-14T13:59:15Z

@kloczek how did you decide on the versions to install in the build env? Can you match the versions from our pdm.lock? I suspect different Faker version or something like that.

sydney-runkle · 2023-11-14T14:01:37Z

Yep, I'm guessing it's a faker version problem.

sydney-runkle · 2023-11-14T14:06:46Z

Happy to leave this issue open though as a reminder to rework these tests - the fact that they break every time we upgrade faker is a bit annoying.

kloczek · 2023-11-14T20:52:16Z

@kloczek how did you decide on the versions to install in the build env? Can you match the versions from our pdm.lock? I suspect different Faker version or something like that.

As I wrote I;m building module as rpm package. My build infra deploys build env for exact package inside freshly spawned LXC zone in which are installed packaged as rpm packages python modules listed in rpm spec file BuildRequires (+its dependencies) downloaded from my distribution repository.
Here is list of BuildRequires:

BuildRequires:  python3dist(build)
BuildRequires:  python3dist(hatch-fancy-pypi-readme)
BuildRequires:  python3dist(hatchling)
BuildRequires:  python3dist(installer)
BuildRequires:  python3dist(wheel)
# CheckRequires:
BuildRequires:  python3dist(annotated-types)
BuildRequires:  python3dist(cloudpickle)
BuildRequires:  python3dist(dirty-equals)
BuildRequires:  python3dist(email-validator)
BuildRequires:  python3dist(faker)
BuildRequires:  python3dist(hypothesis)
BuildRequires:  python3dist(mypy)
BuildRequires:  python3dist(pydantic-core)
BuildRequires:  python3dist(pytest)
BuildRequires:  python3dist(pytest-benchmark)
#BuildRequires: python3dist(pytest-examples)
BuildRequires:  python3dist(pytest-mock)

On above list pytest-examples is commented out because it depends on ruff with which I have issues on packaging (astral-sh/ruff#5566).
On pytest execution all units which requires pytest-examples are for now on --deselect list

%check
%pytest %{!?with_failing_tests: \
        --deselect tests/benchmarks/test_north_star.py::test_north_star_dump_json \
        --deselect tests/benchmarks/test_north_star.py::test_north_star_dump_python \
        --deselect tests/benchmarks/test_north_star.py::test_north_star_json_dumps \
        --deselect tests/benchmarks/test_north_star.py::test_north_star_json_loads \
        --deselect tests/benchmarks/test_north_star.py::test_north_star_validate_json \
        --deselect tests/benchmarks/test_north_star.py::test_north_star_validate_json_strict \
        --deselect tests/benchmarks/test_north_star.py::test_north_star_validate_python \
        --deselect tests/benchmarks/test_north_star.py::test_north_star_validate_python_strict \
        --ignore tests/test_docs.py \
}

If you need more details please let me know.

sydney-runkle · 2023-11-17T14:17:01Z

@davidhewitt let's pair next week on changing these tests 👍

kloczek added bug V2 Bug related to Pydantic V2 pending Awaiting a response / confirmation labels Nov 14, 2023

sydney-runkle added this to the v2.5.1 milestone Nov 14, 2023

sydney-runkle self-assigned this Nov 14, 2023

sydney-runkle added question and removed bug V2 Bug related to Pydantic V2 pending Awaiting a response / confirmation labels Nov 14, 2023

sydney-runkle removed this from the v2.5.1 milestone Nov 14, 2023

sydney-runkle mentioned this issue Nov 21, 2023

2.5.1: changing checksum for north_star_data_bytes fixture #8180

Closed

1 task

sydney-runkle mentioned this issue Nov 29, 2023

Remove hash check from northstar benchmark #8256

Merged

5 tasks

sydney-runkle closed this as completed in #8256 Nov 29, 2023

dvzrv mentioned this issue Jan 1, 2024

2.5.3: benchmark tests still broken due to hash #8459

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

2.5.0: pytest is failing in 8 units because unexpected hash values #8099

2.5.0: pytest is failing in 8 units because unexpected hash values #8099

kloczek commented Nov 14, 2023 •

edited

davidhewitt commented Nov 14, 2023

sydney-runkle commented Nov 14, 2023

sydney-runkle commented Nov 14, 2023

kloczek commented Nov 14, 2023

sydney-runkle commented Nov 17, 2023

2.5.0: pytest is failing in 8 units because unexpected hash values #8099

2.5.0: pytest is failing in 8 units because unexpected hash values #8099

Comments

kloczek commented Nov 14, 2023 • edited

Initial Checks

Description

Python, Pydantic & OS Version

davidhewitt commented Nov 14, 2023

sydney-runkle commented Nov 14, 2023

sydney-runkle commented Nov 14, 2023

kloczek commented Nov 14, 2023

sydney-runkle commented Nov 17, 2023

kloczek commented Nov 14, 2023 •

edited