pypy test has started failing #463

jaraco · 2023-07-07T13:04:21Z

Around two weeks ago, in this build, tests started failing on PyPy, implicating f5a5617. Is it the case that pypy3.9 needs -dev?

The text was updated successfully, but these errors were encountered:

jaraco · 2023-07-07T13:14:05Z

Between that build and the prior run, the PyPy version changed from 7.3.11 (Python 3.9.16) to 7.3.12 (Python 3.9.17).

jaraco · 2023-07-07T13:17:21Z

Testing locally, I have pypy 7.3.10-alpha, and tests pass.

Looks like maybe the reason I have an old pypy is because I built/installed it from source to support macOS ARM. It does appear that 7.3.12 is available on homebrew now.

jaraco · 2023-07-07T13:33:02Z

After realizing that brew install pypy installs an old and deprecated version of Python (yeah, I know pypy still supports Python 2.7; still, I expect a naked "pypy" to install the most modern stable version available), I installed pypy3 and that gave me pypy 7.3.12 (Python 3.10.12). In that environment, I'm unable to replicate the failure.

jaraco · 2023-07-07T13:35:33Z

I am, however, able to replicate the issue using pypy 3.9.

 importlib_metadata main @ tox -e pypy3.9 -- -k egg
.pkg-pypy39: recreate env because python changed version_info=[3, 9, 12, 'final', 0]->[3, 9, 17, 'final', 0] | executable='/opt/pypy-c-jit-105878-d4b7ddecdfcd-macosx_arm64/bin/pypy3.9'->'/opt/homebrew/Cellar/pypy3.9/7.3.12/libexec/bin/pypy3.9'
.pkg-pypy39: remove tox env folder .tox/.pkg-pypy39
.pkg-pypy39: install_requires> python -I -m pip install 'setuptools>=56' 'setuptools_scm[toml]>=3.4.1'
.pkg-pypy39: _optional_hooks> python /Users/jaraco/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg-pypy39: get_requires_for_build_editable> python /Users/jaraco/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg-pypy39: install_requires_for_build_editable> python -I -m pip install wheel
.pkg-pypy39: build_editable> python /Users/jaraco/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
pypy3.9: install_package_deps> python -I -m pip install flufl.flake8 'importlib-resources>=1.3; python_version < "3.9"' packaging pyfakefs 'pytest-black>=0.3.7; platform_python_implementation != "PyPy"' 'pytest-checkdocs>=2.4' pytest-cov 'pytest-enabler>=2.2' 'pytest-mypy>=0.9.1; platform_python_implementation != "PyPy"' 'pytest-perf>=0.9.2' pytest-ruff 'pytest>=6' 'typing-extensions>=3.6.4; python_version < "3.8"' 'zipp>=0.5'
pypy3.9: install_package> python -I -m pip install --force-reinstall --no-deps .tox/.tmp/package/57/importlib_metadata-6.7.1.dev16+gf862e3fd-0.editable-py3-none-any.whl
pypy3.9: commands[0]> pytest -k egg
========================================================================= test session starts ==========================================================================
platform darwin -- Python 3.9.17[pypy-7.3.12-final], pytest-7.4.0, pluggy-1.2.0
cachedir: .tox/pypy3.9/.pytest_cache
rootdir: /Users/jaraco/code/python/importlib_metadata
configfile: pytest.ini
plugins: pyfakefs-5.2.2, enabler-2.2.0, cov-4.1.0, ruff-0.1, checkdocs-2.9.0, perf-0.13.1
collected 140 items / 119 deselected / 21 selected                                                                                                                     

tests/test_api.py .F.....                                                                                                                                        [ 36%]
tests/test_main.py ...F                                                                                                                                          [ 57%]
tests/test_zip.py ........                                                                                                                                       [100%]

=============================================================================== FAILURES ===============================================================================
_____________________________________________________________________ APITests.test_files_egg_info _____________________________________________________________________

self = <tests.test_api.APITests testMethod=test_files_egg_info>

    def test_files_egg_info(self):
        self._test_files(files('egginfo-pkg'))
>       self._test_files(files('egg_with_module-pkg'))

tests/test_api.py:203: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

files = [PackagePath('egg_with_module.py'), PackagePath('setup.py'), PackagePath('egg_with_module_pkg.egg-info/PKG-INFO'), PackagePath('egg_with_module_pkg.egg-info/SOURCES.txt'), PackagePath('egg_with_module_pkg.egg-info/top_level.txt')]

    @staticmethod
    def _test_files(files):
        root = files[0].root
        for file in files:
            assert file.root == root
            assert not file.hash or file.hash.value
            assert not file.hash or file.hash.mode == 'sha256'
            assert not file.size or file.size >= 0
>           assert file.locate().exists()
E           AssertionError: assert False
E            +  where False = <bound method Path.exists of PosixPath('/var/folders/sx/n5gkrgfx6zd91ymxr2sr9wvw00n8zm/T/tmpfv4sscgk/setup.py')>()
E            +    where <bound method Path.exists of PosixPath('/var/folders/sx/n5gkrgfx6zd91ymxr2sr9wvw00n8zm/T/tmpfv4sscgk/setup.py')> = PosixPath('/var/folders/sx/n5gkrgfx6zd91ymxr2sr9wvw00n8zm/T/tmpfv4sscgk/setup.py').exists
E            +      where PosixPath('/var/folders/sx/n5gkrgfx6zd91ymxr2sr9wvw00n8zm/T/tmpfv4sscgk/setup.py') = <bound method PackagePath.locate of PackagePath('setup.py')>()
E            +        where <bound method PackagePath.locate of PackagePath('setup.py')> = PackagePath('setup.py').locate

tests/test_api.py:189: AssertionError
___________________________________________________ PackagesDistributionsEggTest.test_packages_distributions_on_eggs ___________________________________________________

self = <tests.test_main.PackagesDistributionsEggTest testMethod=test_packages_distributions_on_eggs>

    def test_packages_distributions_on_eggs(self):
        """
        Test old-style egg packages with a variation of 'top_level.txt',
        'SOURCES.txt', and 'installed-files.txt', available.
        """
        distributions = packages_distributions()
    
        def import_names_from_package(package_name):
            return {
                import_name
                for import_name, package_names in distributions.items()
                if package_name in package_names
            }
    
        # egginfo-pkg declares one import ('mod') via top_level.txt
        assert import_names_from_package('egginfo-pkg') == {'mod'}
    
        # egg_with_module-pkg has one import ('egg_with_module') inferred from
        # installed-files.txt (top_level.txt is missing)
>       assert import_names_from_package('egg_with_module-pkg') == {'egg_with_module'}
E       AssertionError: assert {'egg_with_module', 'setup'} == {'egg_with_module'}
E         Extra items in the left set:
E         'setup'
E         Use -v to get more diff

tests/test_main.py:451: AssertionError
=========================================================================== warnings summary ===========================================================================
../../../../../opt/homebrew/Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/_testmultiphase_build.py:8
  /opt/homebrew/Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/_testmultiphase_build.py:8: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import imp

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

---------- coverage: platform darwin, python 3.9.17-final-0 ----------
Name                                 Stmts   Miss  Cover   Missing
------------------------------------------------------------------
conftest.py                             10      0   100%
docs/__init__.py                         0      0   100%
docs/conf.py                            13      0   100%
importlib_metadata/__init__.py         393    171    56%   56, 60-61, 107, 115-122, 126, 152-153, 201, 208-211, 215-217, 221-223, 227-229, 232-233, 255-256, 259, 262, 265, 268, 271, 277, 293-294, 308, 315, 319, 323, 337-338, 341-342, 346, 351, 354, 359-373, 406, 424, 437, 454-463, 468, 473, 478, 482, 496-511, 523-524, 537-552, 566-567, 572-573, 576, 579-580, 584, 598-619, 681, 684, 687, 690-694, 697-701, 704, 708-710, 714, 719-737, 740-750, 762-766, 773, 781, 784, 806-807, 812-813, 818, 827, 830-839, 844, 852-853, 869-873
importlib_metadata/_adapters.py         41     24    41%   43-45, 48, 52, 59-62, 65-74, 83-90
importlib_metadata/_collections.py      10      3    70%   21, 24, 30
importlib_metadata/_compat.py           23      0   100%
importlib_metadata/_functools.py        17      5    71%   99-104
importlib_metadata/_meta.py             24      0   100%
importlib_metadata/_py39compat.py       14      6    57%   19-22, 31-35
importlib_metadata/_text.py             25      9    64%   68, 71, 74, 77, 83, 87, 95, 98-99
------------------------------------------------------------------
TOTAL                                  570    218    62%

================================================================================= perf =================================================================================
exercises.py:cached distribution: None
exercises.py:discovery: None
exercises.py:entry_points(): None
exercises.py:entrypoint_regexp_perf: None
exercises.py:uncached distribution: None
======================================================================= short test summary info ========================================================================
FAILED tests/test_api.py::APITests::test_files_egg_info - AssertionError: assert False
FAILED tests/test_main.py::PackagesDistributionsEggTest::test_packages_distributions_on_eggs - AssertionError: assert {'egg_with_module', 'setup'} == {'egg_with_module'}
======================================================= 2 failed, 17 passed, 119 deselected, 1 warning in 4.96s ========================================================
pypy3.9: exit 1 (6.07 seconds) /Users/jaraco/code/python/importlib_metadata> pytest -k egg pid=55311
.pkg-pypy39: _exit> python /Users/jaraco/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
  pypy3.9: FAIL code 1 (17.22=setup[11.15]+cmd[6.07] seconds)
  evaluation failed :( (17.39 seconds)

jaraco · 2023-07-07T13:39:30Z

Well, I did replicate it once, but the failure went away.

 importlib_metadata main @ git clean -fdx
Removing .tox/
Removing __pycache__/
Removing docs/__pycache__/
Removing importlib_metadata.egg-info/
Removing importlib_metadata/__pycache__/
Removing prepare/example/example/__pycache__/
Removing prepare/example2/example2/__pycache__/
Removing tests/__pycache__/
Removing tests/data/__pycache__/
 importlib_metadata main @ tox -e pypy3.9 -- -k egg
.pkg-pypy39: install_requires> python -I -m pip install 'setuptools>=56' 'setuptools_scm[toml]>=3.4.1'
.pkg-pypy39: _optional_hooks> python /Users/jaraco/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg-pypy39: get_requires_for_build_editable> python /Users/jaraco/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg-pypy39: install_requires_for_build_editable> python -I -m pip install wheel
.pkg-pypy39: build_editable> python /Users/jaraco/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
pypy3.9: install_package_deps> python -I -m pip install flufl.flake8 'importlib-resources>=1.3; python_version < "3.9"' packaging pyfakefs 'pytest-black>=0.3.7; platform_python_implementation != "PyPy"' 'pytest-checkdocs>=2.4' pytest-cov 'pytest-enabler>=2.2' 'pytest-mypy>=0.9.1; platform_python_implementation != "PyPy"' 'pytest-perf>=0.9.2' pytest-ruff 'pytest>=6' 'typing-extensions>=3.6.4; python_version < "3.8"' 'zipp>=0.5'
pypy3.9: install_package> python -I -m pip install --force-reinstall --no-deps .tox/.tmp/package/1/importlib_metadata-6.7.1.dev16+gf862e3fd-0.editable-py3-none-any.whl
pypy3.9: commands[0]> pytest -k egg
========================================================================= test session starts ==========================================================================
platform darwin -- Python 3.9.17[pypy-7.3.12-final], pytest-7.4.0, pluggy-1.2.0
cachedir: .tox/pypy3.9/.pytest_cache
rootdir: /Users/jaraco/code/python/importlib_metadata
configfile: pytest.ini
plugins: pyfakefs-5.2.2, enabler-2.2.0, cov-4.1.0, ruff-0.1, checkdocs-2.9.0, perf-0.13.1
collected 140 items / 119 deselected / 21 selected                                                                                                                     

tests/test_api.py .......                                                                                                                                        [ 36%]
tests/test_main.py ....                                                                                                                                          [ 57%]
tests/test_zip.py ........                                                                                                                                       [100%]

=========================================================================== warnings summary ===========================================================================
../../../../../opt/homebrew/Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/_testmultiphase_build.py:8
  /opt/homebrew/Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/_testmultiphase_build.py:8: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import imp

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

---------- coverage: platform darwin, python 3.9.17-final-0 ----------
Name                                 Stmts   Miss  Cover   Missing
------------------------------------------------------------------
conftest.py                             10      0   100%
docs/__init__.py                         0      0   100%
docs/conf.py                            13      0   100%
importlib_metadata/__init__.py         393     35    91%   56, 60-61, 152-153, 208-211, 215-217, 221-223, 227-229, 259, 262, 265, 268, 271, 277, 293-294, 308, 315, 354, 368, 406, 424, 437, 605, 818
importlib_metadata/_adapters.py         41      1    98%   61
importlib_metadata/_collections.py      10      0   100%
importlib_metadata/_compat.py           23      0   100%
importlib_metadata/_functools.py        17      0   100%
importlib_metadata/_meta.py             24      0   100%
importlib_metadata/_py39compat.py       14      6    57%   19-22, 31-35
importlib_metadata/_text.py             25      8    68%   68, 71, 77, 83, 87, 95, 98-99
------------------------------------------------------------------
TOTAL                                  570     50    91%

================================================================================= perf =================================================================================
exercises.py:cached distribution: None
exercises.py:discovery: None
exercises.py:entry_points(): None
exercises.py:entrypoint_regexp_perf: None
exercises.py:uncached distribution: None
============================================================ 19 passed, 119 deselected, 1 warning in 3.73s =============================================================
.pkg-pypy39: _exit> python /Users/jaraco/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
  pypy3.9: OK (15.57=setup[10.78]+cmd[4.79] seconds)
  congratulations :) (15.62 seconds)

WTH?

Note that while I did rebuild the environment above, it wasn't the rebuilding the environment that allowed the tests to start passing. After the original error occurred, I set out to use pdb to triage, and even then, the replicated failure did not occur:

======================================================= 2 failed, 17 passed, 119 deselected, 1 warning in 4.96s ========================================================
pypy3.9: exit 1 (6.07 seconds) /Users/jaraco/code/python/importlib_metadata> pytest -k egg pid=55311
.pkg-pypy39: _exit> python /Users/jaraco/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
  pypy3.9: FAIL code 1 (17.22=setup[11.15]+cmd[6.07] seconds)
  evaluation failed :( (17.39 seconds)
 importlib_metadata main @ tox -e pypy3.9 -- -p no:cov -p no:perf -k egg --pdb
.pkg-pypy39: _optional_hooks> python /Users/jaraco/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg-pypy39: get_requires_for_build_editable> python /Users/jaraco/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg-pypy39: build_editable> python /Users/jaraco/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
pypy3.9: install_package> python -I -m pip install --force-reinstall --no-deps .tox/.tmp/package/58/importlib_metadata-6.7.1.dev16+gf862e3fd-0.editable-py3-none-any.whl
pypy3.9: commands[0]> pytest -p no:cov -p no:perf -k egg --pdb
========================================================================= test session starts ==========================================================================
platform darwin -- Python 3.9.17[pypy-7.3.12-final], pytest-7.4.0, pluggy-1.2.0
cachedir: .tox/pypy3.9/.pytest_cache
rootdir: /Users/jaraco/code/python/importlib_metadata
configfile: pytest.ini
plugins: pyfakefs-5.2.2, enabler-2.2.0, ruff-0.1, checkdocs-2.9.0
collected 135 items / 114 deselected / 21 selected                                                                                                                     

tests/test_api.py .......                                                                                                                                        [ 36%]
tests/test_main.py ....                                                                                                                                          [ 57%]
tests/test_zip.py ........                                                                                                                                       [100%]

=========================================================================== warnings summary ===========================================================================
../../../../../opt/homebrew/Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/_testmultiphase_build.py:8
  /opt/homebrew/Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/_testmultiphase_build.py:8: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import imp

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================================================ 19 passed, 114 deselected, 1 warning in 0.79s =============================================================

jaraco · 2023-07-07T14:02:13Z

I've repeated that command using watch (watch tox -e pypy3.9 -- -p no:cov -p no:perf -k egg) and it's just not failing. It seems it's not an intermittent issue but more of an "initial state" issue. I removed and reinstalled pypy3.9, then ran the command an it failed. Then I ran the command again and it passed.

jaraco · 2023-07-07T14:55:46Z

In an attempt to understand better the differences between a fresh failing run and a passing run, I ran the following:

 importlib_metadata main @ brew install pypy3.9
...
 importlib_metadata main @ mtree -c -K sha256digest -p /opt/homebrew > before.mtree
 importlib_metadata main @ tox -e pypy3.9 -- -p no:cov -p no:perf -k egg > out.txt
 importlib_metadata main @ mtree -p /opt/homebrew < before.mtree

Here's the output from that last command:

Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9 changed
        modification time expected Fri Jul  7 10:14:30 2023.406221284 found Fri Jul  7 10:53:36 2023.365183105
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/encodings/__pycache__ changed
        modification time expected Fri Jul  7 10:14:32 2023.887796946 found Fri Jul  7 10:53:34 2023.518428290
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/encodings/__pycache__/ascii.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/encodings/__pycache__/idna.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/unittest changed
        modification time expected Thu Jun 15 05:31:24 2023.000000000 found Fri Jul  7 10:53:35 2023.050045154
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/unittest/__pycache__ extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/test changed
        modification time expected Thu Jun 15 05:31:24 2023.000000000 found Fri Jul  7 10:53:35 2023.760245849
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/test/__pycache__ extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/test/support changed
        modification time expected Thu Jun 15 05:31:24 2023.000000000 found Fri Jul  7 10:53:35 2023.773638256
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/test/support/__pycache__ extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/html/__pycache__ changed
        modification time expected Fri Jul  7 10:14:32 2023.015249114 found Fri Jul  7 10:53:34 2023.591665055
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/html/__pycache__/parser.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__ changed
        modification time expected Fri Jul  7 10:14:32 2023.290237079 found Fri Jul  7 10:53:36 2023.652534365
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/__future__.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/decimal.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/imp.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/difflib.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/_ctypes_test_build.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/csv.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/cmd.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/fractions.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/filecmp.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/socketserver.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/gzip.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/code.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/stringprep.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/compileall.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/runpy.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/uuid.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/optparse.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/doctest.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/ipaddress.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/_contextvars.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/contextvars.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/_strptime.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/pathlib.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/getpass.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/colorsys.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/pdb.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/mimetypes.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/_pydecimal.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/codeop.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/_pypy_testcapi.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/dataclasses.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/_immutables_map.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/bdb.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/_markupbase.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/_testmultiphase_build.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/argparse.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/__pycache__/hmac.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/xml/etree changed
        modification time expected Thu Jun 15 05:31:24 2023.000000000 found Fri Jul  7 10:53:35 2023.129801578
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/xml/etree/__pycache__ extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/http/__pycache__ changed
        modification time expected Fri Jul  7 10:14:32 2023.026929680 found Fri Jul  7 10:53:34 2023.507601703
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/http/__pycache__/cookiejar.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/http/__pycache__/cookies.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/opt extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/importlib/__pycache__ changed
        modification time expected Fri Jul  7 10:14:30 2023.731750899 found Fri Jul  7 10:53:33 2023.418067146
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/importlib/__pycache__/resources.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/importlib/__pycache__/_common.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/importlib/__pycache__/metadata.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/xmlrpc changed
        modification time expected Thu Jun 15 05:31:24 2023.000000000 found Fri Jul  7 10:53:34 2023.629520488
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/xmlrpc/__pycache__ extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/asyncio changed
        modification time expected Thu Jun 15 05:31:24 2023.000000000 found Fri Jul  7 10:53:34 2023.336739492
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/asyncio/__pycache__ extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/logging/__pycache__ changed
        modification time expected Fri Jul  7 10:14:31 2023.119551107 found Fri Jul  7 10:53:34 2023.576308185
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/logging/__pycache__/handlers.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/logging/__pycache__/config.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/email/__pycache__ changed
        modification time expected Fri Jul  7 10:14:32 2023.036928619 found Fri Jul  7 10:53:33 2023.836649529
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/email/__pycache__/contentmanager.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/email/__pycache__/headerregistry.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/email/__pycache__/generator.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/email/__pycache__/_header_value_parser.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/email/__pycache__/policy.pypy39.pyc extra
Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/_testmultiphase.pypy39-pp73-darwin.so extra

jaraco · 2023-07-07T15:07:32Z

Curiously (and maybe unrelated), there seems to be a glitch in path handling that's causing the duplicate path to be created in /opt/homebrew/Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/opt/homebrew/Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/_testmultiphase.o.

That fact may be implicated in the failure, which I can now repeat without reinstalling pypy, but instead removing _testmultiphase prior to running the tests:

importlib_metadata main @ rm /opt/homebrew/Cellar/pypy3.9/7.3.12/libexec/lib/pypy3.9/_testmultiphase.pypy39-pp73-darwin.so

Based on timing and other common factors, I suspect the issue is related to pypy/pypy#3953.

mtelka · 2023-12-02T20:04:28Z

I do see both test_files_egg_info and test_packages_distributions_on_eggs failing the exactly same way with importlib_metadata 6.9.0 on OpenIndiana when tested with Python 3.9.16.

.

jaraco · 2023-12-03T16:12:31Z

I do see both test_files_egg_info and test_packages_distributions_on_eggs failing the exactly same way with importlib_metadata 6.9.0 on OpenIndiana when tested with Python 3.9.16.

When you say Python 3.9.16, do you mean CPython or PyPy? If the former, that's interesting and surprising as this issue was previously isolated to PyPy... and tests have been passing reliably on CPython across popular platforms throughout. Do you have a way to replicate it?

I have re-enabled the PyPy tests in CI, since upstream skeleton has bumped from 3.9 to 3.10, bypassing the underlying issue.

Moreover, now that I have pypy 7.3.13 (3.9.18), I'm no longer able to replicate the issue locally, even after deleting _testmultiphase.pypy39-pp73-darwin.so. Confirming the upstream fix for pypy/pypy#3953.

 importlib_metadata main @ rm /opt/homebrew/Cellar/pypy3.9/7.3.13/libexec/lib/pypy3.9/_testmultiphase.pypy39-pp73-darwin.so
 importlib_metadata main @ tox -e pypy3.9 -- -p no:cov -p no:perf -k egg
.pkg-pypy39: _optional_hooks> python /Users/jaraco/.local/pipx/venvs/tox/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg-pypy39: get_requires_for_build_editable> python /Users/jaraco/.local/pipx/venvs/tox/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg-pypy39: build_editable> python /Users/jaraco/.local/pipx/venvs/tox/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta
pypy3.9: install_package> python -I -m pip install --force-reinstall --no-deps /Users/jaraco/code/python/importlib_metadata/.tox/.tmp/package/37/importlib_metadata-6.10.1.dev1+g72383028-0.editable-py3-none-any.whl
pypy3.9: commands[0]> pytest -p no:cov -p no:perf -k egg
================================================================== test session starts ===================================================================
platform darwin -- Python 3.9.18[pypy-7.3.13-final], pytest-7.4.3, pluggy-1.3.0
cachedir: .tox/pypy3.9/.pytest_cache
rootdir: /Users/jaraco/code/python/importlib_metadata
configfile: pytest.ini
plugins: checkdocs-2.10.1, pyfakefs-5.3.2, enabler-2.3.1, ruff-0.2.1
collected 136 items / 115 deselected / 21 selected                                                                                                       

tests/test_api.py .......                                                                                                                          [ 36%]
tests/test_main.py ....                                                                                                                            [ 57%]
tests/test_zip.py ........                                                                                                                         [100%]

=========================================================== 19 passed, 115 deselected in 0.83s ===========================================================
.pkg-pypy39: _exit> python /Users/jaraco/.local/pipx/venvs/tox/lib/python3.12/site-packages/pyproject_api/_backend.py True setuptools.build_meta
  pypy3.9: OK (2.56=setup[1.34]+cmd[1.23] seconds)
  congratulations :) (2.63 seconds)

mtelka · 2023-12-03T16:39:40Z

I do see both test_files_egg_info and test_packages_distributions_on_eggs failing the exactly same way with importlib_metadata 6.9.0 on OpenIndiana when tested with Python 3.9.16.

When you say Python 3.9.16, do you mean CPython or PyPy? If the former, that's interesting and surprising as this issue was previously isolated to PyPy... and tests have been passing reliably on CPython across popular platforms throughout. Do you have a way to replicate it?

When I say Python 3.9.16 I mean CPython. And yes, it looks like it is reproducible reliably. When I played with importlib_metadata 6.9.0 yesterday both tests failed all the time, reliably and reproducible. I ran testing several times, definitely more than 5, maybe even more than 10, I didn't count them...

Moreover, now that I have pypy 7.3.13 (3.9.18), I'm no longer able to replicate the issue locally, even after deleting _testmultiphase.pypy39-pp73-darwin.so. Confirming the upstream fix for pypy/pypy#3953.

I do not have CPython > 3.9.16 handy so I cannot confirm.

jaraco · 2023-12-03T18:23:42Z

Although the issue may be similar, I think it's better to track the new failure in a separate issue Thanks for filing #479.

jaraco added a commit that referenced this issue Jul 7, 2023

Disable tests on pypy due to #463

6c79954

jaraco added a commit that referenced this issue Jul 7, 2023

Disable tests on pypy due to #463

ec7bca0

jaraco added a commit that referenced this issue Dec 3, 2023

Restore pypy tests now that 3.10 is the standard. Bypasses issue in #463

7238302

.

jaraco closed this as completed Dec 3, 2023

jaraco mentioned this issue Dec 3, 2023

6.9.0: DiscoveryTests.test_package_discovery fails #479

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pypy test has started failing #463

pypy test has started failing #463

jaraco commented Jul 7, 2023

jaraco commented Jul 7, 2023

jaraco commented Jul 7, 2023 •

edited

jaraco commented Jul 7, 2023

jaraco commented Jul 7, 2023 •

edited

jaraco commented Jul 7, 2023 •

edited

jaraco commented Jul 7, 2023

jaraco commented Jul 7, 2023

jaraco commented Jul 7, 2023 •

edited

mtelka commented Dec 2, 2023

jaraco commented Dec 3, 2023 •

edited

mtelka commented Dec 3, 2023

jaraco commented Dec 3, 2023

pypy test has started failing #463

pypy test has started failing #463

Comments

jaraco commented Jul 7, 2023

jaraco commented Jul 7, 2023

jaraco commented Jul 7, 2023 • edited

jaraco commented Jul 7, 2023

jaraco commented Jul 7, 2023 • edited

jaraco commented Jul 7, 2023 • edited

jaraco commented Jul 7, 2023

jaraco commented Jul 7, 2023

jaraco commented Jul 7, 2023 • edited

mtelka commented Dec 2, 2023

jaraco commented Dec 3, 2023 • edited

mtelka commented Dec 3, 2023

jaraco commented Dec 3, 2023

jaraco commented Jul 7, 2023 •

edited

jaraco commented Jul 7, 2023 •

edited

jaraco commented Jul 7, 2023 •

edited

jaraco commented Jul 7, 2023 •

edited

jaraco commented Dec 3, 2023 •

edited