"Import file mismatch" at test collection time

[lelit]

I tried to upgrade pytest from 8.0.2 to latest 8.1.1 and I got this error:

___________ ERROR collecting tests/conftest.py _____________________
import file mismatch:
imported module 'conftest' has this __file__ attribute:
  .../tests/bio/conftest.py
which is not the same as the test file we want to collect:
  .../tests/conftest.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules

Neither removing all __pycache__ subdirs, nor playing with the --import-mode option fixed it.

The top level tests directory contains this structure:

$ tree -d tests/
tests/
├── __pycache__
├── bio
│   └── __pycache__
├── models
│   └── __pycache__
├── scr
└── server
    └── __pycache__

and there are different conftest.py modules:

$ find tests/ -name 'conftest.py'
tests/conftest.py
tests/bio/conftest.py
tests/server/conftest.py

Given that I could not find a related breaking change alert, I'd say that the "use a unique basename for your test file modules" hint does not apply to conftest.py, right?

Thanks&bye.

Environment

I'm using Python 3.11.8 on a Debian sid, with the following packages in the virtual environment (after downgrading pytest to the working 8.0.2):

$ pip list
Package                         Version  Editable project location
------------------------------- -------- -------------------------
alabaster                       0.7.16
alembic                         1.13.1
attrs                           23.2.0
Babel                           2.14.0
beautifulsoup4                  4.12.3
bleach                          6.1.0
calmjs.parse                    1.3.1
certifi                         2024.2.2
cffi                            1.16.0
chardet                         5.2.0
charset-normalizer              3.3.2
commonmark                      0.9.1
coverage                        7.4.4
cryptography                    42.0.5
docutils                        0.20.1
flake8                          7.0.0
greenlet                        3.0.3
hupper                          1.12.1
idna                            3.6
imagesize                       1.4.1
importlib_metadata              7.0.2
iniconfig                       2.0.0
itsdangerous                    2.1.2
jaraco.classes                  3.2.3
jeepney                         0.8.0
Jinja2                          3.1.3
keyring                         24.3.1
Mako                            1.3.2
markdown-it-py                  2.2.0
MarkupSafe                      2.1.5
mccabe                          0.7.0
mdurl                           0.1.2
metapensiero.extjs.desktop      2.4
metapensiero.sqlalchemy.dbloady 3.0.dev3
metapensiero.sqlalchemy.proxy   6.0.dev5
metapensiero.tool.bump-version  1.3
more-itertools                  10.2.0
mypy                            1.6.1
mypy-extensions                 1.0.0
nh3                             0.2.14
packaging                       24.0
PasteDeploy                     3.1.0
pillow                          10.2.0
pip                             24.0
pkginfo                         1.10.0
plaster                         1.1.2
plaster-pastedeploy             1.0.1
pluggy                          1.4.0
ply                             3.11
progressbar2                    4.4.2
pycodestyle                     2.11.1
pycountry                       23.12.11
pycparser                       2.21
pyflakes                        3.2.0
Pygments                        2.17.2
PyNaCl                          1.5.0
pyparsing                       3.1.2
pyramid                         2.0.2
pyramid-mailer                  0.15.1
pyramid-mako                    1.1.0
pyramid-tm                      2.5
pyRXP                           3.0.1
pytest                          8.0.2
pytest-cov                      4.1.0
python-rapidjson                1.16
python-utils                    3.8.2
pytz                            2024.1
rcssmin                         1.1.2
readme_renderer                 43.0
reportlab                       4.1.0
repoze.sendmail                 4.4.1
requests                        2.31.0
requests-toolbelt               1.0.0
rfc3986                         2.0.0
rich                            13.7.1
rjsmin                          1.2.2
ruamel.yaml                     0.18.6
ruamel.yaml.clib                0.2.8
ruff                            0.1.1
SecretStorage                   3.3.3
setuptools                      69.2.0
six                             1.16.0
snowballstemmer                 2.2.0
SoL                             4.21     /home/lele/wip/sol5/src
soupsieve                       2.5
Sphinx                          7.2.6
sphinxcontrib-applehelp         1.0.8
sphinxcontrib-devhelp           1.0.6
sphinxcontrib-htmlhelp          2.0.5
sphinxcontrib-jsmath            1.0.1
sphinxcontrib-qthelp            1.0.7
sphinxcontrib-serializinghtml   1.1.10
SQLAlchemy                      2.0.28
tqdm                            4.66.2
transaction                     4.0
translationstring               1.4
twine                           5.0.0
typing_extensions               4.10.0
urllib3                         2.2.1
venusian                        3.1.0
Versio                          0.5.0
waitress                        3.0.0
wcwidth                         0.2.13
webencodings                    0.5.1
WebOb                           1.8.7
WebTest                         3.0.0
wheel                           0.43.0
XlsxWriter                      3.2.0
zipp                            3.18.1
zope.deprecation                5.0
zope.interface                  6.2
zope.sqlalchemy                 3.1

[nicoddemus]

Hi @lelit, thanks for the report.

I could not reproduce the issue however, here is my file tree:

└───tests
    │   conftest.py
    │
    ├───bio
    │   │   conftest.py
    │   │   tests_bio.py
    │
    ├───models
    │   │   test_models.py
    │
    ├───scr
    │   │   test_scr.py
    │
    ├───server
    │   │   conftest.py
    │   │   test_server.py   

(all files are empty)

This runs pytest file for me (with zero collected tests of course).

Can you try to provide a minimal working example?

[lelit]

Oh! I will try to distill an MWE, but for now a relatively quick (it takes a couple of minutes in total) recipe is:

$ docker run -ti python:3.11 bash
# cd /tmp
# git clone -b sol5 https://gitlab.com/metapensiero/SoL.git
# cd SoL
# python -m venv env
# source env/bin/activate
# pip install -r requirements/development.txt
# make test

The above works ok, producing

=========================== test session starts ===========================
platform linux -- Python 3.11.8, pytest-8.0.2, pluggy-1.4.0
rootdir: /tmp/SoL
configfile: setup.cfg
plugins: cov-4.1.0
collected 304 items                                                       

src/sol/i18n.py .                                                   [  0%]
src/sol/models/utils.py ....                                        [  1%]
...
========= 299 passed, 5 skipped, 12 warnings in 66.61s (0:01:06) ==========

Then if I do

# pip install pytest==8.1.1
# make test

I get the following instead:

=========================== test session starts ===========================
platform linux -- Python 3.11.8, pytest-8.1.1, pluggy-1.4.0
rootdir: /tmp/SoL
configfile: setup.cfg
plugins: cov-4.1.0
collected 304 items / 1 error                                             

================================= ERRORS ==================================
___________________ ERROR collecting tests/conftest.py ____________________
import file mismatch:
imported module 'conftest' has this __file__ attribute:
  /tmp/SoL/tests/bio/conftest.py
which is not the same as the test file we want to collect:
  /tmp/SoL/tests/conftest.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules

As said, I will try to minimize it.

In the meantime, thanks for your time!

[xflr6]

Thanks! Same here: https://github.com/xflr6/graphviz/actions/runs/8317341967 (with multiple conftest.py files).

[mgorny]

If that's any help, I was able to workaround the problem by specifying --ignore tests/conftest.py --ignore tests/backend/conftest.py ....

[lelit]

Hi, a little update: as promised I tried to come up with an MWE that exhibits the problem, but so far I had no luck in isolating the condition that triggers it. In another, even bigger project I wrote and maintain it does not happen, and neither in a very simplicistic mock, similar to what @nicoddemus did above, but with not completely empty files (test units and conftest.py).
I will try again.

[mgorny]

I suspect --doctest-modules may be relevant.

[lelit]

Thanks, I will take that into consideration, even if the other project I mentioned uses doctests as well but does not manifest the problem.

[lelit]

I'm sorry, I could not reserve some spare time to further investigate the issue, trying to condense a standalone replication recipe.

[lelit]

While I was not able to distill a repeatable MWE recipe, I executed a git bisect running pytest --collect-only in my project, basically doing

git bisect start
git bisect bad 8.2.0
git bisect good 8.0.2
git run sh run.sh

where run.sh is

pip install -U .
cd ~/wip/sol5/
pytest --collect-only

The final outcome is the following:

06dbd3c21ccdf1ac76e8fa264048133cb4660842 is the first bad commit
commit 06dbd3c21ccdf1ac76e8fa264048133cb4660842
Author: Ran Benita <ran@unusedvar.com>
Date:   Sat Jan 13 11:15:05 2024 +0200

    doctest: remove special conftest handling
    
    (Diff better viewed ignoring whitespace)
    
    Since e1c66ab0ad8eda13e5552dfc939e07d7290ecd39, conftest loading is
    handled at the directory level before sub-nodes are collected, so there
    is no need for the doctest plugin to handle it specially.
    
    This was probably the case even before
    e1c66ab0ad8eda13e5552dfc939e07d7290ecd39, but I haven't verified this.

 src/_pytest/doctest.py | 26 ++++++++++----------------
 1 file changed, 10 insertions(+), 16 deletions(-)
bisect found first bad commit

Hope this brings some enlightenment.