Pytest crashes if there is a subdirectory it does not have read access to

[jesnie]

Pytest crashes if there is a subdirectory it does not have read access to.

For example:

rm -rf .venv chaff tests
virtualenv -p python3.10 .venv
source .venv/bin/activate
pip install pytest
pip list
mkdir chaff
mkdir tests
cat <<EOF > tests/test_something.py
def test_something() -> None:
    pass
EOF
pytest tests
chmod a-rwx chaff
pytest tests

yields:

created virtual environment CPython3.10.12.final.0-64 in 70ms
  creator CPython3Posix(dest=/home/jn/src/pytesttest/.venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/jn/.local/share/virtualenv)
    added seed packages: pip==22.0.2, setuptools==59.6.0, wheel==0.37.1
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

<snip pip installation stuff...>

Installing collected packages: tomli, pluggy, packaging, iniconfig, exceptiongroup, pytest
Successfully installed exceptiongroup-1.2.0 iniconfig-2.0.0 packaging-24.0 pluggy-1.4.0 pytest-8.1.1 tomli-2.0.1

Package        Version
-------------- -------
exceptiongroup 1.2.0
iniconfig      2.0.0
packaging      24.0
pip            22.0.2
pluggy         1.4.0
pytest         8.1.1
setuptools     59.6.0
tomli          2.0.1
wheel          0.37.1

=================================================================================================================== test session starts ====================================================================================================================
platform linux -- Python 3.10.12, pytest-8.1.1, pluggy-1.4.0 -- /home/jn/src/pytesttest/.venv/bin/python
cachedir: .pytest_cache
rootdir: /home/jn/src/pytesttest
collected 1 item

tests/test_something.py::test_something PASSED

==================================================================================================================== 1 passed in 0.00s =====================================================================================================================

=================================================================================================================== test session starts ====================================================================================================================
platform linux -- Python 3.10.12, pytest-8.1.1, pluggy-1.4.0 -- /home/jn/src/pytesttest/.venv/bin/python
cachedir: .pytest_cache
rootdir: /home/jn/src/pytesttest
collected 0 items / 1 error

========================================================================================================================== ERRORS ==========================================================================================================================
____________________________________________________________________________________________________________________ ERROR collecting . ____________________________________________________________________________________________________________________
.venv/lib/python3.10/site-packages/pluggy/_hooks.py:501: in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
.venv/lib/python3.10/site-packages/pluggy/_manager.py:119: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
.venv/lib/python3.10/site-packages/_pytest/python.py:212: in pytest_collect_directory
    if pkginit.is_file():
/usr/lib/python3.10/pathlib.py:1322: in is_file
    return S_ISREG(self.stat().st_mode)
/usr/lib/python3.10/pathlib.py:1097: in stat
    return self._accessor.stat(self, follow_symlinks=follow_symlinks)
E   PermissionError: [Errno 13] Permission denied: '/home/jn/src/pytesttest/chaff/__init__.py'
================================================================================================================= short test summary info ==================================================================================================================
ERROR . - PermissionError: [Errno 13] Permission denied: '/home/jn/src/pytesttest/chaff/__init__.py'
===================================================================================================================== 1 error in 0.04s =====================================================================================================================
ERROR: found no collectors for /home/jn/src/pytesttest/tests

I do not believe pytest should be this sensitive to what other subdirectories there might exist.

This is run under Ubuntu 22.04.3 LTS under WSL.

• a detailed description of the bug or problem you are having
• output of pip list from the virtual environment you are using
• pytest and operating system versions
• minimal example if possible

[bluetech]

What is the reason for having inaccessible files in the workspace? Trying to understand if this is something that you intentionally do.

[jesnie]

It's some Docker containers with some databases we use for testing, that has a mounted volume that ends up belonging to another user. We probably could change that if we were sufficiently motivated, but in either case pytest shouldn't crash just because there's a directory it does not have read access to.

[RonnyPfannschmidt]

@bluetech the regression (that was added by introducing Directories as is) is that we now collect multiple folders just to get to the given testdirs

@jesnie can you try if a collect ignore is a sufficient workaround for now

@bluetech i believe there is a need to ensure collect-towards for testpaths and/or test ids will in fact not look in other places that may trigger permission errors

[jesnie]

Sorry about the late reply, but I've been on a small vacation.
Adding a --ignore for the relevant subdirectory does seem to prevent the crash.

[jesnie]

@RonnyPfannschmidt can you not just put a try ... except around the relevant code and catch any permission errors?