Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: python: skip work pytest_pycollect_makeitem work on certain names #7671

Merged
merged 1 commit into from
Aug 27, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 6 additions & 0 deletions changelog/7671.trivial.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
When collecting tests, pytest finds test classes and functions by examining the
attributes of python objects (modules, classes and instances). To speed up this
process, pytest now ignores builtin attributes (like ``__class__``,
``__delattr__`` and ``__new__``) without consulting the ``python_classes`` and
``python_functions`` configuration options and without passing them to plugins
using the ``pytest_pycollect_makeitem`` hook.
23 changes: 23 additions & 0 deletions src/_pytest/python.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
import itertools
import os
import sys
import types
import typing
import warnings
from collections import Counter
Expand Down Expand Up @@ -343,6 +344,26 @@ def reportinfo(self) -> Tuple[Union[py.path.local, str], int, str]:
return fspath, lineno, modpath


# As an optimization, these builtin attribute names are pre-ignored when
# iterating over an object during collection -- the pytest_pycollect_makeitem
# hook is not called for them.
# fmt: off
class _EmptyClass: pass # noqa: E701
IGNORED_ATTRIBUTES = frozenset.union( # noqa: E305
frozenset(),
# Module.
dir(types.ModuleType("empty_module")),
# Some extra module attributes the above doesn't catch.
{"__builtins__", "__file__", "__cached__"},
# Class.
dir(_EmptyClass),
# Instance.
dir(_EmptyClass()),
)
del _EmptyClass
# fmt: on


class PyCollector(PyobjMixin, nodes.Collector):
def funcnamefilter(self, name: str) -> bool:
return self._matches_prefix_or_glob_option("python_functions", name)
Expand Down Expand Up @@ -404,6 +425,8 @@ def collect(self) -> Iterable[Union[nodes.Item, nodes.Collector]]:
# Note: seems like the dict can change during iteration -
# be careful not to remove the list() without consideration.
for name, obj in list(dic.items()):
if name in IGNORED_ATTRIBUTES:
continue
if name in seen:
continue
seen.add(name)
Expand Down
28 changes: 28 additions & 0 deletions testing/python/collect.py
Original file line number Diff line number Diff line change
Expand Up @@ -885,6 +885,34 @@ def test_something():
result = testdir.runpytest_subprocess()
result.stdout.fnmatch_lines(["*1 passed*"])

def test_early_ignored_attributes(self, testdir: Testdir) -> None:
"""Builtin attributes should be ignored early on, even if
configuration would otherwise allow them.

This tests a performance optimization, not correctness, really,
although it tests PytestCollectionWarning is not raised, while
it would have been raised otherwise.
"""
testdir.makeini(
"""
[pytest]
python_classes=*
python_functions=*
"""
)
testdir.makepyfile(
"""
class TestEmpty:
pass
test_empty = TestEmpty()
def test_real():
pass
"""
)
items, rec = testdir.inline_genitems()
assert rec.ret == 0
assert len(items) == 1


def test_setup_only_available_in_subdir(testdir):
sub1 = testdir.mkpydir("sub1")
Expand Down