Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fixtures: fix quadratic behavior in the number of autouse fixtures #7931

Merged
merged 3 commits into from
Oct 24, 2020
Merged

fixtures: fix quadratic behavior in the number of autouse fixtures #7931

merged 3 commits into from
Oct 24, 2020

Conversation

bluetech
Copy link
Member

Fixes #4824. This contains 3 commits which optimize the slow parts of xunit/autouse fixtes, the last commit fixes the remaining quadratic behavior (continuation of #7929). Please see the commit messages for details.

On the bench/xunit.py benchmark, --collect-only, before:

         51895636 function calls (51265006 primitive calls) in 25.618 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000   25.101   25.101 main.py:320(pytest_collection)
        1    0.000    0.000   25.101   25.101 main.py:567(perform_collect)
80560/556    0.020    0.000   24.472    0.044 {method 'extend' of 'list' objects}
85001/15001    0.148    0.000   24.468    0.002 main.py:785(genitems)
    10002    0.044    0.000   23.705    0.002 runner.py:455(collect_one_node)
    10002    0.042    0.000   23.281    0.002 runner.py:340(pytest_make_collect_report)
    10002    0.068    0.000   23.201    0.002 runner.py:298(from_call)
    10002    0.017    0.000   23.087    0.002 runner.py:341(<lambda>)
     5001    0.164    0.000   21.569    0.004 python.py:412(collect)
     5000    0.019    0.000   20.634    0.004 python.py:862(collect)
    30003    0.098    0.000   18.282    0.001 python.py:218(pytest_pycollect_makeitem)
    30000    0.157    0.000   17.843    0.001 python.py:450(_genfunctions)
    30000    0.342    0.000   16.682    0.001 python.py:1615(from_parent)
    40001    0.089    0.000   16.454    0.000 nodes.py:183(from_parent)
    40002    0.100    0.000   16.365    0.000 nodes.py:102(_create)
    30000    0.351    0.000   16.186    0.001 python.py:1553(__init__)
    15000    0.116    0.000   14.607    0.001 fixtures.py:1439(getfixtureinfo)
    15000    0.136    0.000   13.693    0.001 fixtures.py:1492(getfixtureclosure)
    15000    7.966    0.001   13.284    0.001 fixtures.py:1479(_getautousenames)
 37643622    5.336    0.000    5.336    0.000 {method 'startswith' of 'str' objects}
    20018    0.032    0.000    2.534    0.000 {method 'sort' of 'list' objects}
    20000    0.025    0.000    2.493    0.000 python.py:443(sort_key)
    20000    0.060    0.000    2.467    0.000 python.py:328(reportinfo)
        1    0.000    0.000    2.069    2.069 python.py:502(collect)
    20000    0.071    0.000    2.058    0.000 code.py:1160(getfslineno)
     5000    0.008    0.000    1.173    0.000 source.py:115(findsource)
     5000    0.049    0.000    1.166    0.000 inspect.py:772(findsource)

After:

         14681410 function calls (14075780 primitive calls) in 12.041 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000   11.525   11.525 main.py:320(pytest_collection)
        1    0.000    0.000   11.525   11.525 main.py:567(perform_collect)
55560/556    0.015    0.000   10.845    0.020 {method 'extend' of 'list' objects}
85001/15001    0.140    0.000   10.841    0.001 main.py:785(genitems)
    10002    0.042    0.000   10.108    0.001 runner.py:455(collect_one_node)
    10002    0.039    0.000    9.386    0.001 runner.py:340(pytest_make_collect_report)
    10002    0.061    0.000    9.312    0.001 runner.py:298(from_call)
    10002    0.015    0.000    9.209    0.001 runner.py:341(<lambda>)
     5001    0.156    0.000    7.741    0.002 python.py:412(collect)
     5000    0.018    0.000    6.777    0.001 python.py:862(collect)
    30003    0.091    0.000    4.552    0.000 python.py:218(pytest_pycollect_makeitem)
    30000    0.147    0.000    4.137    0.000 python.py:450(_genfunctions)
    40001    0.076    0.000    3.164    0.000 nodes.py:175(from_parent)
    30000    0.076    0.000    3.124    0.000 python.py:1615(from_parent)
    40002    0.081    0.000    3.088    0.000 nodes.py:94(_create)
    30000    0.253    0.000    2.926    0.000 python.py:1553(__init__)
    20018    0.030    0.000    2.459    0.000 {method 'sort' of 'list' objects}
    20000    0.024    0.000    2.423    0.000 python.py:443(sort_key)
    20000    0.058    0.000    2.397    0.000 python.py:328(reportinfo)
        1    0.000    0.000    2.074    2.074 python.py:502(collect)
    20000    0.067    0.000    2.002    0.000 code.py:1160(getfslineno)
    15000    0.107    0.000    1.669    0.000 fixtures.py:1440(getfixtureinfo)
     5000    0.008    0.000    1.177    0.000 source.py:115(findsource)
     5000    0.049    0.000    1.169    0.000 inspect.py:772(findsource)
    20026    0.098    0.000    1.078    0.000 compat.py:112(getfuncargnames)

which is the usual slowness :)

nicoddemus
nicoddemus approved these changes Oct 24, 2020
View reviewed changes
Copy link
Member

@nicoddemus nicoddemus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work @bluetech!

Left two minor comments, feel free to ignore them. 👍

src/_pytest/nodes.py Outdated Show resolved Hide resolved
src/_pytest/nodes.py Outdated
@lru_cache(maxsize=None)
def _splitnode(nodeid: str) -> Tuple[str, ...]:
"""Split a nodeid into constituent 'parts'.
def getparentnodeids(nodeid: str) -> Iterator[str]:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice function and docs!

Did you try to measure with @lru_cache being applied to it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, but I think it shouldn't be a hot spot anymore, and shouldn't be called with the same nodeid a million times like before. But if it does show up in a profile again it might make sense.

@bluetech
fixtures: use a faster replacement for ischildnode
aa0e2d6 
ischildnode can be quite hot in some cases involving many fixtures.
However it is always used in a way that the nodeid is constant and the
baseid is iterated. So we can save work by pre-computing the parents of
the nodeid and use a simple containment test.

The `_getautousenames` function has the same stuff open-coded, so change
it to use the new function as well.
@bluetech
fixtures: change _getautousenames to an iterator
d6becfa 
This reads better.
@bluetech
fixtures: fix quadratic behavior in the number of autouse fixtures
470ea50 
It turns out all autouse fixtures are kept in a global list, and thinned
out for a particular node using a linear scan of the entire list each
time.

Change the list to a dict, and only take the nodes we need.
@bluetech bluetech merged commit 65e6e39 into pytest-dev:master Oct 24, 2020
@bluetech bluetech deleted the xunit-quadratic-2 branch November 6, 2020 21:06
@AnesBenmerzoug AnesBenmerzoug mentioned this pull request Dec 14, 2020
Closed
@nicoddemus nicoddemus mentioned this pull request Mar 4, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nicoddemus nicoddemus nicoddemus approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Very slow test setup in pytest 4.2.0+
2 participants
@bluetech @nicoddemus