Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove compatibility shims for entry points. #405

Merged
merged 8 commits into from Oct 2, 2022

Conversation

jaraco
Copy link
Member

@jaraco jaraco commented Oct 1, 2022

@woutdenolf
Copy link

woutdenolf commented Oct 3, 2022

Starts to be rather cumbersome to support python>=3.7

import sys

try:
    from importlib.metadata import entry_points
except ImportError:
    from importlib_metadata import entry_points

if sys.version_info >= (3,10):
    _entry_points = entry_points(group=namespace)
else:
    try:
        _entry_points = entry_points().get(namespace, [])
    except AttributeError:
        _entry_points = entry_points().select(group=namespace)

@jaraco
Copy link
Member Author

jaraco commented Oct 3, 2022

That's one way to do it. Other options include:

  • Rely on importlib_metadata >= 3.6 for Python < 3.10. As a general rule, you should adopt importlib_metadata any time there's behavior from a future Python version but your application supports older Python versions.
  • Use backports.entry_points_selectable (provides forward compatibility to Python 2.7+).
  • Bypass entry_points altogether and re-implement its behavior using more fundamental interfaces (flake8 took this approach; I don't recommend it).

@woutdenolf
Copy link

Thanks for the feedback!

Rely on importlib_metadata >= 3.6 for Python < 3.10

Many projects just need importlib.metadata so they have importlib-metadata; python_version < '3.8'. Once python 3.7 gets dropped it should become simpler.

Python 3.10 still supports entry_points().select(group=namespace) so using entry_points(group=namespace) is not mandatory.

tsibley added a commit to nextstrain/cli that referenced this pull request Oct 4, 2022
The biggest impact of this change is on Python 3.7 where we were seeing
CI test failures.¹  Those now pass again.  The story is complicated.
Let's start with the facts.

The recent release of importlib-metadata 5.0.0 removed a previously
deprecated entry points API.²

This API is used by flake8.

flake8 4.0.0 onwards declares (more or less):

    importlib-metadata <4.3; python_version < "3.8"

…while prior versions, flake8 3.9.2 and earlier, declared:

    importlib-metadata; python_version < "3.8"

Note that flake8 3.9.2 and earlier still require in practice the upper
bound on the importlib-metadata version, they just don't express it in
package metadata.  (This is what's known as foreshadowing…)

Meanwhile, Sphinx 4.4.0 onwards declares (more or less):

    importlib-metadata >=4.4; python_version < '3.10'

Note that on Python 3.7 and earlier, the latest versions of both flake8
and Sphinx have conflicting version requirements for importlib-metadata.

So when Pip on 3.7 goes to install both, it has to resolve the issue by
downgrading either flake8 or Sphinx until it finds a dependency
solution.  For whatever reason, it choose to keep the newer versions of
Sphinx and importlib-metadata and downgrade flake8.  So it walks back in
flake8 versions until it eventually finds 3.9.2 without the
importlib-metadata version pin.  Pip is satisfied, but although Pip
can't know it, flake8 won't work with importlib-metadata 5.0.0.

By pinning flake8 ≥4.0.0, we're hinting to Pip that only those versions
accurately declare their dependencies given the current situation.  Pip
ultimately can use this better dependency information solve the
importlib-metadata conflict by walking back to a version of Sphinx
≤4.4.0 (which it turns out, don't use importlib-metadata at all).

The flake8 breakage doesn't arise on Python 3.6 because support for that
Python version was dropped by importlib-metadata ≥4.9.0, so the removed
API in 5.0.0 isn't seen.  Without our flake8 pin, Pip still resolves the
conflict by walking flake8 back to 3.9.2 and keeping the latest Sphinx
plus importlib-metadata 4.8.3.  While that importlib-metadata version is
declared as unsupported in later version of flake8, it seems to work
fine in practice. ¯\_(ツ)_/¯

On Python ≥3.8, the importlib-metadata conflict doesn't exist,
presumably because flake8 uses the importlib.metadata standard library
instead.

We'll still build our production docs on RTD with the latest version of
Sphinx, because we use Python 3.10 there, and the downgrading of Sphinx
is limited to 3.7.

(If some of this is making your head ring but also ringing some bells,
this isn't the first time importlib-metadata's development choices and
flake8's development choices have had fallout for us.³)

¹ python/importlib_metadata#409 (comment)
² python/importlib_metadata#405
³ "dev: Ignore a deprecation warning generated by flake8's usage of importlib_metadata" (1a04901)
tsibley added a commit to nextstrain/cli that referenced this pull request Oct 5, 2022
The biggest impact of this change is on Python 3.7 where we were seeing
CI test failures.¹  Those now pass again.  The story is complicated.
Let's start with the facts.

The recent release of importlib-metadata 5.0.0 removed a previously
deprecated entry points API.²

This API is used by flake8.

flake8 4.0.0 onwards declares (more or less):

    importlib-metadata <4.3; python_version < "3.8"

…while prior versions, flake8 3.9.2 and earlier, declared:

    importlib-metadata; python_version < "3.8"

Note that flake8 3.9.2 and earlier still require in practice the upper
bound on the importlib-metadata version, they just don't express it in
package metadata.  (This is what's known as foreshadowing…)

Meanwhile, Sphinx 4.4.0 onwards declares (more or less):

    importlib-metadata >=4.4; python_version < '3.10'

Note that on Python 3.7 and earlier, the latest versions of both flake8
and Sphinx have conflicting version requirements for importlib-metadata.

So when Pip on 3.7 goes to install both, it has to resolve the issue by
downgrading either flake8 or Sphinx until it finds a dependency
solution.  For whatever reason, it choose to keep the newer versions of
Sphinx and importlib-metadata and downgrade flake8.  So it walks back in
flake8 versions until it eventually finds 3.9.2 without the
importlib-metadata version pin.  Pip is satisfied, but although Pip
can't know it, flake8 won't work with importlib-metadata 5.0.0.

By pinning flake8 ≥4.0.0, we're hinting to Pip that only those versions
accurately declare their dependencies given the current situation.  Pip
ultimately can use this better dependency information solve the
importlib-metadata conflict by walking back to a version of Sphinx
≤4.4.0 (which it turns out, don't use importlib-metadata at all).

The flake8 breakage doesn't arise on Python 3.6 because support for that
Python version was dropped by importlib-metadata ≥4.9.0, so the removed
API in 5.0.0 isn't seen.  Without our flake8 pin, Pip still resolves the
conflict by walking flake8 back to 3.9.2 and keeping the latest Sphinx
plus importlib-metadata 4.8.3.  While that importlib-metadata version is
declared as unsupported in later version of flake8, it seems to work
fine in practice. ¯\_(ツ)_/¯

On Python ≥3.8, the importlib-metadata conflict doesn't exist,
presumably because flake8 uses the importlib.metadata standard library
instead.

We'll still build our production docs on RTD with the latest version of
Sphinx, because we use Python 3.10 there, and the downgrading of Sphinx
is limited to 3.7.

(If some of this is making your head ring but also ringing some bells,
this isn't the first time importlib-metadata's development choices and
flake8's development choices have had fallout for us.³)

¹ python/importlib_metadata#409 (comment)
² python/importlib_metadata#405
³ "dev: Ignore a deprecation warning generated by flake8's usage of importlib_metadata" (1a04901)
pvital added a commit to instana/python-sensor that referenced this pull request Mar 8, 2023
importlib_metadata package removed deprecated entry_point interfaces on
version 5.0.0 [1] , and this change impacts celery >= 5.2.7 [2] running
with python 3.7.X (it doesn't impact >= 3.8). For this reason, we
control celery and importlib_metadata versions on python 3.7
environments.
[1] python/importlib_metadata#405
[2] celery/celery#7783

Signed-off-by: Paulo Vital <paulo.vital@ibm.com>
pvital added a commit to instana/python-sensor that referenced this pull request Mar 8, 2023
importlib_metadata package removed deprecated entry_point interfaces on
version 5.0.0 [1] , and this change impacts celery >= 5.2.7 [2] running
with python 3.7.X (it doesn't impact >= 3.8). For this reason, we
control celery and importlib_metadata versions on python 3.7
environments.
[1] python/importlib_metadata#405
[2] celery/celery#7783

Signed-off-by: Paulo Vital <paulo.vital@ibm.com>
Ferenc- pushed a commit to instana/python-sensor that referenced this pull request Mar 8, 2023
importlib_metadata package removed deprecated entry_point interfaces on
version 5.0.0 [1] , and this change impacts celery >= 5.2.7 [2] running
with python 3.7.X (it doesn't impact >= 3.8). For this reason, we
control celery and importlib_metadata versions on python 3.7
environments.
[1] python/importlib_metadata#405
[2] celery/celery#7783

Signed-off-by: Paulo Vital <paulo.vital@ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants