Skip to content

Commit

Permalink
dev: Require flake8 ≥4.0.0
Browse files Browse the repository at this point in the history
The biggest impact of this change is on Python 3.7 where we were seeing
CI test failures.¹  Those now pass again.  The story is complicated.
Let's start with the facts.

The recent release of importlib-metadata 5.0.0 removed a previously
deprecated entry points API.²

This API is used by flake8.

flake8 4.0.0 onwards declares (more or less):

    importlib-metadata <4.3; python_version < "3.8"

…while prior versions, flake8 3.9.2 and earlier, declared:

    importlib-metadata; python_version < "3.8"

Note that flake8 3.9.2 and earlier still require in practice the upper
bound on the importlib-metadata version, they just don't express it in
package metadata.  (This is what's known as foreshadowing…)

Meanwhile, Sphinx 4.4.0 onwards declares (more or less):

    importlib-metadata >=4.4; python_version < '3.10'

Note that on Python 3.7 and earlier, the latest versions of both flake8
and Sphinx have conflicting version requirements for importlib-metadata.

So when Pip on 3.7 goes to install both, it has to resolve the issue by
downgrading either flake8 or Sphinx until it finds a dependency
solution.  For whatever reason, it choose to keep the newer versions of
Sphinx and importlib-metadata and downgrade flake8.  So it walks back in
flake8 versions until it eventually finds 3.9.2 without the
importlib-metadata version pin.  Pip is satisfied, but although Pip
can't know it, flake8 won't work with importlib-metadata 5.0.0.

By pinning flake8 ≥4.0.0, we're hinting to Pip that only those versions
accurately declare their dependencies given the current situation.  Pip
ultimately can use this better dependency information solve the
importlib-metadata conflict by walking back to a version of Sphinx
≤4.4.0 (which it turns out, don't use importlib-metadata at all).

The flake8 breakage doesn't arise on Python 3.6 because support for that
Python version was dropped by importlib-metadata ≥4.9.0, so the removed
API in 5.0.0 isn't seen.  Without our flake8 pin, Pip still resolves the
conflict by walking flake8 back to 3.9.2 and keeping the latest Sphinx
plus importlib-metadata 4.8.3.  While that importlib-metadata version is
declared as unsupported in later version of flake8, it seems to work
fine in practice. ¯\_(ツ)_/¯

On Python ≥3.8, the importlib-metadata conflict doesn't exist,
presumably because flake8 uses the importlib.metadata standard library
instead.

We'll still build our production docs on RTD with the latest version of
Sphinx, because we use Python 3.10 there, and the downgrading of Sphinx
is limited to 3.7.

(If some of this is making your head ring but also ringing some bells,
this isn't the first time importlib-metadata's development choices and
flake8's development choices have had fallout for us.³)

¹ python/importlib_metadata#409 (comment)
² python/importlib_metadata#405
³ "dev: Ignore a deprecation warning generated by flake8's usage of importlib_metadata" (1a04901)
  • Loading branch information
tsibley committed Oct 4, 2022
1 parent 111a15d commit 352b8d7
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion setup.py
Expand Up @@ -124,7 +124,7 @@ def find_namespaced_packages(namespace):
extras_require = {
"dev": [
"docutils<0.16",
"flake8",
"flake8 >=4.0.0",
"mypy",
"nextstrain-sphinx-theme>=2022.5",
"pytest; python_version != '3.9'",
Expand Down

0 comments on commit 352b8d7

Please sign in to comment.