4.1.0 regression: Could not import extension sphinx.builders.epub3 (exception: cannot import name 'RemovedInSphinx40Warning' from 'sphinx.deprecation'

[webknjaz]

Describe the bug

So this morning I've got an automatic dependabot dependency pin update PR changing Sphinx from v4.0.3 to v4.1.0. This caused the build to error out with the text as in $sbj. It is notable that the builder that causes the error is epub3 while the builder invoked via CLI is html.

How to Reproduce

$ git clone https://github.com/ansible/pylibssh
$ cd pylibssh
$ pip install tox
$ tox -e build-docs

Expected behavior

No errors.

Your project

https://github.com/ansible/pylibssh

Screenshots

No response

OS

GHA CI / Ubuntu 20.04

Python version

CPython v3.9.5

Sphinx version

v4.1.0

Sphinx extensions

sphinx.ext.autodoc, sphinx.ext.autosectionlabel, sphinx.ext.doctest, sphinx.ext.extlinks, sphinx.ext.intersphinx, sphinx.ext.todo, sphinx.ext.coverage, sphinx.ext.viewcode, sphinxcontrib.towncrier, myst_parser

Extra tools

No response

Additional context

https://github.com/ansible/pylibssh/pull/238/checks?check_run_id=3044079226#step:11:46

[coordt]

It appears that the loose version requirements for Sphinx's dependencies are causing problems.

A work-around I found was:

pip install -U sphinxcontrib-htmlhelp
pip install -U sphinxcontrib-serializinghtml

[astrojuanlu]

I see the class was removed in #9263. As Sphinx follows SemVer, I think this class should not be removed until version 5?

[jakobandersen]

I see the class was removed in #9263. As Sphinx follows SemVer, I think this class should not be removed until version 5?

Well, I don't think that class is considered to be in the public API. But anyway, the error is during import of sphinx.builders.epub3, which was updated already for v4 to no longer contain the deprecated code. Not sure what is going on.

[ericholscher]

Yea, it looks like the issue is with sphinxcontrib packages that try to do this import. From a broken virtualenv:

-> grep -r RemovedInSphinx40Warning *
sphinxcontrib/htmlhelp/__init__.py:from sphinx.deprecation import RemovedInSphinx40Warning
sphinxcontrib/htmlhelp/__init__.py:                      RemovedInSphinx40Warning)
Binary file sphinxcontrib/htmlhelp/__pycache__/__init__.cpython-38.pyc matches
sphinxcontrib/serializinghtml/__init__.py:from sphinx.deprecation import RemovedInSphinx40Warning, deprecated_alias
sphinxcontrib/serializinghtml/__init__.py:                 RemovedInSphinx40Warning)
Binary file sphinxcontrib/serializinghtml/__pycache__/__init__.cpython-38.pyc matches

The workaround that @coordt mentioned should fix it, but the real fix is a 4.1.1 release that pins the proper version of these dependencies, so they will be forced to upgrade to a working version. @tk0miya ^^

[ericholscher]

I put together a PR with a proposed fix: #9439