Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sphinx~=7.3.0 breaks sphinx-immaterial #345

Open
JacobChesslo opened this issue Apr 17, 2024 · 10 comments · May be fixed by #346
Open

sphinx~=7.3.0 breaks sphinx-immaterial #345

JacobChesslo opened this issue Apr 17, 2024 · 10 comments · May be fixed by #346
Labels
bug Something isn't working

Comments

@JacobChesslo
Copy link

JacobChesslo commented Apr 17, 2024

Sphinx recently updated to 7.3.*.

This came with some public interface breakages, which sphinx-immaterial uses, detailed in this issue: sphinx-doc/sphinx#12295.

Additionally, sphinx-immaterial uses some more private-leaning methods and functions from sphinx (sphinx-doc/sphinx#12297), which have the potential to be moved/broken/removed in the future.

@2bndy5
Copy link
Collaborator

2bndy5 commented Apr 17, 2024

sphinx-immaterial uses some more private-leaning methods and functions from sphinx

As for using private APi in sphinx, we really can't avoid this without removing some theme-specific features. It would nice if we didn't have to, but Sphinx domains are a bit rigid. See #113 (comment)

ImportError: cannot import name 'PyTypedField' from 'sphinx.domains.python'

As for this, I don't see PyTypedField on the list of Deprecated API. I do see that it is now re-exported in v7.3.5.

@AA-Turner I don't think it was a good idea to move public API into private modules. That move tells me that only sphinx.ext.* is allowed to use those API members, which is not very practical. Think about all the other third-party sphinx extensions posing an alternative implementation to autodoc (and not just for the python domain).

AttributeError: module 'sphinx.domains.cpp' has no attribute 'ASTOperator'

It looks like the domain refactor in sphinx-doc/sphinx@v7.2.6...v7.3.5 was not specific to just the python domain. Our docs now fail (with v7.3.5) about cpp domain as well. This one might take us a while to resolve...

@2bndy5 2bndy5 added the bug Something isn't working label Apr 17, 2024
tobias-urdin added a commit to tobias-urdin/documentation that referenced this issue Apr 17, 2024
We need to pin sphinx<7.3.0 because the
sphinx-immterial theme doesn't work with
it yet [1].

[1] jbms/sphinx-immaterial#345
@JacobChesslo
Copy link
Author

I was able to get a sphinx-immaterial build working nominally with just an import of most/all things removed in pre 7.3.0 sphinx.domains.{c,cop,python}, sphinx-doc/sphinx#12297, so that's hopeful that the issue is restricted to a subset of those changes.

@AA-Turner
Copy link

You'll need to change

sphinx.domains.python._parse_arglist = parse_arglist

to

try:
    if sphinx.version_info >= (7, 3):
        sphinx.domains.python._annotation._parse_arglist = parse_arglist
        sphinx.domains.python._object._parse_arglist = parse_arglist
    else:
        sphinx.domains.python._parse_arglist = parse_arglist
except AttributeError:
    # log some warning rather than crashing

@AA-Turner
Copy link

Sphinx 7.3.6 has been released with fixes.

A

@2bndy5
Copy link
Collaborator

2bndy5 commented Apr 18, 2024

jinja_context conains the module sys as a value

jinja_contexts = {
"sys": {"sys": sys},

which is causing:

pickling environment... WARNING: cannot cache unpickable configuration value: 'jinja_contexts' (because it contains a function, class, or module object)

This poses a problem in

.. jinja:: sys
{%- set example_python_apigen_modules = {
"my_module": "my_api/",
"my_other_module": "other_api/my_other_module.",
}
%}
{%- set example_python_apigen_objects = [
("my_module.foo", ""),
("my_module.Foo", ""),
("my_module.Foo.method", ""),
("my_module.Foo.__init__", "json"),
("my_module.Foo.__init__", "values"),
("my_module.Bar", ""),
("my_other_module.Baz", ""),
]
%}
{%- set python_apigen_get_docname = sys.modules["sphinx_immaterial.apidoc.python.apigen"]._get_docname %}
.. list-table::
:widths: auto
:header-rows: 1
* - Python object
- Overload
- Document (case-sensitive)
- Document (case-insensitive)
{%- for full_name, overload_id in example_python_apigen_objects %}
* - :python:`{{ full_name }}`
- {{ "``" + overload_id + "``" if overload_id else "" }}
- :file:`{{ python_apigen_get_docname(example_python_apigen_modules, full_name, overload_id, False) }}`
- :file:`{{ python_apigen_get_docname(example_python_apigen_modules, full_name, overload_id, True) }}`
{%- endfor %}

where most of it can be generated from conf.py, but using a function extracted from a sys path is a unique obstacle:
{%- set python_apigen_get_docname = sys.modules["sphinx_immaterial.apidoc.python.apigen"]._get_docname %}

- :file:`{{ python_apigen_get_docname(example_python_apigen_modules, full_name, overload_id, False) }}`
- :file:`{{ python_apigen_get_docname(example_python_apigen_modules, full_name, overload_id, True) }}`

Other jinja_context["sys"] usage is in

.. jinja:: sys
{%- for x in sys.modules["sphinx_immaterial.apidoc.python.type_annotation_transforms"].TYPING_NAMES %}
- :python:`{{ x }}` -> :python:`typing.{{ x }}`
{%- endfor %}

But that should be an easier fix.

@2bndy5 2bndy5 linked a pull request Apr 18, 2024 that will close this issue
@2bndy5
Copy link
Collaborator

2bndy5 commented Apr 18, 2024

Geez. Even the typing for public API changed quite significantly.

Sphinx.add_config_value (sphinx-doc/sphinx@259118d)

  • no longer takes a bool | str as the rebuild param (didn't know that usage was deprecated -- its not in the deprecated list)
  • now uses narrow typing for the types param. According to mypy, all our specified types are wrong (even though they work with v7.3.x)

I'm not going to lie. This one is a pain in the ass. There's usually one or two things we need to update for a minor version bump... 😞 Also, v7.3 is ripe with merge commits in the master branch (which makes it harder to git blame a PR or instigating issue when looking for rationality). It would be a good idea to start putting a reference to a discussion (PR or issue) in the merge commit's description.

@AA-Turner
Copy link

Thank you for the feedback, Brendan (@2bndy5).

Even the typing for [Sphinx.add_config_value] changed quite significantly.

The intent here was to make the type hints (intended usage) stricter, whilst the runtime acceptable values didn't change. Any examples of mypy failing would be useful.

Sphinx.add_config_value no longer takes a bool | str as the rebuild param

It still does at runtime, the static typing information has become stricter, though, to reflect intended use.

Sphinx.add_config_value now uses narrow typing for the types param.
According to mypy, all our specified types are wrong (even though they work with v7.3.x)

All previously valid values should still be valid. I'm happy to tweak the type hints if needed (suggestions welcome).

A

@2bndy5
Copy link
Collaborator

2bndy5 commented Apr 19, 2024

All previously valid values should still be valid. I'm happy to tweak the type hints if needed (suggestions welcome).

I don't think Collection[type] satisfies the previous use cases. I got errors saying

error: Argument "types" to "add_config_value" of "Sphinx" has incompatible
type "tuple[type[bool], type[None]]"; expected "type | Collection[type] | ENUM"  [arg-type]
            types=(bool, type(None)),
                  ^~~~~~~~~~~~~~~~~~

When I change Collection[type] to Sequence[type], I no longer get this error from mypy. For now, I've just type: ignored any of our usage that doesn't satisfy Collection[type].

@AA-Turner
Copy link

AA-Turner commented Apr 19, 2024

Collection ought be more general than Sequence (as e.g. a set is a Collection but not a Sequence).

Mypy runs fine for me on the first two of the below three test-cases, and only reports an error for the third -- I suspect this might actually be an error with Mypy's implementation/special casing of NoneType? Certainly from a type theory perspective I think the annotations in add_config_value are correct.

from sphinx.application import Sphinx


def setup(app: Sphinx) -> None:
    app.add_config_value(
        'spam',
        None,
        '',
        (bool, str, type(None))
    )
    app.add_config_value(
        'ham',
        None,
        '',
        (bool, str)
    )
    app.add_config_value(
        'ham',
        None,
        '',
        (bool, type(None))  # mypy error: incompatible type ...
    )

Sorry to cause trouble, though, it isn't intended and thank you for your co-operation in trying to improve the situation.

A

@2bndy5
Copy link
Collaborator

2bndy5 commented Apr 19, 2024

Thanks for your consideration. For passers by, this error was reported with mypy v1.8.0 and v1.9.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants