Sphinx 4.2.0 potential incompatibility with ElasticSearch

[FreddieDev]

Describe the bug

Hi all,

My Python project uses:

• elasticsearch (7.14.1)
• elasticsearch-dsl (7.4.0)
• Sphinx to auto-generated documentation.

Issues started when we updated Sphinx from 3.0.4 to 4.2.0.

Example:
Usage of Elastic's Nested object (from elasticsearch_dsl import Nested), results in the Sphinx error:
TypeError: unhashable type: 'Nested'

Interestingly, Nested from the source code it looks like Nested does contain a hash method, however it's hardcoded to always return 0:

If there are further details I can provide, please let me know.
Hoping someone else has ran into this too, or if they can clarify if this is a Sphinx or Elastic issue.

Many thanks

How to Reproduce

My usage was as follows:

class CustomerPackage(InnerDoc):
    """A CustomerPackage Object."""

    @property
    def id(self):
        """Unique GUID for the CustomerPackage."""
        return self.meta.id

    type: Text = Text(analyzer="snowball")
    """ CustomerPackage Type. """

    description: Text = Text(analyzer="snowball")
    """ CustomerPackage Description. """

    lost: bool = Boolean()
    """ Determines if the CustomerPackage was lost in transmission."""

    package_content: Nested(PackageContent) = Nested(PackageContent, multi=False)
    """ Content involved in the message. """

Expected behavior

Prior to the update, Sphinx would build as expected.

Your project

N/A, apologies for lacking detail though I am not permitted to share more source code

Screenshots

No response

OS

Linux

Python version

3.8

Sphinx version

4.2.0

Sphinx extensions

No response

Extra tools

ElasticSearch

Additional context

Elastic ticket:
elastic/elasticsearch-dsl-py#1552

[tk0miya]

Could you make a minimal reproducible example, please? It must be helpful to me.

[FreddieDev]

Sure thing, I will hopefully be able to do this in the new year, then I'll update you.

Thanks for the offer

[FreddieDev]

Hi @tk0miya , I've created a mini project that reproduces the issue:
https://gitlab.com/fchessell/sphinx-bug-demo

I hope this helps and thanks again for reaching out

[FreddieDev]

Is anyone able to assist with this please?

[tk0miya]

Sorry for the late. I'm looking this from now on.

[tk0miya]

Note:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/sphinx/cmd/build.py", line 280, in build_main
    app.build(args.force_all, filenames)
  File "/usr/local/lib/python3.9/site-packages/sphinx/application.py", line 343, in build
    self.builder.build_update()
  File "/usr/local/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 290, in build_update
    self.build(['__all__'], to_build)
  File "/usr/local/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 307, in build
    updated_docnames = set(self.read())
  File "/usr/local/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 414, in read
    self._read_serial(docnames)
  File "/usr/local/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 435, in _read_serial
    self.read_doc(docname)
  File "/usr/local/lib/python3.9/site-packages/sphinx/builders/__init__.py", line 475, in read_doc
    doctree = read_doc(self.app, self.env, self.env.doc2path(docname))
  File "/usr/local/lib/python3.9/site-packages/sphinx/io.py", line 189, in read_doc
    pub.publish()
  File "/usr/local/lib/python3.9/site-packages/docutils/core.py", line 217, in publish
    self.document = self.reader.read(self.source, self.parser,
  File "/usr/local/lib/python3.9/site-packages/sphinx/io.py", line 109, in read
    self.parse()
  File "/usr/local/lib/python3.9/site-packages/docutils/readers/__init__.py", line 77, in parse
    self.parser.parse(self.input, document)
  File "/usr/local/lib/python3.9/site-packages/sphinx/parsers.py", line 101, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 170, in run
    results = StateMachineWS.run(self, input_lines, input_offset,
  File "/usr/local/lib/python3.9/site-packages/docutils/statemachine.py", line 241, in run
    context, next_state, result = self.check_line(
  File "/usr/local/lib/python3.9/site-packages/docutils/statemachine.py", line 459, in check_line
    return method(match, context, next_state)
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 2769, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 393, in new_subsection
    newabsoffset = self.nested_parse(
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 281, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/usr/local/lib/python3.9/site-packages/docutils/statemachine.py", line 241, in run
    context, next_state, result = self.check_line(
  File "/usr/local/lib/python3.9/site-packages/docutils/statemachine.py", line 459, in check_line
    return method(match, context, next_state)
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 2769, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 327, in section
    self.new_subsection(title, lineno, messages)
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 393, in new_subsection
    newabsoffset = self.nested_parse(
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 281, in nested_parse
    state_machine.run(block, input_offset, memo=self.memo,
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 196, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/usr/local/lib/python3.9/site-packages/docutils/statemachine.py", line 241, in run
    context, next_state, result = self.check_line(
  File "/usr/local/lib/python3.9/site-packages/docutils/statemachine.py", line 459, in check_line
    return method(match, context, next_state)
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 2342, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 2354, in explicit_construct
    return method(self, expmatch)
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 2096, in directive
    return self.run_directive(
  File "/usr/local/lib/python3.9/site-packages/docutils/parsers/rst/states.py", line 2146, in run_directive
    result = directive_instance.run()
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/autodoc/directive.py", line 162, in run
    documenter.generate(more_content=self.content)
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/autodoc/__init__.py", line 983, in generate
    self.document_members(all_members)
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/autodoc/__init__.py", line 860, in document_members
    documenter.generate(
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/autodoc/__init__.py", line 1773, in generate
    return super().generate(more_content=more_content,
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/autodoc/__init__.py", line 983, in generate
    self.document_members(all_members)
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/autodoc/__init__.py", line 1764, in document_members
    super().document_members(all_members)
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/autodoc/__init__.py", line 860, in document_members
    documenter.generate(
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/autodoc/__init__.py", line 973, in generate
    self.add_directive_header(sig)
  File "/usr/local/lib/python3.9/site-packages/sphinx/ext/autodoc/__init__.py", line 2607, in add_directive_header
    objrepr = stringify_typehint(annotations.get(self.objpath[-1]))
  File "/usr/local/lib/python3.9/site-packages/sphinx/util/typing.py", line 324, in stringify
    elif annotation in INVALID_BUILTIN_CLASSES:
TypeError: unhashable type: 'Keyword'

[tk0miya]

I posted #10183 to fix this bug. Thank you for reporting!