LaTeX docs build ends in failure with Sphinx 4.5.0

[befeleme]

Describe the bug

I've built coq against Sphinx 4.5.0 as a part of Fedora's integration. The LaTeX part of documentation failed to build.
Judging by the traceback, this is possibly a Sphinx regression, as in_desc_signature was only introduced lately in this commit: 6768577.

Traceback:

Running Sphinx v4.5.0
making output directory... done
loading pickled environment... checking bibtex cache... up to date
done
Copying /builddir/build/BUILD/coq-8.15.1/doc/sphinx/index.latex.rst to /builddir/build/BUILD/coq-8.15.1/doc/sphinx/index.rst
Copying /builddir/build/BUILD/coq-8.15.1/doc/sphinx/zebibliography.latex.rst to /builddir/build/BUILD/coq-8.15.1/doc/sphinx/zebibliography.rst
building [mo]: targets for 0 po files that are out of date
building [latex]: all documents
updating environment: 0 added, 2 changed, 0 removed
reading sources... [ 50%] index
reading sources... [100%] zebibliography
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
processing CoqRefMan.tex... index language/core/index <snip> changes zebibliography 
resolving references...
done
writing... failed
Exception occurred:
  File "/usr/lib/python3.10/site-packages/sphinx/writers/latex.py", line 759, in visit_desc_content
    assert self.in_desc_signature
AssertionError

Extended traceback:

mock-chroot> sh-5.1# cat sphinx-err-q9izokdn.log 
# Sphinx version: 4.5.0
# Python version: 3.10.4 (CPython)
# Docutils version: 0.17.1 release
# Jinja2 version: 3.0.3
# Last messages:
#   addendum/parallel-proof-processing
#   appendix/history-and-changes/index
#   history
#   changes
#   zebibliography
#   
#   resolving references...
#   done
#   writing...
#   failed
# Loaded extensions:
#   sphinx.ext.mathjax (4.5.0) from /usr/lib/python3.10/site-packages/sphinx/ext/mathjax.py
#   sphinxcontrib.applehelp (1.0.2) from /usr/lib/python3.10/site-packages/sphinxcontrib/applehelp/__init__.py
#   sphinxcontrib.devhelp (1.0.2) from /usr/lib/python3.10/site-packages/sphinxcontrib/devhelp/__init__.py
#   sphinxcontrib.htmlhelp (2.0.0) from /usr/lib/python3.10/site-packages/sphinxcontrib/htmlhelp/__init__.py
#   sphinxcontrib.serializinghtml (1.1.5) from /usr/lib/python3.10/site-packages/sphinxcontrib/serializinghtml/__init__.py
#   sphinxcontrib.qthelp (1.0.3) from /usr/lib/python3.10/site-packages/sphinxcontrib/qthelp/__init__.py
#   alabaster (0.7.12) from /usr/lib/python3.10/site-packages/alabaster/__init__.py
#   sphinx.ext.ifconfig (4.5.0) from /usr/lib/python3.10/site-packages/sphinx/ext/ifconfig.py
#   sphinx.ext.todo (4.5.0) from /usr/lib/python3.10/site-packages/sphinx/ext/todo.py
#   sphinxcontrib.bibtex (2.4.1) from /usr/lib/python3.10/site-packages/sphinxcontrib/bibtex/__init__.py
#   coqrst.coqdomain (0.1) from /builddir/build/BUILD/coq-8.15.1/doc/tools/coqrst/coqdomain.py
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/sphinx/cmd/build.py", line 276, in build_main
    app.build(args.force_all, filenames)
  File "/usr/lib/python3.10/site-packages/sphinx/application.py", line 330, in build
    self.builder.build_update()
  File "/usr/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 283, in build_update
    self.build(['__all__'], to_build)
  File "/usr/lib/python3.10/site-packages/sphinx/builders/__init__.py", line 350, in build
    self.write(docnames, list(updated_docnames), method)
  File "/usr/lib/python3.10/site-packages/sphinx/builders/latex/__init__.py", line 300, in write
    docwriter.write(doctree, destination)
  File "/usr/lib/python3.10/site-packages/docutils/writers/__init__.py", line 78, in write
    self.translate()
  File "/usr/lib/python3.10/site-packages/sphinx/writers/latex.py", line 92, in translate
    self.document.walkabout(visitor)
  File "/usr/lib/python3.10/site-packages/docutils/nodes.py", line 227, in walkabout
    if child.walkabout(visitor):
  File "/usr/lib/python3.10/site-packages/docutils/nodes.py", line 227, in walkabout
    if child.walkabout(visitor):
  File "/usr/lib/python3.10/site-packages/docutils/nodes.py", line 227, in walkabout
    if child.walkabout(visitor):
  [Previous line repeated 12 more times]
  File "/usr/lib/python3.10/site-packages/docutils/nodes.py", line 219, in walkabout
    visitor.dispatch_visit(self)
  File "/usr/lib/python3.10/site-packages/sphinx/util/docutils.py", line 516, in dispatch_visit
    method(node)
  File "/usr/lib/python3.10/site-packages/sphinx/writers/latex.py", line 759, in visit_desc_content
    assert self.in_desc_signature
AssertionError

How to Reproduce

Unfortunately I'm unable to provide the steps outside of the RPM build context, I can't seem to get the upstream dev environment correctly. Happy to test any change in the RPM build system.

Expected behavior

No response

Your project

https://github.com/coq/coq

Screenshots

No response

OS

Fedora Linux 37

Python version

3.10.4

Sphinx version

4.5.0

Sphinx extensions

listed above

Extra tools

No response

Additional context

No response

[cblegare]

Hello @befeleme , I am not a maintainer here, simply interested in helping a bit.

Are you able to review the rendered document yourself? In order to work around this, you could try to run Python with the -O or -OO options to skip the assertion. Of course you might want to double check everything is fine.

[ssbarnea]

I wonder if that has anything to do with jinja2. https://readthedocs.org/projects/molecule/builds/16644898/

Exception occurred:
  File "/home/docs/checkouts/readthedocs.org/user_builds/molecule/envs/latest/lib/python3.8/site-packages/sphinx/builders/latex/__init__.py", line 266, in write
    docname, targetname, title, author, themename = entry[:5]
ValueError: not enough values to unpack (expected 5, got 4)
The full traceback has been saved in /tmp/sphinx-err-fl7pcizt.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!

[cblegare]

@ssbarnea I don't think so. I encourage you to open a separate issue. I think I see what is happening with yours but I would rather not use this issue here for an explanation.

[cblegare]

So you know, @ssbarnea I've commented your PR above ansible/molecule#3510 (comment)

[befeleme]

Hello @befeleme , I am not a maintainer here, simply interested in helping a bit.

Are you able to review the rendered document yourself? In order to work around this, you could try to run Python with the -O or -OO options to skip the assertion. Of course you might want to double check everything is fine.

I've tried to omit the failing assert which moves the build a bit further but then it fails anyway.

pdflatex: Command for 'pdflatex' gave return code 1
      Refer to 'CoqRefMan.log' for details

In the CoqRefMan.log there is a lot of undefined references warnings and some errors like this, always concerting \pysigstopsignatures:

LaTeX Warning: Hyper reference `language/core/basic:grammar-token-ident' on pag
e 282 undefined on input line 1.

! Undefined control sequence.
\pysigstopsignatures ...store@itemsep@and@parskip 
                                                  
l.26217 \pysigstopsignatures
                            
The control sequence at the end of the top line
of your error message was never \def'ed. If you have
misspelled it (e.g., `\hobx'), type `I' and the correct
spelling (e.g., `I\hbox'). Otherwise just continue,
and I'll forget about whatever was undefined.

As to rendered PDF file, it failed to produce TOC and the index doesn't contain the page numbers (?? instead).

This still feels like Sphinx regression, I've build the same documentation with Sphinx 4.4.0 and none of the above problems occur.

[SnarkBoojum]

I'm now hitting the issue too -- and with the coq documentation too, so there are two explanations:

1. the sphinx in_desc feature is broken and the coq documentation although correct triggers a bug ;
2. the coq documentation itself has a problem and instead of getting a clear error message, sphinx errors out.

Both require a fix in sphinx -- and the later will also need a fix in coq.

It would be nice if we could extract a minimal example from this huge documentation...

[AA-Turner]

It would be nice if we could extract a minimal example from this huge documentation...

@SnarkBoojum / @befeleme -- I would really appreciate such a MRE. Please could you try deleting half of the documentation and seeing if the error occurs, and repeating etc? I'm on Windows and from the Coq documentation it seems heavily geared towards unix -- getting hopefully a single page of failing docs would be a massive help.

A

[jamesjer]

Running sphinx with -P shows that we are processing doc/sphinx/language/core/basic.rst in the coq source tree. I'm looking at the coq 8.15.2 release tarball. We are processing this text, starting on line 332 of that document:

     .. cmd:: Comments {* {| @one_term | @string | @natural } }

        Prints "Comments ok" and does not change
        the state of the document.

LaTeX for that first line, the grammar production, has been generated. We are looking at the line starting with "Prints" when the assertion fails.

[jamesjer]

Here's the issue. In the coq 8.15.2 release, look at doc/tools/coqrst/coqdomain.py, lines 49-61:

# FIXME: Patch this in Sphinx
# https://github.com/coq/coq/issues/12361
def visit_desc_signature(self, node):
    hyper = ''
    if node.parent['objtype'] != 'describe' and node['ids']:
        for id in node['ids']:
            hyper += self.hypertarget(id)
    self.body.append(hyper)
    if not node.get('is_multiline'):
        self._visit_signature_line(node)
    else:
        self.body.append('%\n\\pysigstartmultiline\n')
LaTeXTranslator.visit_desc_signature = visit_desc_signature

That doesn't set in_desc_signature, so naturally it is false when the assertion checks it.

[jamesjer]

There is already a coq PR to address this: coq/coq#16193

[AA-Turner]

Sorry this one fell through -- thanks for looking into it further, looks like this isn't our bug, so closing.

Thanks!

A