Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assertion Error in nbsphinx.py during latex build #584

Closed
AAndersn opened this issue Jul 13, 2021 · 10 comments · Fixed by #591
Closed

Assertion Error in nbsphinx.py during latex build #584

AAndersn opened this issue Jul 13, 2021 · 10 comments · Fixed by #591

Comments

@AAndersn
Copy link

When using nbsphinx to build on readthedocs.org, nbsphinx has started to crash today (07/12/21) when asked to build latex:

python -m sphinx -b latex -D language=en -d _build/doctrees . _build/latex

The error message in the terminal is

processing notebook_1 notebook_2 notebook_n
resolving references...
done
writing... failed

Exception occurred:
  File "/home/docs/checkouts/readthedocs.org/user_builds/gridappsd-hackathon/envs/develop/lib/python3.7/site-packages/nbsphinx.py", line 2151, in depart_codearea_latex
    assert 'Verbatim' in lines[0]
AssertionError
The full traceback has been saved in /tmp/sphinx-err-oj7amn5e.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!

I have been able to reproduce this message on my local machine, with the following log message:

# Sphinx version: 4.1.0
# Python version: 3.8.5 (CPython)
# Docutils version: 0.16 release
# Jinja2 version: 2.11.0
# Last messages:
#   api_usage/3.3--Lesson-3.3--Using-the-PowerGrid-Models-API
#   api_usage/3.4--Lesson-3.4--Using-the-Configuration-File-API
#   api_usage/3.5--Lesson-3.5--Creating-Running-Simulation-API
#   api_usage/3.6--Lesson-3.6--Controlling-Simulation-API
#   api_usage/3.7--Lesson-3.7--Using-the-Timeseries-API
#   
#   resolving references...
#   done
#   writing...
#   failed
# Loaded extensions:
#   sphinx.ext.mathjax (4.1.0) from /home/ubuntu/venv/lib/python3.8/site-packages/sphinx/ext/mathjax.py
#   sphinxcontrib.applehelp (1.0.2) from /home/ubuntu/venv/lib/python3.8/site-packages/sphinxcontrib/applehelp/__init__.py
#   sphinxcontrib.devhelp (1.0.2) from /home/ubuntu/venv/lib/python3.8/site-packages/sphinxcontrib/devhelp/__init__.py
#   sphinxcontrib.htmlhelp (2.0.0) from /home/ubuntu/venv/lib/python3.8/site-packages/sphinxcontrib/htmlhelp/__init__.py
#   sphinxcontrib.serializinghtml (1.1.5) from /home/ubuntu/venv/lib/python3.8/site-packages/sphinxcontrib/serializinghtml/__init__.py
#   sphinxcontrib.qthelp (1.0.3) from /home/ubuntu/venv/lib/python3.8/site-packages/sphinxcontrib/qthelp/__init__.py
#   alabaster (0.7.12) from /home/ubuntu/venv/lib/python3.8/site-packages/alabaster/__init__.py
#   nbsphinx (0.8.6) from /home/ubuntu/venv/lib/python3.8/site-packages/nbsphinx.py
#   sphinx.ext.autodoc.preserve_defaults (1.0) from /home/ubuntu/venv/lib/python3.8/site-packages/sphinx/ext/autodoc/preserve_defaults.py
#   sphinx.ext.autodoc.type_comment (4.1.0) from /home/ubuntu/venv/lib/python3.8/site-packages/sphinx/ext/autodoc/type_comment.py
#   sphinx.ext.autodoc (4.1.0) from /home/ubuntu/venv/lib/python3.8/site-packages/sphinx/ext/autodoc/__init__.py
#   sphinx.ext.intersphinx (4.1.0) from /home/ubuntu/venv/lib/python3.8/site-packages/sphinx/ext/intersphinx.py
#   sphinx.ext.todo (4.1.0) from /home/ubuntu/venv/lib/python3.8/site-packages/sphinx/ext/todo.py
#   sphinx.ext.coverage (4.1.0) from /home/ubuntu/venv/lib/python3.8/site-packages/sphinx/ext/coverage.py
#   sphinx.ext.ifconfig (4.1.0) from /home/ubuntu/venv/lib/python3.8/site-packages/sphinx/ext/ifconfig.py
#   sphinx.ext.viewcode (4.1.0) from /home/ubuntu/venv/lib/python3.8/site-packages/sphinx/ext/viewcode.py
#   sphinx.ext.githubpages (4.1.0) from /home/ubuntu/venv/lib/python3.8/site-packages/sphinx/ext/githubpages.py
Traceback (most recent call last):
  File "/home/ubuntu/venv/lib/python3.8/site-packages/sphinx/cmd/build.py", line 280, in build_main
    app.build(args.force_all, filenames)
  File "/home/ubuntu/venv/lib/python3.8/site-packages/sphinx/application.py", line 343, in build
    self.builder.build_update()
  File "/home/ubuntu/venv/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 290, in build_update
    self.build(['__all__'], to_build)
  File "/home/ubuntu/venv/lib/python3.8/site-packages/sphinx/builders/__init__.py", line 357, in build
    self.write(docnames, list(updated_docnames), method)
  File "/home/ubuntu/venv/lib/python3.8/site-packages/sphinx/builders/latex/__init__.py", line 308, in write
    docwriter.write(doctree, destination)
  File "/home/ubuntu/venv/lib/python3.8/site-packages/docutils/writers/__init__.py", line 78, in write
    self.translate()
  File "/home/ubuntu/venv/lib/python3.8/site-packages/sphinx/writers/latex.py", line 99, in translate
    self.document.walkabout(visitor)
  File "/home/ubuntu/venv/lib/python3.8/site-packages/docutils/nodes.py", line 214, in walkabout
    if child.walkabout(visitor):
  File "/home/ubuntu/venv/lib/python3.8/site-packages/docutils/nodes.py", line 214, in walkabout
    if child.walkabout(visitor):
  File "/home/ubuntu/venv/lib/python3.8/site-packages/docutils/nodes.py", line 214, in walkabout
    if child.walkabout(visitor):
  [Previous line repeated 5 more times]
  File "/home/ubuntu/venv/lib/python3.8/site-packages/docutils/nodes.py", line 227, in walkabout
    visitor.dispatch_departure(self)
  File "/home/ubuntu/venv/lib/python3.8/site-packages/sphinx/util/docutils.py", line 486, in dispatch_departure
    method(node)
  File "/home/ubuntu/venv/lib/python3.8/site-packages/nbsphinx.py", line 2151, in depart_codearea_latex
    assert 'Verbatim' in lines[0]
AssertionError

My repo and source notebooks have not changed (readthedocs is building from the same commit)

image

image

@AAndersn
Copy link
Author

Update: It seems to be a compatibility issue with sphinx 4.1.0.

Pinning sphinx to version 4.0.2 using the requirements.txt file resolved the build error.

aida-ahmadi added a commit to emerge-erc/ALminer that referenced this issue Jul 13, 2021
@aida-ahmadi
Copy link

Update: It seems to be a compatibility issue with sphinx 4.1.0.

Pinning sphinx to version 4.0.2 using the requirements.txt file resolved the build error.

@AAndersn Thanks for figuring this out. I was stuck on it for hours yesterday.

@mgeier
Copy link
Member

mgeier commented Jul 13, 2021

Thanks for the report!

Apparently (as git bisect told me), this was caused by sphinx-doc/sphinx#9166.

I'll have to look into it a bit more for how to fix this.

@aiqc
Copy link

aiqc commented Jul 15, 2021

thank you! build passes with these dependencies:

requiresments.txt

sphinx==4.0.2
ipython==7.19.0
nbsphinx>=0.8.6
sphinx_copybutton==0.3.1
sphinx_rtd_theme==0.5.0
sphinxext-opengraph==0.4.1
jinja2>=3.0.1

dschick pushed a commit to dschick/udkm1Dsim that referenced this issue Jul 16, 2021
@mgeier
Copy link
Member

mgeier commented Jul 16, 2021

See also sphinx-doc/sphinx#9458.

@jfbu
Copy link
Contributor

jfbu commented Jul 19, 2021

Commenting out the assert lines I observe that the latex build output of nbsphinx own documentation contains:

\begin{sphinxuseclass}{nbinput}
\begin{sphinxuseclass}{nblast}
{
\sphinxsetup{VerbatimColor={named}{nbsphinx-code-bg}}
\sphinxsetup{VerbatimBorderColor={named}{nbsphinx-code-border}}
\begin{sphinxVerbatim}[commandchars=\\\{\}]
\llap{\color{nbsphinxin}[ ]:\,\hspace{\fboxrule}\hspace{\fboxsep}}
\end{sphinxVerbatim}
}

\end{sphinxuseclass}
\end{sphinxuseclass}

The fix to the issue is to define latex environment sphinxnbinput and sphinxnblast which will do the

\sphinxsetup{VerbatimColor={named}{nbsphinx-code-bg}}
\sphinxsetup{VerbatimBorderColor={named}{nbsphinx-code-border}}

thing (I don't know to which class assign these tasks), rather than having theme explicitly show up in the latex code. Also they would not add the explicit { and } which are not needed as LaTeX environments limit scopes.

Same problem with the class nboutput. I observe this kind of code:

\begin{sphinxuseclass}{nboutput}
\begin{sphinxuseclass}{nblast}
{

\kern-\sphinxverbatimsmallskipamount\kern-\baselineskip
\kern+\FrameHeightAdjust\kern-\fboxrule
\vspace{\nbsphinxcodecellspacing}

\sphinxsetup{VerbatimColor={named}{white}}
\sphinxsetup{VerbatimBorderColor={named}{nbsphinx-code-border}}
\begin{sphinxuseclass}{output_area}
\llap{\color{nbsphinxout}[2]:\,\hspace{\fboxrule}\hspace{\fboxsep}}\begin{sphinxuseclass}{}


\begin{sphinxVerbatim}[commandchars=\\\{\}]
42
\end{sphinxVerbatim}

\end{sphinxuseclass}
\end{sphinxuseclass}
}

\end{sphinxuseclass}
\end{sphinxuseclass}

which has 4 nested classes, from outer to inner nboutput, nblast, output_area, and (I don't know how this comes about) the "empty" class.

I'll see if task does not appear too complicated in the time I dispose of and possibly make a PR. I don't see an easy replacement to the assert 'Verbatim' in lines[0] and perhaps simply removing those 2 lines in nbsphinx.py is enough of a temporary hotfix? Indeed, regarding building nbsphinx.pdf, I got no further issue apart from one warning about a missing reference

Latexmk: List of undefined refs and citations:
  Reference `markdown-cells:' on page 27 undefined on input line 2384
Latexmk: Summary of warnings from last run of *latex:
  Latex failed to resolve 1 reference(s)
Latexmk: All targets (nbsphinx.pdf) are up-to-date

So simply removing these assert lines (whose purpose I don't know) appears to fix the build problems.

@jfbu
Copy link
Contributor

jfbu commented Jul 19, 2021

After having looked a bit more I realize I don't have a clear enough view of the nbsphinx proceedings to understand clearly who originates what, for example output_area class. I started doing things like this:

% Define an environment to support the nbinput class (Sphinx>=4.1.0)
\newenvironment{sphinxclassnbinput}{%
    \sphinxsetup{VerbatimColor={named}{nbsphinx-code-bg}}%
    \sphinxsetup{VerbatimBorderColor={named}{nbsphinx-code-border}}%
}{}

to be added to LATEX_PREAMBLE. This would go in pair with conditionally removing the

        out.append(r'\sphinxsetup{VerbatimColor={named}{nbsphinx-code-bg}}')

and

    out.append(
        r'\sphinxsetup{VerbatimBorderColor={named}{nbsphinx-code-border}}')

lines in depart_codearea_latex(self, node). But I see also stderr class.

In brief, Sphinx 4.1.0 allows to support via LaTeX environments what is implemented in HTML via CSS classes. But the current nbsphinx code is backwards compatible and needs no adaptation apart from removing the two "assert".

In future though, a transfer of the hacking via DocUtils nodes and added visit and depart handlers to simply adding to LATEX_PREAMBLE a whole bunch of environment definitions, one for each class, seems the way to refactor the nbsphinx LaTeX code into something simpler, and much more parallel to HTML output. I can not engage into it at current time as I don't know the nbsphinx codebase enough.

@mgeier
Copy link
Member

mgeier commented Aug 1, 2021

Sorry for my late response!

So simply removing these assert lines (whose purpose I don't know) appears to fix the build problems.

It indeed allows the build to pass, but the resulting PDF doesn't look as expected.

nbsphinx needs to insert the output prompt at the correct line (right after the sphinxVerbatim environment is opened).

Less importantly (but still), there might be some cases where additional blank lines have to be inserted in the beginning or the end of the output (because reST syntax doesn't allow leading or trailing blank lines).

I checked the LaTeX lines in question (for an output cell only containing the number 42) produced by Sphinx 4.0.2 (prior to changes made by nbsphinx):

['\\begin{sphinxVerbatim}[commandchars=\\\\\\{\\}]', '42', '\\end{sphinxVerbatim}']

With the latest Sphinx version, those lines are changed to:

['\\begin{sphinxuseclass}{output_area}', '\\begin{sphinxuseclass}{}', '', '', '\\begin{sphinxVerbatim}[commandchars=\\\\\\{\\}]', '42', '\\end{sphinxVerbatim}', '', '\\end{sphinxuseclass}', '\\end{sphinxuseclass}']

It would be interesting to properly use the sphinxuseclass environments, but as @jfbu said, I'd like nbsphinx to be compatible with older Sphinx versions

@mgeier
Copy link
Member

mgeier commented Aug 1, 2021

I've just created #591, which hopefully fixes this issue.

@mgeier
Copy link
Member

mgeier commented Aug 10, 2021

I have just created a new nbsphinx release (version 0.8.7) which should fix this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants