Support containers in LaTeX output.

[dham]

• Feature

Purpose

Support containers in the LaTeX writer by outputting the start and end of a LaTeX group, wrapping the beginning and end of a LaTeX environment. The class name(s) are passed as an argument to the
environment.

Detail

Container directives currently result in no output in the LaTeX writer, which is unfortunate because they are a very useful mechanism for adding an essentially arbitrary formatting hook in a document. A previous solution was proposed in #1336 which would have emitted an environment with the name of the directive class. This was not merged because emitting previously undefined environment names will break the latexpdf build unless the user also defines the environment, for example in latex_elements.

This proposed solution avoids that problem by always emitting a sphinxcontainer environment. The class name(s) are passed into the environment as an argument. The environment emitted is wrapped in \bgroup and \egroup so that it also constitutes a LaTeX group. It is argued that this best mirrors the html behaviour and is therefore least likely to surprise a user.

A definition of the sphinxcontainer environment is added in a style file input by sphinx.sty, so inserting container directives will not break the latexpdf build. Should the user wish to insert LaTeX formatting based on a container directive, then they can redefine sphinxcontainer in latex_elements["preamble"]. For example, if an .rst file contains:

.. container:: red

   red text

.. container:: blue

   black text

more black text.

and conf.py contains:

latex_elements = {
    "preamble": r"""
    \usepackage{etoolbox}
    \renewenvironment{sphinxcontainer}[1]{
        \ifstrequal{#1}{red}{\color{red}}{}
    }{}
    """
}

then red text will be rendered in red, and all the other text will be displayed in black.

Relates

• reST container directive for latex output #1336

Limitations

The documentation for container links straight through to the docutils documentation, so I don't know where I should document this change.

Life is a bit more complicated for the user if they need to customise the end of the environment depending on the container class, or if the container specifies multiple classes. However, both of these are really LaTeX limitations. I suggest that this small change provides all the relevant information through to LaTeX, and that fixing LaTeX's limitations in this matter should not be seen as Sphinx's problem.

[tk0miya]

In docutils, the container node will be rendered as \DUclass{classname}{ content } macro.
https://repo.or.cz/docutils.git/blob/HEAD:/docutils/docutils/writers/latex2e/__init__.py#l1560
https://repo.or.cz/docutils.git/blob/HEAD:/docutils/docutils/writers/latex2e/docutils.sty#l31

It would be better to follow the concept.

@jfbu Do you have opinion for this?

[dham]

In docutils, the container node will be rendered as \DUclass{classname}{ content } macro.
https://repo.or.cz/docutils.git/blob/HEAD:/docutils/docutils/writers/latex2e/__init__.py#l1560
https://repo.or.cz/docutils.git/blob/HEAD:/docutils/docutils/writers/latex2e/docutils.sty#l31

It would be better to follow the concept.

I'm a little confused. I agree that a macro would be better than an environment, but the links you provide seem to define a DUclass environment and insert \begin{DUclass}{classname} and \end{DUclass}.

I am completely happy to refactor this to insert a macro as suggested, but that doesn't actually seem consistent with what docutils are doing. Please let me know what you would prefer.

[tk0miya]

Ah, sorry. It's my bad. I'd like to mention the DUclass "environment".

[dham]

OK, so I am a little unclear as to what you are asking for. Would you like me to rename the environment as DUclass? I can do that, happily, though it would not be very consistent with the other LaTeX names that sphinx uses.

[tk0miya]

My large concern is generating an environment for each class, not for the whole of the node. For example, a container node having "red bold" will be converted to \begin{DUclass}{red}\begin{DUclass}{bold} content \end{DUclass}{bold}\end{DUclass}{red}. And users can customize the behavior for red and bold individually. It will resolve a limitation you mentioned.

Life is a bit more complicated for the user if they need to customise the end of the environment depending on the container class, or if the container specifies multiple classes. However, both of these are really LaTeX limitations.

About naming, I don't have opinion for this. So I think either '\DUclassor\sphinxclass` (?) is okay to me.

[dham]

I've changed the code to one environment per class as requested, and renamed that environment to sphinxcode. Note that it's not possible to pass parameters to \end so the problem of not knowing which container you are ending persists. There are a few approaches which could be taken:

1. I currently open and close a group around each environment. This makes declarations like \color end in the right place.
2. Declaring a macro containing the class name at the start is possible, but basically requires a stack. There is a stack implementation on ctan, fifo-stack, but we probably don't want to add that random LaTeX dependency.
3. The xparse package provides environments in which the end clause can see the arguments, but again this is an additional LaTeX dependency.
4. You decide not to add any more dependencies and leave it to the user to use one of the above if they need them.

[tk0miya]

With docutils' style, the following code will be converted the latter one internally

\begin{DUclass}{red}\begin{DUclass}{bold} content \end{DUclass}{bold}\end{DUclass}{red}

\DUCLASSred
\DUCLASSbold
content
\endDUCLASSbold
\endDUCLASSred

So users can change the behavior for each class by defining \DUCLASS*** and \endDUCLASS***.

I think it's important to allow customizing each class. It enables that one extension creates "red" class and its macro, and another extension creates "bold" class and its macro.

Additionally, could you add a testcase for this?

Note: I confirmed the docutils' macro via this code:

\documentclass{report}
\usepackage{color}

% copied from https://repo.or.cz/docutils.git/blob/HEAD:/docutils/docutils/writers/latex2e/docutils.sty
\newenvironment{DUclass}[1]%
  {% "#1" does not work in end-part of environment.
   \def\DocutilsClassFunctionName{DUCLASS#1}
     \csname \DocutilsClassFunctionName \endcsname}%
  {\csname end\DocutilsClassFunctionName \endcsname}%

\def\DUCLASSred{\color{red}}
\def\endDUCLASSred{}
\def\DUCLASSbold{\bf}
\def\endDUCLASSbold{}

\begin{document}
  \begin{DUclass}{red}
    \begin{DUclass}{bold}
      hello
    \end{DUclass}
    hello
  \end{DUclass}
  hello
\end{document}

[tk0miya]

Ah, sorry. My last comment was wrong.

\begin{DUclass}{red}\begin{DUclass}{bold} content \end{DUclass}\end{DUclass}

No argument is passed to the \end{DUclass}.

[dham]

OK will do. I am a little unclear on how a test of LaTeX output is supposed to work. Can you point me at an example.

[dham]

The requested code changes are in. I've browsed the tests and am still slightly mystified as to how they work, so would appreciate some guidance.

[tk0miya]

I've browsed the tests and am still slightly mystified as to how they work, so would appreciate some guidance.

1. Add a Sphinx project for testing under tests/roots directory (like tests/roots/test-latex-container).
2. Add a testcase to tests/test_*.py (like tests/test_build_latex.py).

In this case, please add a test case into tests/test_build_latex.py like following:

@pytest.mark.sphinx('latex', testroot='latex-container')  # The name of sphinx project that added at step 1.
def test_latex_nested_tables(app, status, warning):
    app.builder.build_all()
    result = (app.outdir / 'test.tex').read_text()
    assert r'blah blah' in result

[dham]

Thanks for the advice. A test is now included.

[dham]

Just emerged from marking hell. I see an administrator has triggered the tests and they have passed. Is there anything more I should do in order to get this merged?

Many thanks,

David

[jfbu]

LGTM with nits!

Thanks for nice contribution and sorry for delay

I assumed during reviewing this is only block-level mark-up as per docutils doc so we don't have to worry about space tokens.

[jfbu]

Also, I wonder if #1 needs some sanitizing of any sort. Accented letters or other special chars are not usable as LaTeX environment names and could cause breakage. But digits and spaces will be ok.

[jfbu]

In our latex codebase we have this

sphinx/sphinx/texinputs/sphinxlatexobjects.sty

Lines 167 to 180 in e0b1e10

	% From docutils.writers.latex2e
	% inline markup (custom roles)
	% \DUrole{#1}{#2} tries \DUrole#1{#2}
	\providecommand*{\DUrole}[2]{%
	\ifcsname DUrole\detokenize{#1}\endcsname
	\csname DUrole\detokenize{#1}\endcsname{#2}%
	\else% backwards compatibility: try \docutilsrole#1{#2}
	\ifcsname docutilsrole\detokenize{#1}\endcsname
	\csname docutilsrole\detokenize{#1}\endcsname{#2}%
	\else
	#2%
	\fi
	\fi
	}

which was inherited many years ago from DocUtils I presume. Notice it uses \detokenize to avoid issues with non-ascii or special characters. We may at a later stage (after this is merged once updated) consider similar.

But then we would need to provide some documented interface like this

\newsphinxclassenv{red}{\color{red}}{}

which would facilitate Sphinx user define the sphinxclassred environment. This looks silly but now the user can use grün for example in place of red (assuming here no special escape happens regarding ..container:: grün).

The definition would simply be

\newcommand\newsphinxclassenv[3][{\newenvironment{\detokenize{#1}}{#2}{#3}}

but then we have to use \detokenize in the master sphinxuseclass dispatcher as in the DUrole above.

This can be added later after this will be merged once updated.

EDIT: and then we can do a \renewsphinxclassenv [our renew could work even in environment in fact does not exist, I have always considered the LaTeX behaviour of raising an error in such case an interface error] and a \providesphinxclassenv, making nice for user if in future the functionality is used by extensions. [to be complete also some conditional \sphinxifclassenvexists{code if it exists}{code if it does not exist} but perhaps I am going too far].

The sphinxclass<detokenized name> environments defined by \newsphinxclassenv et alia are of a simple no-parameter type, as we have no interface for that.

[dham]

OK. I think I claim I have dealt with all the requested changes. I haven't touched the handling of special characters as I believe you suggest this should be postponed to future work.

[jfbu]

LGTM! Thanks for contribution.

We can add later handling of non-ascii class names.

[jfbu]

Waiting for testing to complete, then I will push to this branch a CHANGES entry and merge.

Thanks for great addition!

[jfbu]

rather I merge now and will push the CHANGES entry later once I fix may github email privacy settings...

[tk0miya]

Looks good to me too!

@jfbu Thank you for reviewing :-)

[dham]

This is my first contribution to the main sphinx project. Thank you for making it a very positive experience. I hope to have reason to contribute more in the future.