Directive hlist not implemented in LaTeX

[yves-chevallier]

Describe the bug
According to the documentation hlist must contain a bullet list. It will transform it into a more compact list by either distributing more than one item horizontally, or reducing spacing between items, depending on the builder.

It works so-so in html, but it is not implemented in LaTeX:

    def visit_hlist(self, node: Element) -> None:
        # for now, we don't support a more compact list format
        # don't add individual itemize environments, but one for all columns

Improvement?

I think it could be implemented using a multicol environment with this preamble:

\usepackage{multicol}
\newcommand{\fixspacing}{\vspace{0pt plus 1filll}\mbox{}}

I wrote a small example. However I did not find how to inject some headers in the LaTeX document. Modifying app.builder.env.config doesn't seem to work.

import re
from sphinx.writers.latex import LaTeXTranslator

def visit_hlist(self, node):
    self.compact_list += 1
    columns = len(node.children)
    self.body.append('\\begin{multicols*}{%d}' % columns)
    self.body.append('\\begin{itemize}\\setlength{\\itemsep}{0pt}'
                        '\\setlength{\\parskip}{0pt}\n')
    if self.table:
        self.table.has_problematic = True


def depart_hlist(self, node):
    self.compact_list -= 1
    self.body.append('\\end{itemize}\n')
    self.body.append('\\fixspacing\n\\end{multicols*}\n')


def inject_packages(app):
    """TODO: Not Working..."""
    config = app.builder.env.config
    to_add = '%% For hlist directive\n'
    if not re.findall(r'\\usepackage(\[.*?\])?\{multicol\}',
                      config.latex_elements['preamble']):
        to_add += '\n\\usepackage{multicol}\n'
    to_add += r'\newcommand{\fixspacing}{\vspace{0pt plus 1filll}\mbox{}}'
    to_add += '\n'
    config.latex_elements['preamble'] += to_add


def setup(app):
    LaTeXTranslator.visit_hlist = visit_hlist
    LaTeXTranslator.depart_hlist = depart_hlist

    app.connect('builder-inited', inject_packages)

    return {
        'version': '0.1',
        'parallel_read_safe': True,
        'parallel_write_safe': True,
    }

[tk0miya]

I'm still not familiar with LaTeX. So all contributions are welcome :-)

[yves-chevallier]

I could make a PR for this. Could you tell me how I can easily inject some header code into the LaTeX template? The inject_packages(app) function doesn't work and from what I saw I have to dig really deep to change this.

[tk0miya]

Please modify sphinx/texinputs/sphinx.sty or sphinx/templates/latex/latex.tex_t directly. You don't need to modify latex_elements['preamble'] to insert a code to preamble-part.

[yves-chevallier]

Yeah, but when you write an extension, you don't want to hack the sphinx.sty, or ask the user to do it so.

When writing extensions that need to add some LaTeX packages, you need to add the \usepackage{whatever} at the beginning of the document, I am trying to find a way to do this...

[yves-chevallier]

I found a better way of implementing the hlist for HTML/LaTeX output which work with both bullet_list and enumerated_list:

And with LaTex:

Here a full extension that could be implemented in Sphinx-doc

import re

from sphinx.writers.latex import LaTeXTranslator
from sphinx.writers.html5 import HTML5Translator

from sphinx import addnodes
from docutils import nodes
from sphinx.util.docutils import SphinxDirective
from docutils.parsers.rst import Directive, directives

latex_preamble = r"""
\newcounter{multicolminlines}
\setcounter{multicolminlines}{1}

\makeatletter
\xpatchcmd\balance@columns
   {\ifnum\dimen@<\topskip
     \mult@info\@ne
       {Start value
          \the\dimen@  \space ->
          \the\topskip \space (corrected)}%
     \dimen@\topskip
   \fi}
   {\skip@\c@multicolminlines\baselineskip
   \advance\skip@-\baselineskip
   \advance\skip@\topskip
   \ifnum\dimen@<\skip@
     \mult@info\@ne
       {Start value
          \the\dimen@  \space ->
          \the\skip@ \space (corrected)}%
     \dimen@\skip@
   \fi
   }
   {\typeout{Success!}}{\patchFAILED}

\define@key{hlist@keys}{columns}{\def\hlist@columns{#1}}%
\define@key{hlist@keys}{minlines}{\def\hlist@minlines{#1}}%

\newcommand{\fixspacing}{\vspace{0pt plus 1filll}\mbox{}}

\newenvironment{hlist}[1][]{%
    \setkeys{hlist@keys}{columns=4,minlines=4,#1}
    \setcounter{multicolminlines}{\hlist@minlines}
    \begin{multicols}{\hlist@columns}

}{

    \fixspacing
    \end{multicols}
}
\makeatother
"""

def visit_latex_hlist(self, node):
    options = [
        f'columns={node["columns"]}' if node['columns'] else '',
        f'min-lines={mnode["min-lines"]}' if node['min-lines'] else ''
    ]
    self.body.append('\\begin{hlist}%s\n' % ','.join(options))
    if self.table:
        self.table.has_problematic = True


def depart_latex_hlist(self, node):
    self.body.append('\\end{hlist}\n')


def visit_html_hlist(self, node):
    cols = node['columns']
    self.body.append((
        f'<div style="-webkit-column-count: {cols};'
        f'-moz-column-count: {cols}; column-count: {cols};">'
    ))


def depart_html_hlist(self, node):
    self.body.append('</div>\n')


class HList(SphinxDirective):
    """
    Directive for a list that gets compacted horizontally.
    """
    has_content = True
    required_arguments = 0
    optional_arguments = 0
    final_argument_whitespace = False
    option_spec = {
        'columns': int,
        'max-height': int,
    }

    def run(self):
        hlist = addnodes.hlist()
        hlist['columns'] = self.options.get('columns', None)
        hlist['min-lines'] = self.options.get('min-lines', None)
        hlist.document = self.state.document
        self.state.nested_parse(self.content, self.content_offset, hlist)

        if len(hlist.children) != 1 or not isinstance(hlist.children[0],
                                                      (nodes.bullet_list, nodes.enumerated_list)):
            reporter = self.state.document.reporter
            return [reporter.warning('.. hlist content is not a list', line=self.lineno)]

        return [hlist]


def add_latex_environment(app):
    if hasattr(app.builder, 'context'):
        context = app.builder.context
        context['preamble'] += latex_preamble


def setup(app):
    LaTeXTranslator.visit_hlist = visit_latex_hlist
    LaTeXTranslator.depart_hlist = depart_latex_hlist

    HTML5Translator.visit_hlist = visit_html_hlist
    HTML5Translator.depart_hlist = depart_html_hlist

    app.connect('builder-inited', add_latex_environment)

    app.add_directive('hlist', HList, override=True)

    app.add_latex_package('multicol', 'balancingshow')
    app.add_latex_package('regexpatch')
    app.add_latex_package('keyval')

    return {
        'version': '0.1',
        'parallel_read_safe': True,
        'parallel_write_safe': True,
    }

[tk0miya]

Yeah, but when you write an extension, you don't want to hack the sphinx.sty, or ask the user to do it so.

Please use app.add_latex_package() to add additional LaTeX packages.
https://www.sphinx-doc.org/en/master/extdev/appapi.html#sphinx.application.Sphinx.add_latex_package

I am using a dirty/hack function to remove the orphan

item

in each item with:

docutils determined to co-exist with the paragraph nodes after HTML5 writer. So please don't remove them. We should add CSS to control them.

[yves-chevallier]

I do use add_latex_package() but this does not allow to add abitrary LaTeX code such as \newcommand. The only way is to add it right before preamble.

[yves-chevallier]

Ok for the css correction.

[tk0miya]

I do use add_latex_package() but this does not allow to add abitrary LaTeX code such as \newcommand. The only way is to add it right before preamble.

How about adding your own .sty file (for example, hlist.sty)? Then you can use it via app.add_latex_package(). Anyway, such a workaround is not needed for Sphinx core. Please modify sphinx.sty directly.

[yves-chevallier]

@tk0miya I modified my previous answer with a kind-of working extension. I am sure I need more testing, but do you want me to propose a PR for this feature? On which branch?

[tk0miya]

I think adding implementation is better than nothing even if young. So +1 for merging it to the core. And 3.x branch is good to me. But I understand you'd like to test it as an extension. In any case, the next stable release will be at October. We're not in a hurry.

[jfbu]

The patch to multicol is not motivated, not documented, not sourced, and not maintainable. Certainly we can never merge this to Sphinx core. I get undecipherable error message with uptodate TeXLive 2020, from inserting it into a latex document preamble:

./testmulticollist.tex:30: Undefined control sequence.
<argument> \LaTeX3 error: 
                           Use \cs_replacement_spec:N not \token_get_replace...
l.30    {\typeout{Success!}}{\patchFAILED}

using
test8072patch.tex.txt

We will never patch multicol but yes absolutely we can use it naturally. At the simplest, we simply add to LaTeX writer the code to wrap the itemize, or enumerate, or description in LaTeX inside a multicols environment with a specified number of columns. As long as we don't try to force final result to have

• a minimal number of lines,
• or a maximal number of lines,
• or to automatically choose a number of columns adapted to contents,
  there is no difficulty.

Here is with index.rst

Welcome to FOO's documentation!
===============================


Test list
---------

Before

.. raw:: latex

   \begin{multicols}{5}

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

- alphabetical

.. raw:: latex

   \end{multicols}

After

and

latex_elements = {
    'preamble': r'\usepackage{multicol}',
}

then we get this output

[jfbu]

Better with

.. raw:: latex

   \raggedcolumns
   \end{multicols}

in the test, sorry. This is needed else multicols will spread out the last column, in my test we had coincidentally a number of items equal to a multiple of number of columns 5.

Notice that multicols will always proceed from top to bottom, filling first column, then second column, etc...

If we want to first fill first line, then second line, etc..., multicols is not good package for that.

@tk0miya I can start from work on this but I need to know if the output must have the items organized horizontally or vertically. For the former, maybe there is a LaTeX package or we can create the needed macros, for the latter we can basically use multicols as above. I also need to know if the number of columns will be a mandatory argument with possibly a default value (say 3) if absent, or if the code should try to determine it automatically.

[jfbu]

The tasks package might be useful to us here. However it has limitation for us that it can not contain code-blocks. Also, it is relatively recent package and we have to check if all features we will use are available at TL2015 for example. Anyway this looks like good possibility if, as I expect, hlist output should look like

1. apple 2. banana 3. strawberry
4. apple 5. banana 6. strawberry

[jfbu]

Well, the hlist example renders in HTML like this

A list of short items

that should be displayed

horizontally

(with last column flushed up in classic html_theme) so this is vertical first.

Then we only need multicol package. There will be no limitation on item contents, they can contain themselves nested lists, or literals. It however we want horizontal distribution, then multicol is not the way. tasks LaTeX package has limitations (no nesting, i.e. an item can not [edited] be or contain a list contain another hlist but it can contain a regular list ), no code-blocks, perhaps (not tested) issues (or not) with footnotes, but could be at least provide minimal support at small maintenance cost to us.

@tk0miya do you have an opinion?

[jfbu]

On further thoughts the tasks package offers functionality we can't use in an automated manner.

As we control all the output mark-up, there is no difficulty of realizing horizontal lists with minipages. Here is output from
testhlistminipage.tex.txt

Basically we obtain a table-like output but without all the vagaries of LaTeX table.

I think this is the way to go.

Pros:

• no extra package (for tasks it seems it stabilized only in 2019)
• simple LaTeX mark-up we know will never break ; nesting is possible, code-blocks ok, footnotes also with the tools we already have.

Cons:

• actually html output is unsatisfactory (vertically arranged) so why bother the effort in LaTeX ? simply use multicol and we are done ! However we will never manage to obtain with multicol that the items end up at vertically matching locations (as in example above where items are output from left to right and then top to bottom and in each horizontal there is top alignment which could be a goal even if we stacked the items vertically first, horizontally second) -- except if they all occupy exact same vertical space.
• I still have to think how to handle enumeration if needed.

[jfbu]

LaTeX2e also has a not widely known tabbing environment (inherited from original Lamport LaTeX), but trying to use it will certainly generates loads of problems. Better to go the minipage of given width way as in the test file attached to my previous comment.