LaTeX: support for booktabs-style and zebra-striped tables #10759

jfbu · 2022-08-08T06:52:26Z

This transfers here development from #6666 to continue it and merge with #6671.

revamp initial work of @sephalon in Add support for booktabs-style tables to LaTeX builder #6666 for compatibility with current Sphinx LaTeX and to address initial issues. This has been done originally at Add support for booktabs-style tables to LaTeX builder #6666 PR but I am moving it here and will adapt WIP status until the whole thing is done.
merge initial work of @sephalon from Add support for zebra-striped tables to LaTeX builder #6671
allow some tables to be styled the traditional way in documents globally using the booktabs option
allow tables to be styled the booktabs way in documents globally not using the booktabs option
as for booktabs style, think about locally styled tables; LaTeX aspects are not complicated based upon booktabs-styling with Add support for booktabs-style tables to LaTeX builder #6666 as merged here, the problem to obtain as light-weight mark-up as possible. Simplest seems to go via classes but then we must used latex- prefixed names to dispel user expectation that it also applies to say html build. Actually using a container directive is possibly better as on LaTeX side it will wrap in a scope limiting environment. Maybe for first installment we can simply advice using some such recipe (container directive plus raw latex) which does not need extra code in Sphinx itself.

finally I used classes with no latex- prefix even though they also may influence html output, but this sounds actually good

removed longish paragraph which was in part obsoleted and too complex to edit; in brief all changes are opt-in except that

for merged cells in a single row the row colour (or the dedicated merge row colour) is now obeyed, so people who added manual usage of \rowcolor to their documents had only white merged cells but now the merged cell in single row (only) will react to row colour.
for people who added manual usage of \rowcolors command to their document, the effect will be overwritten by this PR. Sphinx does not itself use \rowcolors due to the latter limitations (and the fact that not all people have xcolor). Sphinx intervenes via its table templates and own code to influence the row colours, and this will overrule effect of manually inserted \rowcolors. I don't know if this means the PR should go to master rather than 5.x.

Render tables without vertical rules and horizontal rules of varying thickness (with additional space above and below) using the booktabs package. This is a rebase of original commit which was on master branch prior to 2.1 release. In this rebase the \RequirePackage{booktabs} has been removed from sphinx.sty, as it has to be conditional on latex_booktabs (or whatever its name will be) configuration option, and will be repositioned in a subsequent commit.

Render tables with alternating background colors for even and odd rows (so called "zebra striping").

jfbu · 2022-08-11T15:54:02Z

Memo: bb859c6 probably closes #8220 as it makes mention of rst-class more prominent and links to it.

jfbu · 2022-08-13T16:54:12Z

In order to illustrate the effect of this PR I am adding here snapshots from our own document, using:

latex_table_style = ['booktabs', 'colorrows']

and the default colours activated by 'colorrows':

first table

another

Now a long table with a tabularcolumns directive.

on next page:

The effect was obtained via this

>{\sphinxcolorblend{!95!red}\centering\noindent\bfseries\color{red}}\Y{.12}

as configuration for the third column. It works here because my TeX installation has the LaTeX xcolor.sty. On user devices with xcolor.sty not installed (I don't know actual situation but many years ago this was not automatic in Linux distros), the \sphinxcolorblend will be ignored but trigger a warning at very end of latex build console output (and also at very end of pdf after indices).

Now a complex grid table with a custom tabularcolumns (with no vertical lines)

.. tabularcolumns:: >{\sphinxcolorblend{!90!blue}}TTT>{\columncolor{yellow}\sphinxnorowcolor}TT

and booktabs style

.. rst-class:: booktabs

It illustrates various features. The conf.py is:

latex_table_style = ['colorrows']

latex_elements = {
    'preamble': r'\usepackage{booktabs}',
    'sphinxsetup': '''TableRowColorOdd=red!10,
                      TableRowColorEven= blue!10,
                      TableRowColorHeader=   {gray}{0.75},
                      TableMergeColorHeader=green,
                      TableMergeColorOdd=green!20,
                      TableMergeColorEven=green!10''',
}

where the colours have been chosen at random (and test the new acceptance of \colorlet-like syntax). Also r'\usepackage{booktabs}' was added to the 'preamble' of latex_elements as the document does not use globally 'booktabs' in latex_table_style but this table requires it (else, there would be a warning at very end of the latex build console output and at very end of pdf after indices).

One sees in this table:

merged cells remain with no background colour, except for the case of single-row multi-column cells. They then use the special Merge color, if set.
in the header each row uses the same colouring style
the fifth column displays the normal colours corresponding to 'sphinxsetup'
the fourth column obeys yellow column color thanks to \sphinxnorowcolor
the first column uses \sphinxcolorblend, whose effect is ignored in merged cells.

grid table
----------

.. tabularcolumns:: >{\sphinxcolorblend{!90!blue}}TTT>{\columncolor{yellow}\sphinxnorowcolor}TT

.. rst-class:: booktabs

+---------+---------+---------+---+--------+
| header1 | header2-3         | a | normal |
+---------+---------+---------+---+--------+
| header1 | header2-3         | a | normal |
+---------+---------+---------+---+--------+
| header1 | header2-3         | a | normal |
+=========+=========+=========+===+========+
| cell1-1 | cell1-2 | cell1-3     | normal |
+---------+         +---------+---+--------+
| cell2-1 |         | cell2-3     | normal |
+         +---------+---------+---+--------+
|         | cell3-2-par1      | d | normal |
+---------+                   +---+--------+
| cell4-1 | cell3-2-par2      | e | normal |
+---------+---------+---------+---+--------+
| cell5-1 single-row merged   | f | normal |
+---------+---------+---------+---+--------+

jfbu · 2022-08-13T19:19:12Z

The latest (now squashed) commits in this PR fix some issues with merged cells which were not yet documented as such: when a tabularcolumns directive uses no |, nevertheless merged cells would sometimes use them; also, horizontal widths was not computed correctly (but the discrepancy was not really visible). On the other hand if the tabularcolumns directive contains a |, nothing changes, i.e. merged cells are bordered too vertically, even though in complex column spec one could imagine only some columns are bordered. In theory, the LaTeX writer could analyse where the | are located in the colspec and use this to influence its handling of merged cells, but this should be topic of another PR: this one only takes care of no | at all in the tabularcolumns colspec.

jfbu · 2022-08-13T20:22:28Z

again one of those transient test_build_linkcheck.py failures... https://github.com/sphinx-doc/sphinx/runs/7821615851?check_suite_focus=true which are not related to this PR...

This is a combination of 28 commits... - Simplify a bit a conditional in the longtable template This also puts the target for a longtable with a label but no caption above the toprule for better hyperlinking (testing shows hyperlink target can not end up alone at bottom of previous page). - Refactor and trim doc about LaTeX tables... to make room for more, later - Enlarge allowed syntax for colour assignments via 'sphinxsetup' - latex_table_style new configuration value and coloured rows For the user interface tried to look for inspiration in https://docutils.sourceforge.io/docs/user/config.html#table-style which mentions booktabs and borderless. They also mention captionbelow which we can implement later, now that architecture is here. They don't mention coloured rows. - Test on our own document... looks fine! - Update LaTeX table tests and templates Modify longtable templates to put LaTeX macros each on its line Table body insertion without removing previous EOL may give output which contain some empty lines but longtable defines \par token to be same as \empty. - Work-around an incompatibility of \cline with row colours, improve docs - Reverse priority of classes to allow overruling booktabs by standard after parsing source but before letting LaTeX writer act - Closes sphinx-doc#8220 Commit sphinx-doc@bb859c6 already improved a bit, this finishes it (as :rst:dir:`rst-class` was actually not linking to anywhere). - Update CHANGES for PR sphinx-doc#10759 - Let booktabs style defaults to *not* using \cmidrule. They actually don't make much sense there, as all \hline's are removed. - Enhance customizability at LaTeX code level (via code executed prior to table rendering, e.g. from a container class environment). - Patch booktabs \cmidrule, as if it used via \sphinxcline, there is a vertical space problem in case of there are two in the same row due to booktabs \futurelet not knowing \sphinxcline - Add \sphinxnorowcolor which allows construct such as this one in a tabularcolumns directive: >{\columncolor{blue}\sphinxnorowcolor} else LaTeX always overrides column colour by row colour - Add TableMergeColorHeader, TableMergeColorOdd, TableMergeColorEven so single-row merged cells can be styled especially - Extend row colours to all header rows not only the first one (all header rows will share same colour settings) - Auto-adjust to a no '|'-colspec for optimal handling of merged cell - Add \sphinxcolorblend - Needed to also detect if a '|' is in tabularcolumns, tests updated - Fix refactoring in this series which broke table.colsep update - Add test which would have shown regression regarding table.colsep - Fix another regression regarding \sphinxcline + booktabs Can not add test for that, because it shows only after PDF build. - Final testing and code comments update

jfbu · 2022-08-17T19:47:29Z

I have added in my clone at jfbu@abab0bf an extra commit in order for booktabs + colorrows to give tables with no "white gaps". Here are some snapshots of what it produces in our own document:

edit: I took the snapshots at a time the "night-shift no blue waves" had already ticked-in... so this might change a bt the colours...

This one has a problem because the small gap underneath the top rule is filled with the default colour for headers (slighly darker gray than the one for odd rows). To fix this is a bit tough on macro side because it will need \sphinxtoprule to peek ahead to see if next token is \endhead, \endfirsthead, or \sphinxtableatstartofbodyhook which is only way to realize there is no header and thus adapt the colour. Easier would be to add mark-up to the table to signal to LaTeX macros that it has no header.

Also this one has a problem, because we see that the column color tint does not appy. For it to apply we would have to use an altogether different approach than using the \specialrule of booktabs.

Besides globally on zooming on pictures I think I am seeing anti-aliasing which might be a sign there is some extra space I did not take into account, andI would need to dig into \specialrule code from booktabs which I don't really feel like doing.

So this is bit problematic (when I started writing this comment and took the snapshots I was not aware of all problems).

update 1: the problem with colour for tables with no header has been fixed at jfbu@765ccda. And regarding zooming on the pictures I should rather have zoomed on the PDFs and this looks fine although it seems either my eyes or the PDF viewer does anti-aliasing at intermediate zoom levels but this depends on colours. to be honest, zooming at high levels on tables reveals the effet of the \sphinxcolorpanelextraoverhang parameter whose default is 0.1pt which makes the colour background extend a tiny bit more to left and right than for the filled gap and the horizontal rules, perhaps I should propagate this extraoverhang to the width of the rules... (this thing never ends)

One wonders after colorizing the gaps why keep the black rules at all... and why LaTeX produces generally so cramped tables. (LaTeX's \arraystretch is hopeless but works for simplest cells with single-line contents).

update 2: I have continued at https://github.com/jfbu/sphinx/tree/latex_booktabs_nowhitegap to polish some details. However there is a problem for which I feel there will not be easy solution (probably needed to hack longtable): the \midrule is part of the longtable headers so its action is frozen and decided when longtable parses until \endfirsthead, \endhead. So the colour used until the midrule will be always the "RowColorOdd" one. But on top of next page the colour of first non-header row depends on parity one can not know it in advance. The probably easiest way to fix would be to hack longtable so that when on top of a new page body rows always start with "odd" colour, rather than continuing the alternance from bottom of previous page. Or one simply accepts the situation. Usually the "RowColorOdd" should match not too bad with the "RowColorHeader" anyhow but it might contrast a bit with "RowColorEven". Not sure I want to solve this (I was supposed to have finished this PR something like 10 days ago...). And there is a somewhat similar problem with the bottom rules... (they come from \endfoot and choose the colour when longtable first parses them I think; must check the longtable code again).

Relates sphinx-doc#10759

This avoids at high level of zooming, depending on PDF viewers a line of non-coloured or anti-aliased pixels at the joining of two rectangles of same colour due to rounding effects. Nota Bene: the background colour panels extend a tiny bit beyond left and right limits of table due to another parameter used for similar reasons motivated by horizontally merged cells: \sphinxcolorpanelextraoverhang (also defaulting to 0.1pt) Relates sphinx-doc#10759 This is a combination of 4 commits - some code comments for future edit if this is to be used upstream - safer usage of \spx@table@crazyfork in case #1 turns out to be \par - add \sphinxifthistablewithcolorrowsTF to simplify coding - \sphinxbooktabsspecialruleoverhang for perfect same-colour-joins

jfbu · 2022-08-19T06:48:02Z

edited (of course...)

In one of the longtable snapshots of #10759 (comment), one sees that header colour is applied to the table continuation hints.

But this was an oversight. In first phase of development of this PR I noticed things such as:

when testing with \rowcolors command (in its case, this applied even to the table caption, not only to continuation hints as in this picture). So I dropped it and initially did not add row colours to the headers. They were other problems, then these problems were solved and I re-added row colours to header rows, and forgot about image above because testing only with wide long tables whose width was larger than the continution hints...

With the extra patch now pushed one obtains for example on top of a continuation page in our own document:

and at bottom of a page for the same long longtable of Deprecated APIs:

About this question from earlier version of this comment:

I guess this removal of the header row colour from contination hints should be default?

It has to be not only the default, but with no added customizing option.

One reason in particular is that the colour is applied only up to the long table width, but the continuation hints may be larger than this width (but this width is hidden to not influence the table cells layout).

This way, custom user replacement code for \sphinxbooktabs{top,mid,bottom}rule is relatively painless to inject in preamble or via container class. Also, it was needed to add some LaTeX code to the longtable template in order to avoid a colour-related problem with \sphinxbottomrule. The 'sphinxsetup' key now named booktabscolorgaps defaults to being true and is left only as a way to provide an easy "turn-off" if the issue with the longtable colours after the mid rule and above the bottom rule is blocking to some. But a priori I envision at 6.0.0 to simply remove this undocumented option as the said issue is not serious and more to be considered a fact-of-life. Besides using booktabs+colorrows is strictly opt-in at this time. I considered various LaTeX ways around this longtable thing (it constructs headers and footers once and for all, one can not let them depend on some "dynamic" context; there is no easy way for a body row to know it is first on the page after header, and anyway would require adding mark-up or considerably complicate the \everycr)) but none of them are even remotely reasonable. Also, a non-documented booktabsnogaps 'sphinxsetup' key has been added: it suppresses all the extra spacing booktabs injects around its top, mid, and bottom rules. Will solve the booktabs+colorrows+longtable conundrum from previous paragraph.

jfbu · 2022-08-20T14:21:05Z

I have now merged the separate work on colorizing booktabs gaps mentioned in earlier comments so that it is done per default and tables look as in #10759 (comment). I consider now it is better indeed to colorize these gaps if booktabs+colorrows is chosen by user conf.py, and the longtable aspects should not be considered blocking. ~~However I needed to make one extra addition to the longtable template.~~ finally I moved the change away from the longtable template

This brings to an end (? I was already saying PR was ready 10 days ago...) my work on this... as my Sphinx+LaTeX time will now be more scarce in coming months I wanted to finish a coherent whole.

This is better not to overload our template. Besides it makes it easier for user to use another longtable template.

jfbu · 2022-08-20T20:01:52Z

Two short notes:

in the Python code I used colsep to designate either'' or '|'. The name comes from initial Add support for booktabs-style tables to LaTeX builder #6666. It looks natural as shorthand for "column separator". But LaTeX uses \tabcolsep for the dimension register denoting whitespace (on each side of |). So the name is a bit surprising to LaTeX user. I didn't look for a better one.
recently there is a new LaTeX package https://github.com/lvjr/tabularray which looks promising. It may in future help to replace all three of tabular, tabulary and longtable perhaps (I notice it uses optionally internally varwidth package as Sphinx does via explicit mark-up). But this can not be until a few years because it is undergoing active development (first release in May 2021, see its ChangeLog, with some breaking changes). By the way it uses a colsep key in its table config with the meaning afaict of interfacing to some equivalent of \tabcolsep illustrating the terminology problem mentioned in previous item.

jfbu · 2022-09-24T10:57:00Z

@AA-Turner Although the various comments I added when I was actively working on this may give a frightening impression, this PR is completed and I am not planning any further change... it adds opt-in features only (some are tested in our own document via a slightly modified conf.py). We can wait to see if @tk0miya has comments. For 6.0.0 release I would like to make some configuration the default, so that tables look better in LaTeX.

jfbu · 2022-10-05T16:49:21Z

If no objection I will merge this in 5.x in a few days, so that it makes it to 5.3.0. The changes are opt-in only. For 6.0.0 I will propose probably to make some of it the new default for rendering tables in LaTeX.

Edit: In view of #10896 I will merge this (if no objection) tomorrow October 6th, around 8PM European time ;-) Ah edit again, release is actually planned for the next week=end, so I will wait until Wednesday October 12th...

Remove in passing a note from latex.rst which was of no real use to Sphinx LaTeX users.

jfbu · 2022-10-12T15:04:41Z

now merging a few hours earlier than announced due to personal constraints

alea jacta est ;-)

real activation will be for 6.0

sephalon · 2022-10-12T15:50:06Z

@jfbu Thank you very much for your hard work to bring these features upstream! :-)

jfbu · 2022-10-12T16:55:21Z

@sephalon sorry I messed up in the squashed merge and the git commit lost your co-authorship... and I can not force push to 5.x...

Stefan Wiehler and others added 4 commits August 6, 2022 18:38

Re-implement option latex_use_booktabs, adapt existing tests

a32331c

Fix typo in doc

dcc70dd

Extend testing to cover usage of booktabs

a3498c6

jfbu added type:enhancement enhance or introduce a new feature builder:latex labels Aug 8, 2022

jfbu added this to the 5.2.0 milestone Aug 8, 2022

jfbu marked this pull request as draft August 8, 2022 06:52

This was referenced Aug 8, 2022

Add support for booktabs-style tables to LaTeX builder #6666

Closed

Add support for zebra-striped tables to LaTeX builder #6671

Closed

Cherry-pick: Add support for zebra-striped tables to LaTeX builder

993e970

Render tables with alternating background colors for even and odd rows (so called "zebra striping").

jfbu force-pushed the latex_booktabs branch from a1745d4 to 993e970 Compare August 8, 2022 07:45

Fix flake8 reported errors in previous cherry-picked commit

c99f150

jfbu force-pushed the latex_booktabs branch 2 times, most recently from 0561b76 to fae68d2 Compare August 10, 2022 22:01

jfbu marked this pull request as ready for review August 10, 2022 22:02

jfbu force-pushed the latex_booktabs branch from fae68d2 to 13d81f9 Compare August 10, 2022 22:05

jfbu requested a review from tk0miya August 10, 2022 22:06

jfbu added a commit to jfbu/sphinx that referenced this pull request Aug 11, 2022

Update CHANGES for PR sphinx-doc#10759

d34d27b

jfbu added a commit to jfbu/sphinx that referenced this pull request Aug 11, 2022

Update CHANGES for PR sphinx-doc#10759

4ae595c

jfbu force-pushed the latex_booktabs branch from d34d27b to 22339c6 Compare August 11, 2022 17:29

jfbu force-pushed the latex_booktabs branch from 51fd494 to a2a02c0 Compare August 13, 2022 18:47

jfbu force-pushed the latex_booktabs branch from 003e204 to e9d1ff4 Compare August 14, 2022 07:47

jfbu force-pushed the latex_booktabs branch from e9d1ff4 to bfc84de Compare August 14, 2022 10:49

jfbu added 2 commits August 17, 2022 22:27

Fix colour for booktabs gap after top rule if no header

765ccda

Relates sphinx-doc#10759

jfbu added 3 commits August 19, 2022 20:30

Suppress expression of header row color in longtable continuations

c816b7f

One reason in particular is that the colour is applied only up to the long table width, but the continuation hints may be larger than this width (but this width is hidden to not influence the table cells layout).

Merge branch 'latex_booktabs_nowhitegap' into latex_booktabs

c3fa817

jfbu force-pushed the latex_booktabs branch from 55934a9 to 6a6a770 Compare August 20, 2022 17:11

Move \rownum reset away from longtable template into afterendhook

cd5c203

This is better not to overload our template. Besides it makes it easier for user to use another longtable template.

jfbu force-pushed the latex_booktabs branch from 6a6a770 to cd5c203 Compare August 20, 2022 17:20

AA-Turner modified the milestones: 5.2.0, 5.3.0 Sep 23, 2022

Merge branch '5.x' into latex_booktabs

b1579bd

jfbu mentioned this pull request Sep 24, 2022

5.2.0 release plan #10853

Closed

jfbu added 3 commits September 25, 2022 08:24

Merge branch '5.x' into latex_booktabs

e302439

Update CHANGES for PR sphinx-doc#10759

97d2b49

Merge branch '5.x' into latex_booktabs

2f1486b

Improve documentation of new latex_table_style regarding colours

31fd0bf

Remove in passing a note from latex.rst which was of no real use to Sphinx LaTeX users.

jfbu merged commit e7c0881 into sphinx-doc:5.x Oct 12, 2022

jfbu added a commit that referenced this pull request Oct 12, 2022

Update CHANGES for PR #10759

3deb527

jfbu deleted the latex_booktabs branch October 12, 2022 15:29

jfbu added a commit that referenced this pull request Oct 12, 2022

s/5.2.0/5.3.0 in various code comments as #10759 got merged in 5.3.0

b5ba1ed

github-actions bot locked as resolved and limited conversation to collaborators Nov 12, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LaTeX: support for booktabs-style and zebra-striped tables #10759

LaTeX: support for booktabs-style and zebra-striped tables #10759

jfbu commented Aug 8, 2022 •

edited

jfbu commented Aug 11, 2022

jfbu commented Aug 13, 2022 •

edited

jfbu commented Aug 13, 2022 •

edited

jfbu commented Aug 13, 2022

jfbu commented Aug 17, 2022 •

edited

jfbu commented Aug 19, 2022 •

edited

jfbu commented Aug 20, 2022 •

edited

jfbu commented Aug 20, 2022

jfbu commented Sep 24, 2022

jfbu commented Oct 5, 2022 •

edited

jfbu commented Oct 12, 2022

sephalon commented Oct 12, 2022

jfbu commented Oct 12, 2022

LaTeX: support for booktabs-style and zebra-striped tables #10759

LaTeX: support for booktabs-style and zebra-striped tables #10759

Conversation

jfbu commented Aug 8, 2022 • edited

jfbu commented Aug 11, 2022

jfbu commented Aug 13, 2022 • edited

jfbu commented Aug 13, 2022 • edited

jfbu commented Aug 13, 2022

jfbu commented Aug 17, 2022 • edited

jfbu commented Aug 19, 2022 • edited

jfbu commented Aug 20, 2022 • edited

jfbu commented Aug 20, 2022

jfbu commented Sep 24, 2022

jfbu commented Oct 5, 2022 • edited

jfbu commented Oct 12, 2022

sephalon commented Oct 12, 2022

jfbu commented Oct 12, 2022

jfbu commented Aug 8, 2022 •

edited

jfbu commented Aug 13, 2022 •

edited

jfbu commented Aug 13, 2022 •

edited

jfbu commented Aug 17, 2022 •

edited

jfbu commented Aug 19, 2022 •

edited

jfbu commented Aug 20, 2022 •

edited

jfbu commented Oct 5, 2022 •

edited