Spurious space in default parameter values that are negative numbers in HTML output.

[WarrenWeckesser]

Describe the bug

For several projects, I've noticed a problem in the HTML output of functions that have a parameter with a default value that is a negative number. In the rendered HTML, there is a spurious space between the minus sign and the first digit. A typical example is axis=-1 being rendered as axis=- 1. This issue was originally raised with SciPy.

Here are links to examples in several projects:

SciPy:

• https://scipy.github.io/devdocs/reference/generated/scipy.optimize.direct.html:
  see f_min.
• https://scipy.github.io/devdocs/reference/generated/scipy.optimize.LinearConstraint.html:
  see lb.

NumPy:

• https://numpy.org/doc/stable/reference/generated/numpy.unwrap.html

Pandas:

• https://pandas.pydata.org/docs/reference/api/pandas.factorize.html

Matplotlib:

• https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.waitforbuttonpress.html

I wasn't able to find an existing issue for this.

I don't know which versions of Sphinx were used in all those projects, so I don't know if the problem still exists in the latest version of Sphinx. Also, it looks like those projects all use the PyData Sphinx theme, so it is possible that the problem is the theme and not Sphinx itself.

How to Reproduce

See the links.

Expected behavior

No response

Your project

See the links

Screenshots

No response

OS

See the above comments.

Python version

Probably varied; see the links.

Sphinx version

Maybe several; see the links.

Sphinx extensions

No response

Extra tools

No response

Additional context

No response

[AA-Turner]

Please provide a minimal reproducer project, and additionally test with 'basic' or 'alabaster' to ensure it is not a bug in the pydata theme.

A

[WarrenWeckesser]

@AA-Turner, sorry the bug report is not more thorough. I probably won't be able to dig into this any further. Perhaps some of the PyData theme maintainers could take a look and check if this issue is more appropriate for that project: ping @choldgraf, @12rambau

[WarrenWeckesser]

One more data point: the issue also occurs in the Jax documentation, e.g.

• https://jax.readthedocs.io/en/latest/_autosummary/jax.numpy.cross.html
• https://jax.readthedocs.io/en/latest/_autosummary/jax.random.categorical.html

According to their 'conf.py', the theme used by Jax is 'sphinx_book_theme'.

[choldgraf]

Sounds like a pydata theme issue indeed! Please open an issue there so others can discuss

https://github.com/pydata/pydata-sphinx-theme

Also the book theme inherits from the pydata theme so it'd make sense that they have the same issue

[WarrenWeckesser]

I have opened a corresponding issue in the pydata theme repo, so I'll close this issue. We can reopen it if the pydata theme devs figure out that the problem is in Sphinx and not the theme.

[choldgraf]

Actually I just looked into it a little bit, and I think it might be a bug in autodoc and the pygments styling/structure. Here's the HTML of that section in the pydata theme:

<span class="default_value"><span class="pre">-</span> <span class="pre">inf</span></span>

Importantly, note there is a space between the two span elements. I think that this is generated HTML by autodoc and not theme-specific after all, right?

[AA-Turner]

@choldgraf do you have the reST source that the snippet was generated from?

A

[choldgraf]

Well as one example from SciPy:

• The rendered docstring is here: https://scipy.github.io/devdocs/reference/generated/scipy.optimize.direct.html
• The python source is here: https://github.com/scipy/scipy/blob/main/scipy/optimize/_direct_py.py#L41-L282
• The rST source is: https://github.com/scipy/scipy/blob/main/doc/source/reference/optimize.minimize-tnc.rst (this isn't quite for the same function but I think this is the general pattern they're following)
• Here's a big rST index that they use to create links as well: https://raw.githubusercontent.com/scipy/scipy/main/doc/API.rst.txt

[AA-Turner]

Minimal reproducer:

import shutil
from pathlib import Path

from sphinx.cmd.make_mode import run_make_mode

def write(filename, text): Path(filename).write_text(text, encoding="utf-8")

write("conf.py", '''\
import os, sys
sys.path.insert(0, os.path.abspath(".."))
extensions = ["sphinx.ext.autodoc"]
''')

write("extra_white.py", '''\
def func(axis=-1):
    ...
''')

write("index.rst", '''\
.. autofunction:: extra_white.func
''')

shutil.rmtree("_build", ignore_errors=True)
run_make_mode(["html", ".", "_build", "-T", "-W"])

The spurious extra whitespace is present as Chris notes as a literal space character between the two span elements.

A

[AA-Turner]

This is actually a problem in the ast unparser rather than autodoc.

>>> import ast
>>> import sphinx.pycode.ast
>>> default = ast.parse("def func(axis=-1): ...", "<string>").body[0].args.defaults[0]
>>> print(ast.dump(default))
UnaryOp(op=USub(), operand=Constant(value=1))
>>> ast.unparse(default)
'-1'
>>> sphinx.pycode.ast.unparse(default, "def func(axis=-1): ...")
'- 1'

A

[choldgraf]

Oooh this is a good one!

[ezio-melotti]

I just run into this same issue while reviewing python/python-docs-zh-tw#260, since I noticed a read(n=- 1) in the output. I thought it was a translation error but noticed the problem exists in the main Python docs too: https://docs.python.org/3/library/asyncio-stream.html#asyncio.StreamReader.read.

The linked PR seems to fix the issue, however something similar also happens with binary operators (and other arbitrary expressions), such as

.. function:: func(x=2**16)
.. function:: func(x=1<<16)
.. function:: func(x=2**16-1)

which get rendered as

func(x=2 ** 16)
func(x=2 << 16)
func(x=2 ** 16 - 1)

Ideally it would be better if Sphinx just preserved the original spacing, but I understand this might not be easily doable and/or not worth the trouble.
Perhaps ** could be special-cased too since it's usually written without spaces, but the other cases can be left alone, also because they are quite uncommon.

---

While investigating the problem and before coming across this issue, I also wrote a test that I added in tests/test_domain_py.py (just after test_pyfunction_with_number_literals):

def test_pyfunction_with_signed_numbers(app):
    text = ".. py:function:: hello(age=-1, height=+1)"
    doctree = restructuredtext.parse(app, text)
    assert_node(doctree[1][0][1],
                [desc_parameterlist, ([desc_parameter, ([desc_sig_name, "age"],
                                                        [desc_sig_operator, "="],
                                                        [nodes.inline, "-1"])],
                                      [desc_parameter, ([desc_sig_name, "height"],
                                                        [desc_sig_operator, "="],
                                                        [nodes.inline, "+1"])])])

I'm not sure if it's worth adding this "higher-level" test to the PR too, but if you think it would be useful let me know -- I can also tweak it to add a few more cases.

[AA-Turner]

Using stdlib ast:

In [17]: ast.unparse(ast.parse("def func(axis=1<<16): ...", "<string>").body[0].args.defaults[0])
Out[17]: '1 << 16'
In [18]: ast.unparse(ast.parse("def func(axis=2**16): ...", "<string>").body[0].args.defaults[0])
Out[18]: '2 ** 16'

We could similarly special case visit_BinOp for Pow(), LShift() and RShift() but I am not sure. Really the solution is to use a CST parser but the stdlib doesn't have one, and I'm not sure about adding a runtime dependency on LibCST.

A