DOC: -np.inf as default argument not rendering correctly in DIRECT docstring

[dschmitz89]

There is a minor annoyance in the docstring of the recently added direct algorithm:

sphinx cannot correctly parse -np.inf . I tried to google the issue but could not find an immediate fix. Maybe a more experienced sphinx user has an idea?

[WarrenWeckesser]

Other occurrences of the problem with -np.inf:

• https://scipy.github.io/devdocs/reference/generated/scipy.optimize.LinearConstraint.html
• https://scipy.github.io/devdocs/reference/generated/scipy.optimize.Bounds.html

It looks like there are two issues here:

• Sphinx/autodoc evaluates default values, and in the rendered docs, it displays the result of the evaluation, not the original literal text. In Sphinx 4.0+, this can be disabled with configuration variable autodoc_preserve_defaults (see Fix #759: autodoc: Add sphinx.ext.autodoc.preserve_defaults extension sphinx-doc/sphinx#8771).

• With negative default values, a spurious space is inserted between the minus sign and the first digit. Examples of this issue that do not involve -np.inf include

  • https://scipy.github.io/devdocs/reference/generated/scipy.optimize.dual_annealing.html; see the accept parameter;
  • https://scipy.github.io/devdocs/reference/generated/scipy.stats.mstats.hdmedian.html; see the axis parameter.

  The same problem shows up in the NumPy docs; see the axis parameter in these docstrings:

  • https://numpy.org/doc/stable/reference/generated/numpy.trapz.html
  • https://numpy.org/doc/stable/reference/generated/numpy.unwrap.html

  But not all negative default values result in the spurious space:

  • https://numpy.org/doc/stable/reference/generated/numpy.diff.html

Maybe the numpydoc devs have seen this before: @jarrodmillman, @larsoner, @rossbar, is this a known issue?

[WarrenWeckesser]

Another example: the spurious space shows up in the rendering of the na_sentinel default value in the Pandas docs for pandas.factorize.

[WarrenWeckesser]

I added the "upstream bug" tag. This is probably a bug in Sphinx itself or in the theme used by SciPy, NumPy and Pandas.

[tupui]

If there is nothing we can/should do, then we should close it IMO.

[WarrenWeckesser]

It might be better to leave it open until the cause of the problem has been identified, and we know which version of which upstream tool fixes the problem.

[dschmitz89]

Other occurrences of the problem with -np.inf:

• https://scipy.github.io/devdocs/reference/generated/scipy.optimize.LinearConstraint.html
• https://scipy.github.io/devdocs/reference/generated/scipy.optimize.Bounds.html

It looks like there are two issues here:

• Sphinx/autodoc evaluates default values, and in the rendered docs, it displays the result of the evaluation, not the original literal text. In Sphinx 4.0+, this can be disabled with configuration variable autodoc_preserve_defaults (see Fix #759: autodoc: Add sphinx.ext.autodoc.preserve_defaults extension sphinx-doc/sphinx#8771).

Would adding autodoc_preserve_defaults = True to conf.py be enough to fix this?

[WarrenWeckesser]

@dschmitz89, it turns out that the problem is a bug in Sphinx: sphinx-doc/sphinx#10550

The proposed fix is in sphinx-doc/sphinx#10551

I'll close this issue when that Sphinx pull request is merged, because then we'll have a definitive resolution of the issue: the problem is fixed in Sphinx 5.1.0.

Using autodoc_preserve_defaults might, in fact, fix the issue reported here by avoiding the sphinx bug altogether, but I haven't tried it. I think the use of that option would require some discussion and some trial runs to see how it affects the rendered default values throughout the SciPy docs.

[dschmitz89]

@WarrenWeckesser : from what I understood, the Sphinx bug will fix the additional space but we would still see -inf instead of -np.inf. Changing sphinx config can be painful, I agree, maybe @tupui or another seasoned scipy doc expert has an opinion.

[tupui]

I would be fine with seeing -inf instead of -np.inf. There is no such thing in Python's grammar to confuse it with.

[AA-Turner]

autodoc_preserve_defaults wouldn't work as it still uses the AST, example reproducer script below.

import shutil
from pathlib import Path

from sphinx.cmd.make_mode import run_make_mode

def write(filename, text): Path(filename).write_text(text, encoding="utf-8")

write("conf.py", '''\
import os, sys
sys.path.insert(0, os.path.abspath(".."))
extensions = ["sphinx.ext.autodoc"]
autodoc_preserve_defaults = True
''')

write("extra_white.py", '''\
def func(axis=-0x1): ...
''')

write("index.rst", '''\
.. autofunction:: extra_white.func
''')

shutil.rmtree("_build", ignore_errors=True)
run_make_mode(["html", ".", "_build", "-T", "-W"])

A

[WarrenWeckesser]

Thanks @AA-Turner, that makes sense. So if we used that option in SciPy now we would get f_min=- np.inf in the rendered direct docstring.

[tupui]

I confirmed the spacing issue will be resolved with Sphinx 5.1.0 (see sphinx-doc/sphinx#10551 (comment)).

[dschmitz89]

-inf instead of -np.inf might not be too bad but what if a default argument is some complex class like scipy.stats.rv_continuous ?

[tupui]

-inf instead of -np.inf might not be too bad but what if a default argument is some complex class like scipy.stats.rv_continuous ?

Well this should not really happen as it would be a bad design. Default arguments should always be simple things like string, scalars, booleans, None. Anything else would break (e.g. using an empty dict which leads to bugs) and is an anti pattern.

[WarrenWeckesser]

Over in the Sphinx repo, sphinx-doc/sphinx#10551 has been merged and the fix for the extra space will be in Sphinx 5.1.0.

I'm closing this issue, because I think the extra space was the main problem. Of course, our web site won't reflect this change until we build it with Sphinx 5.1, so if anyone is even more conservative about closing this than I am and thinks it should remain open until our web site is actually fixed, feel free to reopen the issue.

I think the question of enabling autodoc_preserve_defaults should be discussed in a new issue.

[tupui]

I am even less conservative so 😉
But the next dev on master will have it, and also 1.9 since we don't have upper pins for Sphinx. So all good.

What we might want to discuss on the ML is the other issue/enhancement.

[WarrenWeckesser]

But the next dev on master will have it, and also 1.9 since we don't have upper pins for Sphinx. So all good.

The fix will be in Sphinx 5.1.0, which is not released yet. It looks like Sphinx has a release cadence of about two months, and 5.0.0 was released May 29, so I wouldn't expect 5.1.0 to be released until the end of July.

[WarrenWeckesser]

Status update: It looks like the 1.9.0 docs were built with Sphinx <5.1.0, so they still have the spurious space, e.g. https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.direct.html.

A newer version of Sphinx is used to build the development version of the docs, so in those the problem is fixed: https://scipy.github.io/devdocs/reference/generated/scipy.optimize.direct.html

[tupui]

Maybe we should just bump the minimal version for Sphinx.