Add a feature flag to install docutils<0.18 on sphinx 2 projects

[ericholscher]

This will hopefully allow us to roll out changes to fix old sphinx projects that are broken.

The rollout plan will be:

• Deploy this change
• Apply it to known broken projects, to ensure it fixes the issue and doesn't cause new ones
• Apply it to a sample of projects, and ensure they continue to build
• Roll it out as a default to all projects, but it will only apply to ones not using sphinx latest

[stsewd]

Parenthesis are missing for the multiline expression

[humitos]

This looks good to me.

I had to do some small checks before approving this PR:

• ✅ we have USE_SPHINX_LATEST enabled in our production db for all new projects after Oct. 20, 2020, 8:30 p.m..
• ✅ we are installing Sphinx<2 for those projects not using USE_SPHINX_LATEST:

  readthedocs.org/readthedocs/doc_builder/python_environments.py

  Lines 507 to 511 in a81f14c

  	self.project.get_feature_value(
  	Feature.USE_SPHINX_LATEST,
  	positive='sphinx',
  	negative='sphinx<2',
  	),

• unrelated to this, but may worth to mention it here as well, we install an old version of our theme when not USE_SPHINX_LATEST:

  readthedocs.org/readthedocs/doc_builder/python_environments.py

  Lines 514 to 518 in a81f14c

  	self.project.get_feature_value(
  	Feature.USE_SPHINX_LATEST,
  	positive='sphinx-rtd-theme',
  	negative='sphinx-rtd-theme<0.5',
  	),

[agjohnson]

It was just noted in chat, but seems there has been some movement at doctuils and Sphinx on updating some of the conflicts between the packages: sphinx-doc/sphinx#9807

So, we might see a docutils 0.18.1 soon, and seems a Sphinx 1.8.6/etc as well.

Looks like our feature flag pinned 1.x Sphinx will get a 1.8.6 release, so we might be close to settled on this.

unrelated to this, but may worth to mention it here as well, we install an old version of our theme when not USE_SPHINX_LATEST

So, that seems like it was needed because sphinx_rtd_theme 0.4.3 doesn't support Sphinx 2. I'm not sure why we didn't, but seems we might want to backport a fix and release a 0.4.4 pinning Sphinx < 2. Also though, Sphinx 1.8 is supported by the latest release so I don't even know if this is even applicable anymore.

[paulmueller]

@ericholscher If you are referring to readthedocs/sphinx_rtd_theme#1115, wouldn't you have to pin to docutils 0.16.0?

[ericholscher]

Looks like the bugfix release of sphinx releases should have fixed this issue in a better way. Closing this 🎉

sphinx-doc/sphinx#9807