diff --git a/CHANGES b/CHANGES index d2ddae71c8..54b09e5563 100644 --- a/CHANGES +++ b/CHANGES @@ -47,6 +47,8 @@ Features added * #9391: texinfo: improve variable in ``samp`` role * #9578: texinfo: Add :confval:`texinfo_cross_references` to disable cross references for readability with standalone readers +* #9822 (and #9062), add new Intersphinx role :rst:role:`external` for explict + lookup in the external projects, without resolving to the local project. Bugs fixed ---------- diff --git a/doc/usage/extensions/intersphinx.rst b/doc/usage/extensions/intersphinx.rst index a3e65bed64..2bcce68d0e 100644 --- a/doc/usage/extensions/intersphinx.rst +++ b/doc/usage/extensions/intersphinx.rst @@ -8,20 +8,25 @@ .. versionadded:: 0.5 -This extension can generate automatic links to the documentation of objects in -other projects. - -Usage is simple: whenever Sphinx encounters a cross-reference that has no -matching target in the current documentation set, it looks for targets in the -documentation sets configured in :confval:`intersphinx_mapping`. A reference -like ``:py:class:`zipfile.ZipFile``` can then link to the Python documentation +This extension can generate links to the documentation of objects in external +projects, either explicitly through the :rst:role:`external` role, or as a +fallback resolution for any other cross-reference. + +Usage for fallback resolution is simple: whenever Sphinx encounters a +cross-reference that has no matching target in the current documentation set, +it looks for targets in the external documentation sets configured in +:confval:`intersphinx_mapping`. A reference like +``:py:class:`zipfile.ZipFile``` can then link to the Python documentation for the ZipFile class, without you having to specify where it is located exactly. -When using the "new" format (see below), you can even force lookup in a foreign -set by prefixing the link target appropriately. A link like ``:ref:`comparison -manual ``` will then link to the label "comparisons" in the -doc set "python", if it exists. +When using the :rst:role:`external` role, you can force lookup to any external +projects, and optionally to a specific external project. +A link like ``:external:ref:`comparison manual ``` will then link +to the label "comparisons" in whichever configured external project, if it +exists, +and a link like ``:external+python:ref:`comparison manual ``` will +link to the label "comparisons" only in the doc set "python", if it exists. Behind the scenes, this works as follows: @@ -30,8 +35,8 @@ Behind the scenes, this works as follows: * Projects using the Intersphinx extension can specify the location of such mapping files in the :confval:`intersphinx_mapping` config value. The mapping - will then be used to resolve otherwise missing references to objects into - links to the other documentation. + will then be used to resolve both :rst:role:`external` references, and also + otherwise missing references to objects into links to the other documentation. * By default, the mapping file is assumed to be at the same location as the rest of the documentation; however, the location of the mapping file can also be @@ -79,10 +84,10 @@ linking: at the same location as the base URI) or another local file path or a full HTTP URI to an inventory file. - The unique identifier can be used to prefix cross-reference targets, so that + The unique identifier can be used in the :rst:role:`external` role, so that it is clear which intersphinx set the target belongs to. A link like - ``:ref:`comparison manual ``` will link to the label - "comparisons" in the doc set "python", if it exists. + ``external:python+ref:`comparison manual ``` will link to the + label "comparisons" in the doc set "python", if it exists. **Example** @@ -162,21 +167,50 @@ linking: The default value is an empty list. - When a cross-reference without an explicit inventory specification is being - resolved by intersphinx, skip resolution if it matches one of the - specifications in this list. + When a non-:rst:role:`external` cross-reference is being resolved by + intersphinx, skip resolution if it matches one of the specifications in this + list. For example, with ``intersphinx_disabled_reftypes = ['std:doc']`` a cross-reference ``:doc:`installation``` will not be attempted to be - resolved by intersphinx, but ``:doc:`otherbook:installation``` will be - attempted to be resolved in the inventory named ``otherbook`` in + resolved by intersphinx, but ``:external+otherbook:doc:`installation``` will + be attempted to be resolved in the inventory named ``otherbook`` in :confval:`intersphinx_mapping`. At the same time, all cross-references generated in, e.g., Python, declarations will still be attempted to be resolved by intersphinx. - If ``*`` is in the list of domains, then no references without an explicit - inventory will be resolved by intersphinx. + If ``*`` is in the list of domains, then no non-:rst:role:`external` + references will be resolved by intersphinx. + +Explicitly Reference External Objects +------------------------------------- + +The Intersphinx extension provides the following role. + +.. rst:role:: external + + .. versionadded:: 4.4 + + Use Intersphinx to perform lookup only in external projects, and not the + current project. Intersphinx still needs to know the type of object you + would like to find, so the general form of this role is to write the + cross-refererence as if the object is in the current project, but then prefix + it with ``:external``. + The two forms are then + + - ``:external:domain:reftype:`target```, + e.g., ``:external:py:class:`zipfile.ZipFile```, or + - ``:external:reftype:`target```, + e.g., ``:external:doc:`installation```. + + If you would like to constrain the lookup to a specific external project, + then the key of the project, as specified in :confval:`intersphinx_mapping`, + is added as well to get the two forms + - ``:external+invname:domain:reftype:`target```, + e.g., ``:external+python:py:class:`zipfile.ZipFile```, or + - ``:external+invname:reftype:`target```, + e.g., ``:external+python:doc:`installation```. Showing all links of an Intersphinx mapping file ------------------------------------------------ diff --git a/sphinx/ext/intersphinx.py b/sphinx/ext/intersphinx.py index 7f3588ade7..2f8ab2588e 100644 --- a/sphinx/ext/intersphinx.py +++ b/sphinx/ext/intersphinx.py @@ -26,15 +26,17 @@ import concurrent.futures import functools import posixpath +import re import sys import time from os import path -from typing import IO, Any, Dict, List, Optional, Tuple +from types import ModuleType +from typing import IO, Any, Dict, List, Optional, Tuple, cast from urllib.parse import urlsplit, urlunsplit from docutils import nodes -from docutils.nodes import Element, TextElement -from docutils.utils import relative_path +from docutils.nodes import Element, Node, TextElement, system_message +from docutils.utils import Reporter, relative_path import sphinx from sphinx.addnodes import pending_xref @@ -43,10 +45,13 @@ from sphinx.config import Config from sphinx.domains import Domain from sphinx.environment import BuildEnvironment +from sphinx.errors import ExtensionError from sphinx.locale import _, __ +from sphinx.transforms.post_transforms import ReferencesResolver from sphinx.util import logging, requests +from sphinx.util.docutils import CustomReSTDispatcher, SphinxRole from sphinx.util.inventory import InventoryFile -from sphinx.util.typing import Inventory, InventoryItem +from sphinx.util.typing import Inventory, InventoryItem, RoleFunction logger = logging.getLogger(__name__) @@ -466,6 +471,144 @@ def missing_reference(app: Sphinx, env: BuildEnvironment, node: pending_xref, return resolve_reference_detect_inventory(env, node, contnode) +class IntersphinxDispatcher(CustomReSTDispatcher): + """Custom dispatcher for external role. + + This enables :external:***:/:external+***: roles on parsing reST document. + """ + + def role(self, role_name: str, language_module: ModuleType, lineno: int, reporter: Reporter + ) -> Tuple[RoleFunction, List[system_message]]: + if len(role_name) > 9 and role_name.startswith(('external:', 'external+')): + return IntersphinxRole(role_name), [] + else: + return super().role(role_name, language_module, lineno, reporter) + + +class IntersphinxRole(SphinxRole): + # group 1: just for the optionality of the inventory name + # group 2: the inventory name (optional) + # group 3: the domain:role or role part + _re_inv_ref = re.compile(r"(\+([^:]+))?:(.*)") + + def __init__(self, orig_name: str) -> None: + self.orig_name = orig_name + + def run(self) -> Tuple[List[Node], List[system_message]]: + assert self.name == self.orig_name.lower() + inventory, name_suffix = self.get_inventory_and_name_suffix(self.orig_name) + if inventory and not inventory_exists(self.env, inventory): + logger.warning(__('inventory for external cross-reference not found: %s'), + inventory, location=(self.env.docname, self.lineno)) + return [], [] + + role_name = self.get_role_name(name_suffix) + if role_name is None: + logger.warning(__('role for external cross-reference not found: %s'), name_suffix, + location=(self.env.docname, self.lineno)) + return [], [] + + result, messages = self.invoke_role(role_name) + for node in result: + if isinstance(node, pending_xref): + node['intersphinx'] = True + node['inventory'] = inventory + + return result, messages + + def get_inventory_and_name_suffix(self, name: str) -> Tuple[Optional[str], str]: + assert name.startswith('external'), name + assert name[8] in ':+', name + # either we have an explicit inventory name, i.e, + # :external+inv:role: or + # :external+inv:domain:role: + # or we look in all inventories, i.e., + # :external:role: or + # :external:domain:role: + inv, suffix = IntersphinxRole._re_inv_ref.fullmatch(name, 8).group(2, 3) + return inv, suffix + + def get_role_name(self, name: str) -> Optional[Tuple[str, str]]: + names = name.split(':') + if len(names) == 1: + # role + default_domain = self.env.temp_data.get('default_domain') + domain = default_domain.name if default_domain else None + role = names[0] + elif len(names) == 2: + # domain:role: + domain = names[0] + role = names[1] + else: + return None + + if domain and self.is_existent_role(domain, role): + return (domain, role) + elif self.is_existent_role('std', role): + return ('std', role) + else: + return None + + def is_existent_role(self, domain_name: str, role_name: str) -> bool: + try: + domain = self.env.get_domain(domain_name) + if role_name in domain.roles: + return True + else: + return False + except ExtensionError: + return False + + def invoke_role(self, role: Tuple[str, str]) -> Tuple[List[Node], List[system_message]]: + domain = self.env.get_domain(role[0]) + if domain: + role_func = domain.role(role[1]) + + return role_func(':'.join(role), self.rawtext, self.text, self.lineno, + self.inliner, self.options, self.content) + else: + return [], [] + + +class IntersphinxRoleResolver(ReferencesResolver): + """pending_xref node resolver for intersphinx role. + + This resolves pending_xref nodes generated by :intersphinx:***: role. + """ + + default_priority = ReferencesResolver.default_priority - 1 + + def run(self, **kwargs: Any) -> None: + for node in self.document.traverse(pending_xref): + if 'intersphinx' not in node: + continue + contnode = cast(nodes.TextElement, node[0].deepcopy()) + inv_name = node['inventory'] + if inv_name is not None: + assert inventory_exists(self.env, inv_name) + newnode = resolve_reference_in_inventory(self.env, inv_name, node, contnode) + else: + newnode = resolve_reference_any_inventory(self.env, False, node, contnode) + if newnode is None: + typ = node['reftype'] + msg = (__('external %s:%s reference target not found: %s') % + (node['refdomain'], typ, node['reftarget'])) + logger.warning(msg, location=node, type='ref', subtype=typ) + node.replace_self(contnode) + else: + node.replace_self(newnode) + + +def install_dispatcher(app: Sphinx, docname: str, source: List[str]) -> None: + """Enable IntersphinxDispatcher. + + .. note:: The installed dispatcher will uninstalled on disabling sphinx_domain + automatically. + """ + dispatcher = IntersphinxDispatcher() + dispatcher.enable() + + def normalize_intersphinx_mapping(app: Sphinx, config: Config) -> None: for key, value in config.intersphinx_mapping.copy().items(): try: @@ -497,7 +640,9 @@ def setup(app: Sphinx) -> Dict[str, Any]: app.add_config_value('intersphinx_disabled_reftypes', [], True) app.connect('config-inited', normalize_intersphinx_mapping, priority=800) app.connect('builder-inited', load_mappings) + app.connect('source-read', install_dispatcher) app.connect('missing-reference', missing_reference) + app.add_post_transform(IntersphinxRoleResolver) return { 'version': sphinx.__display_version__, 'env_version': 1, diff --git a/sphinx/util/docutils.py b/sphinx/util/docutils.py index c3a6ff9e2e..5ab7666496 100644 --- a/sphinx/util/docutils.py +++ b/sphinx/util/docutils.py @@ -166,16 +166,14 @@ def patch_docutils(confdir: Optional[str] = None) -> Generator[None, None, None] yield -class ElementLookupError(Exception): - pass - +class CustomReSTDispatcher: + """Custom reST's mark-up dispatcher. -class sphinx_domains: - """Monkey-patch directive and role dispatch, so that domain-specific - markup takes precedence. + This replaces docutils's directives and roles dispatch mechanism for reST parser + by original one temporarily. """ - def __init__(self, env: "BuildEnvironment") -> None: - self.env = env + + def __init__(self) -> None: self.directive_func: Callable = lambda *args: (None, []) self.roles_func: Callable = lambda *args: (None, []) @@ -189,13 +187,35 @@ def enable(self) -> None: self.directive_func = directives.directive self.role_func = roles.role - directives.directive = self.lookup_directive - roles.role = self.lookup_role + directives.directive = self.directive + roles.role = self.role def disable(self) -> None: directives.directive = self.directive_func roles.role = self.role_func + def directive(self, + directive_name: str, language_module: ModuleType, document: nodes.document + ) -> Tuple[Optional[Type[Directive]], List[system_message]]: + return self.directive_func(directive_name, language_module, document) + + def role(self, role_name: str, language_module: ModuleType, lineno: int, reporter: Reporter + ) -> Tuple[RoleFunction, List[system_message]]: + return self.role_func(role_name, language_module, lineno, reporter) + + +class ElementLookupError(Exception): + pass + + +class sphinx_domains(CustomReSTDispatcher): + """Monkey-patch directive and role dispatch, so that domain-specific + markup takes precedence. + """ + def __init__(self, env: "BuildEnvironment") -> None: + self.env = env + super().__init__() + def lookup_domain_element(self, type: str, name: str) -> Any: """Lookup a markup element (directive or role), given its name which can be a full name (with domain). @@ -226,17 +246,20 @@ def lookup_domain_element(self, type: str, name: str) -> Any: raise ElementLookupError - def lookup_directive(self, directive_name: str, language_module: ModuleType, document: nodes.document) -> Tuple[Optional[Type[Directive]], List[system_message]]: # NOQA + def directive(self, + directive_name: str, language_module: ModuleType, document: nodes.document + ) -> Tuple[Optional[Type[Directive]], List[system_message]]: try: return self.lookup_domain_element('directive', directive_name) except ElementLookupError: - return self.directive_func(directive_name, language_module, document) + return super().directive(directive_name, language_module, document) - def lookup_role(self, role_name: str, language_module: ModuleType, lineno: int, reporter: Reporter) -> Tuple[RoleFunction, List[system_message]]: # NOQA + def role(self, role_name: str, language_module: ModuleType, lineno: int, reporter: Reporter + ) -> Tuple[RoleFunction, List[system_message]]: try: return self.lookup_domain_element('role', role_name) except ElementLookupError: - return self.role_func(role_name, language_module, lineno, reporter) + return super().role(role_name, language_module, lineno, reporter) class WarningStream: diff --git a/tests/roots/test-ext-intersphinx-role/conf.py b/tests/roots/test-ext-intersphinx-role/conf.py new file mode 100644 index 0000000000..a54f5c2ad6 --- /dev/null +++ b/tests/roots/test-ext-intersphinx-role/conf.py @@ -0,0 +1,3 @@ +extensions = ['sphinx.ext.intersphinx'] +# the role should not honor this conf var +intersphinx_disabled_reftypes = ['*'] diff --git a/tests/roots/test-ext-intersphinx-role/index.rst b/tests/roots/test-ext-intersphinx-role/index.rst new file mode 100644 index 0000000000..58edb7a1a0 --- /dev/null +++ b/tests/roots/test-ext-intersphinx-role/index.rst @@ -0,0 +1,44 @@ +- ``module1`` is only defined in ``inv``: + :external:py:mod:`module1` + +.. py:module:: module2 + +- ``module2`` is defined here and also in ``inv``, but should resolve to inv: + :external:py:mod:`module2` + +- ``module3`` is not defined anywhere, so should warn: + :external:py:mod:`module3` + +.. py:module:: module10 + +- ``module10`` is only defined here, but should still not be resolved to: + :external:py:mod:`module10` + +- a function in inv: + :external:py:func:`module1.func` +- a method, but with old style inventory prefix, which shouldn't work: + :external:py:meth:`inv:Foo.bar` +- a non-existing role: + :external:py:nope:`something` + +.. default-domain:: cpp + +- a type where the default domain is used to find the role: + :external:type:`std::uint8_t` +- a non-existing role in default domain: + :external:nope:`somethingElse` + +- two roles in ``std`` which can be found without a default domain: + + - :external:doc:`docname` + - :external:option:`ls -l` + + +- a function with explicit inventory: + :external+inv:c:func:`CFunc` +- a class with explicit non-existing inventory, which also has upper-case in name: + :external+invNope:cpp:class:`foo::Bar` + + +- explicit title: + :external:cpp:type:`FoonsTitle ` diff --git a/tests/test_ext_intersphinx.py b/tests/test_ext_intersphinx.py index 7f369e9a3c..b2ad8afe52 100644 --- a/tests/test_ext_intersphinx.py +++ b/tests/test_ext_intersphinx.py @@ -524,3 +524,48 @@ def log_message(*args, **kwargs): stdout, stderr = capsys.readouterr() assert stdout.startswith("c:function\n") assert stderr == "" + + +@pytest.mark.sphinx('html', testroot='ext-intersphinx-role') +def test_intersphinx_role(app, warning): + inv_file = app.srcdir / 'inventory' + inv_file.write_bytes(inventory_v2) + app.config.intersphinx_mapping = { + 'inv': ('http://example.org/', inv_file), + } + app.config.intersphinx_cache_limit = 0 + app.config.nitpicky = True + + # load the inventory and check if it's done correctly + normalize_intersphinx_mapping(app, app.config) + load_mappings(app) + + app.build() + content = (app.outdir / 'index.html').read_text() + wStr = warning.getvalue() + + html = '' + assert html.format('foo.html#module-module1') in content + assert html.format('foo.html#module-module2') in content + assert "WARNING: external py:mod reference target not found: module3" in wStr + assert "WARNING: external py:mod reference target not found: module10" in wStr + + assert html.format('sub/foo.html#module1.func') in content + assert "WARNING: external py:meth reference target not found: inv:Foo.bar" in wStr + + assert "WARNING: role for external cross-reference not found: py:nope" in wStr + + # default domain + assert html.format('index.html#std_uint8_t') in content + assert "WARNING: role for external cross-reference not found: nope" in wStr + + # std roles without domain prefix + assert html.format('docname.html') in content + assert html.format('index.html#cmdoption-ls-l') in content + + # explicit inventory + assert html.format('cfunc.html#CFunc') in content + assert "WARNING: inventory for external cross-reference not found: invNope" in wStr + + # explicit title + assert html.format('index.html#foons') in content