Add intersphinx role

[tk0miya]

Feature or Bugfix

• Feature

Purpose

• Add :intersphinx:***: role to create a reference for external project without looking up local project.
• It can be used as :intersphinx:py:class:`int`
• refs: domain: c: fix warning for intersphinx targets #8418 (comment)

Remaining Tasks

• testcase
• document
• changes

[tk0miya]

I think :external:***: is also good naming for this.

[jakobandersen]

I think both are ok, but I lean towards changing to external as it seems to convey the semantics of what it is in a more direct manner.

[choldgraf]

given that refs are often quite comment, I think there is value in compactness here. For this reason I think external is better than intersphinx, but I'd also propose that eref is used instead of external (short for "external ref"). What do folks think about that?

[jakobandersen]

See comment below regarding eref.

[tk0miya]

The warning will shown twice because ReferencesResolver will process this again!

[jakobandersen]

Can't we just skip the warning here? I guess missing_reference will take care of it then?

[jakobandersen]

I made some commits at https://github.com/jakobandersen/sphinx/tree/intersphinx_role to implement the following:

• Change to external instead of intersphinx (see comment above).
• A few more tests.
• Remove the explicit warning in the new code.
• Add explicitness to the inventory specification: it's icky that an explicit inventory is indicated by a <key>: prefix as it may be ambiguous with whatever the role has. For this role we have the chance to make it explicit so I suggest the format :invKey:roleArg, i.e., with a leading :. If no specific inventory is desired then roleArg, and if the role argument starts with :, then \:restOfRoleArg.
  Combined with explicit titles with get the full format :invKey:title <target>.
  For now it is simply haxed back to the old format before calling missing_reference but with part of the changes in Intersphinx delegation to domains #8929 we could have a sort of "missing references, any inv" and "missing references, this particular inv" in a later PR.

[jakobandersen]

In a way it seems a bit strange to call the role when it is not supposed to resolve anything. But it is needed for correct formatting, and I guess almost no domain role at the moment performs any real work when it is run, so this works.
We should remember this spot when we at some point overhaul intersphinx.

[astrojuanlu]

I am having some problems with intersphinx on the tutorial (see this thread #9424 (comment)) so I would like to know more about this feature, but I'm not sure about the status of @jakobandersen new proposal. I'd be happy to lend a hand documenting it or giving early feedback if you give me three or four examples of how would it look like in practice (is it :intersphinx:sphinx:doc:contents?).

[jakobandersen]

... how would it look like in practice (is it :intersphinx:sphinx:doc:contents?).

With this PR as it is now it would be :intersphinx:doc:`contents` , or :intersphinx:doc:`sphinx:contents` .
My suggestion is that it becomes :external:doc:`contents` , or :external:doc:`:sphinx:contents` .

Maybe there are too many :, so the inventory prefix should be with different characters as delimiters instead. Already "reserved" characters are ! and ~.

[astrojuanlu]

I see. This is of course bikeshedding but, now that you asked, it could be :external:doc:`sphinx>contents` or :external:doc:`sphinx+contents` (hence > or +).

[jakobandersen]

I see. This is of course bikeshedding but, now that you asked, it could be :external:doc:`sphinx>contents` or :external:doc:`sphinx+contents` (hence > or +).

I think this is the time to bikeshed, and we should indeed get the syntax right.
The problem is that the parsing can be ambiguous. For many roles the argument is simple, but in general you should think of contents being an arbitrary string. For example, the ordinary : immediately gets problematic for C++ where you could have :external:cpp:class:`A::B` . Intersphinx doesn't understand C++, so it can't see if it should extract an inventory specification. Another example, which however is a bit weird in the external role, but not unrealistic in the long run, is the rendering of C or C++ expressions. For example :external:cpp:expr:`x3::float_ + x3::int_` , where the two x3:: names are from the Boost Spirit library.
And this is just the roles that currently exist (and that I know of).

Therefore, I think we must have some kind of special character in the beginning (which optionally can get escaped) to mark whether an inventory specification is present or not. I find :invKey:content problematic because you have : both in the role name and the role argument, so another suggestion is to use one of the brackets:

• <inv>content,
• (inv)content,
• {inv}content, or
• [inv]content.
  The angle brackets may be problematic as you can then have <invKey>title <target> and users need to remember which <> means what.
  Do the others have problems? E.g., while myST is not officially supported by Sphinx it would be nice to avoid brackets that make the syntax weird there.

[choldgraf]

Is this PR primarily trying to resolve the confusion with overloading ref for both local and external refs? Is this going to deprecate the old ref behavior?

[jakobandersen]

Is this PR primarily trying to resolve the confusion with overloading ref for both local and external refs? Is this going to deprecate the old ref behavior?

It originates from #8418 where something like :c:func:`proj1:fn` was tried, which with this PR becomes something like :external:c:func:`:proj1:fn` . But this is a general mechanism for all cross-references generated from role, including :ref:, where the user doesn't want the normal role but an intersphinx lookup. So I think :eref: would look too much like :ref: when it's a general mechanism.
In time (after some intersphinx refactoring) I think it may be appropriate to deprecate for old syntax, but for the roles where it currently works I don't see a reason so deprecate it yet.

[choldgraf]

@jakobandersen that makes sense to me. I guess :ext: would be too short?

One other quick thought - maybe this is technically infeasible, but rather than using the content block of the external ref to define the "intersphinx registry" to use, would it be possible to generate a role domain on-the-fly for each intersphinx mapping?

e.g., it would be quite intuitive (to me) if you could do:

Here's an external ref to a label :external:somedocwebsite:ref`someref`

And one to a doc :external:anotherdocwebsite:doc`somedoc`

and so on.

[jakobandersen]

@jakobandersen that makes sense to me. I guess :ext: would be too short?

I wouldn't mind it being ext.

One other quick thought - maybe this is technically infeasible, but rather than using the content block of the external ref to define the "intersphinx registry" to use, would it be possible to generate a role domain on-the-fly for each intersphinx mapping?

It would indeed be prettier to have it as part of the role. It already dynamically checks for the domain role, so it could check for the inventory name as well. The icky part is that the inventory name is optional, and the domain role may be just the second component if there is a default_domain. E.g., :ext:meth: is the same as :ext:py:meth: when you have executed .. default_domain:: py at some point.
But, other that : it looks like we can use the characters -._+ in role names, and it indeed works with those characters in my local test.
So, for :ext:py:meth: with an explicit inventory we could have

• :ext:inv-py:meth:
• :ext:inv.py:meth:
• :ext:inv_py:meth:
• :ext:inv+py:meth:

Not sure what I think about those options yet, but if possible I think we indeed should keep the inventory out of the argument.

[astrojuanlu]

+1 for :ext:inv.py:meth:!

[jakobandersen]

I have made #9822 as a continuation based on the discussion. I think it is basically complete.