New Sphinx tutorial (intersphinx, postponed) #9424

astrojuanlu · 2021-07-08T20:15:42Z

Feature or Bugfix

Feature

Purpose

(Still draft) This is a continuation of the work we did in #9276 and #9355. It expands some sections already started in the first part and adds some new ones, as originally proposed in #9165:

Writing narrative documentation with Sphinx (adding a section on intersphinx)
Describing code in Sphinx

I plan to add quite some content to (7) (still not pushed), and leave "Autogenerating documentation from code in Sphinx" for the fourth PR. Again, the idea is to keep these pull requests manageable, iterate in small chunks, and in the meanwhile keep the tutorial in a complete state.

Rendered version: https://sphinx--9424.org.readthedocs.build/en/9424/tutorial/index.html

cc @ericholscher

Relates

ericholscher

Looks like a good section. I wonder if we should reference the RTD guide on this to learn more? Or maybe backport our guide into the Sphinx docs for intersphinx?

doc/tutorial/index.rst

Co-authored-by: Eric Holscher <25510+ericholscher@users.noreply.github.com>

astrojuanlu · 2021-07-08T21:06:35Z

Thanks @ericholscher for the early review, I addressed your comments. About this:

I wonder if we should reference the RTD guide on this to learn more? Or maybe backport our guide into the Sphinx docs for intersphinx?

I can add a link for now, although if the Sphinx developers agree, we should probably backport the guide into the Sphinx docs. Not sure what the best place would be though, I will open a separate issue.

doc/tutorial/index.rst

jakobandersen · 2021-07-09T11:06:10Z

doc/tutorial/index.rst

+   }
+
+You can now introduce a cross-reference to the ``contents`` section of the
+Sphinx documentation as follows, optionally prefixing it by ``sphinx:``, the


This prefixing is a bit of an icky issue. It doesn't work in general for all roles (and fundamentally can't), so in the end this should illustrate the feature with the role introduced by #9062.

I was not aware of that pull request. Do we want to introduce it even if it's not merged yet? Also, out of curiosity, can you expand a bit (or give an example) that doesn't work?

The tutorial should definitely be in sync with the implementation, so it's more that we should be aware of updating this part when relevant. Though, I guess this part will be merged after 4.1 (going out this weekend I believe), so there is a chance of the PR being ready in the same release as this.
As concrete examples, the roles in the C and C++ domains do not support this prefix. There are more details in #8418 (comment), but the fundamental issue is that when you call a role, e.g., :py:class: you ask the Python domain extension to do whatever it likes. This is most likely to insert a pending_xref node with meta-information, which that same extension is being asked to resolve at a later stage. When you suddenly add the prefix, you explicitly don't want that domain to handle it, but hope it understands and fails to resolve the xref, such that intersphinx can get a chance to resolve according to the prefix. Hence, fundamentally the syntax should be with a role that directly calls intersphinx directly, without other extensions involved (until a later stage, see also #8929).

As I commented above, one of the goal of intersphinx is transparency. So I prefer not to use prefix for cross references. As you explained, it is "optional" feature. So it would be better to explain the extension by non-prefixed example at first, I think.

Fair enough, it looked more clar to me that way, but I'm +1 on explaning the fully consistent and transparent way instead. Will rephrase this part.

Co-authored-by: Jakob Lykke Andersen <jakobandersen@users.noreply.github.com>

tk0miya · 2021-07-11T12:48:22Z

Are you going to add all subsections of tutorials to doc/tutorial/index.rst? Is it better to separate this page to multiple ones?

astrojuanlu · 2021-07-11T13:16:42Z

@tk0miya Sooner or later I wanted to split this into pages, yes. Do you think it's the moment already?

tk0miya · 2021-07-11T16:45:07Z

doc/tutorial/index.rst

+   ]
+
+The way to know whether you can reference an external project is to check
+if it has a publicly available ``objects.inv``, which can be parsed calling


The goal of the Intersphinx extension is referring to remote objects transparently like referring to local objects. We've usually found local objects via reading source documents, I believe. So I think the way to find the remote objects is also reading source documents, first. Is searching objects.inv really better way to do that? I don't think so...

This tutorial does not describe the relationship between the std:doc and :doc: role. Where they came from? I'm worried this explanation makes beginners confused.

So I think the way to find the remote objects is also reading source documents, first. Is searching objects.inv really better way to do that? I don't think so...

This was only to explain what the file is. Perhaps it is too technical? I find myself using this quite often lately, perhaps because I'm still learning the intricacies of intersphinx. @webknjaz has https://webknjaz.github.io/intersphinx-untangled/ and I think it's quite useful too.

In any case, I can clarify that this is optional and that it works in the same way as your own docs (except for custom roles, like Sphinx own :confval:.

This tutorial does not describe the relationship between the std:doc and :doc: role. Where they came from? I'm worried this explanation makes beginners confused.

It doesn't because I need to learn it myself...!

I had no luck on the Sphinx docs but I found this somewhere else:

Now I understand your point. I will think if I can clarify this earlier, otherwise rephrase this part.

I'd love to see this table in the official docs.

We've usually found local objects via reading source documents, I believe. So I think the way to find the remote objects is also reading source documents, first. Is searching objects.inv really better way to do that? I don't think so...

I've never had to do this by reading the source — it's hard to locate such things when you don't know where they are defined and how. Besides, the source is not always linked or available.
This is why I came up with intersphinx-untangled — it helps me discover things that are actually available. I sometimes even search by URL to see what roles link there.

I think there should be some sort of a reference explorer (I mentioned this somewhere on the tracker earlier).

It highlights a good point: that we should have better tooling around intersphinx inventories, to help users and hide the implementation details.

Yep: #8211

In this case, if I understand it correctly, the std considers each document to be an entity implicitly declared, and offers :doc: to refer them. Similarly, I think when you create a label somewhere, it is really an entity declared in the std domain, which you can refer to with :ref:. @tk0miya, is this last part correct?

Yes. Some kinds of objects (including labels and documents) belong to the std domain. It behaves like a system domain. So users do not need to give the domain name on referencing objects like :std:doc or :std:ref:.
https://github.com/sphinx-doc/sphinx/blob/4.x/sphinx/domains/std.py#L562-L570

This was very surprising to me, and I think it makes more difficult to introduce the topic: it's not as easy as "write a reference as you would write it in your docs", the user also has to understand some extra rules about how intersphinx targets are generated. Am I missing something @tk0miya @ jakobandersen ?

I've never thought referencing remote document via intersphinx. So I did not know such a behavior. Indeed, it would be a bug.

I've never thought referencing remote document via intersphinx. So I did not know such a behavior. Indeed, it would be a bug.

Thanks for chiming in @tk0miya ! I opened a discussion on https://groups.google.com/g/sphinx-dev/c/ZfslSpDGNUs/m/RjfzzFzUCgAJ about it, I think it's somewhat addressed by #9459

tk0miya · 2021-07-12T11:32:44Z

doc/tutorial/index.rst

+   }
+
+You can now introduce a cross-reference to the ``contents`` section of the
+Sphinx documentation as follows, optionally prefixing it by ``sphinx:``, the


As I commented above, one of the goal of intersphinx is transparency. So I prefer not to use prefix for cross references. As you explained, it is "optional" feature. So it would be better to explain the extension by non-prefixed example at first, I think.

tk0miya · 2021-07-12T11:36:07Z

@tk0miya Sooner or later I wanted to split this into pages, yes. Do you think it's the moment already?

Yes, I think so. I guess this is why you'd like to add a content directive at last PR. I think no reason to avoid adding a new page.

ChaiByte · 2021-07-22T05:06:31Z

Really useful. I think @FateScript will be interested in it.

astrojuanlu · 2021-07-22T09:03:34Z

For the record, we are holding off this part of the tutorial until there is more progress on the new intersphinx features (like #9459, #9062) and so that we can reach an agreement on whether users should be using objects.inv files or not.

astrojuanlu · 2021-07-22T09:21:04Z

(I didn't intend to close the PR, but it happened automatically because I renamed the branch, sorry 🤷🏽 it lives here now https://github.com/astrojuanlu/sphinx/tree/new-tutorial-part-intersphinx)

astrojuanlu added 2 commits July 8, 2021 22:06

Fix leftover terminology inconsistencies

0bea92d

Add section on intersphinx

06824e2

ericholscher approved these changes Jul 8, 2021

View reviewed changes

astrojuanlu and others added 3 commits July 8, 2021 22:41

Style improvements

9dade12

Co-authored-by: Eric Holscher <25510+ericholscher@users.noreply.github.com>

Make grep command more explicit

8511726

Reorder intersphinx steps to make them more natural

89aa4f9

jakobandersen reviewed Jul 9, 2021

View reviewed changes

Style improvement

87c3650

Co-authored-by: Jakob Lykke Andersen <jakobandersen@users.noreply.github.com>

tk0miya added type:docs type:proposal a feature suggestion labels Jul 11, 2021

tk0miya reviewed Jul 12, 2021

View reviewed changes

This was referenced Jul 13, 2021

Add intersphinx role #9062

Closed

Intersphinx, refactoring and intersphinx_disable_domains #9459

Merged

astrojuanlu changed the title ~~New Sphinx tutorial, part III~~ New Sphinx tutorial (intersphinx, posponed) Jul 22, 2021

astrojuanlu changed the title ~~New Sphinx tutorial (intersphinx, posponed)~~ New Sphinx tutorial (intersphinx, postponed) Jul 22, 2021

astrojuanlu closed this Jul 22, 2021

astrojuanlu deleted the new-tutorial-part-iii branch July 22, 2021 09:19

astrojuanlu mentioned this pull request Aug 9, 2021

New Sphinx tutorial, part III #9534

Merged

github-actions bot locked as resolved and limited conversation to collaborators Aug 22, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New Sphinx tutorial (intersphinx, postponed) #9424

New Sphinx tutorial (intersphinx, postponed) #9424

astrojuanlu commented Jul 8, 2021 •

edited

ericholscher left a comment

astrojuanlu commented Jul 8, 2021

jakobandersen Jul 9, 2021

astrojuanlu Jul 9, 2021

jakobandersen Jul 9, 2021

tk0miya Jul 12, 2021

astrojuanlu Jul 13, 2021

tk0miya commented Jul 11, 2021

astrojuanlu commented Jul 11, 2021

tk0miya Jul 11, 2021

astrojuanlu Jul 13, 2021

astrojuanlu Jul 13, 2021

astrojuanlu Jul 13, 2021

ericholscher Jul 13, 2021

webknjaz Jul 22, 2021

webknjaz Jul 22, 2021

tk0miya Jul 22, 2021

tk0miya Jul 22, 2021

astrojuanlu Jul 22, 2021

tk0miya Jul 12, 2021

tk0miya commented Jul 12, 2021

ChaiByte commented Jul 22, 2021

astrojuanlu commented Jul 22, 2021

astrojuanlu commented Jul 22, 2021

New Sphinx tutorial (intersphinx, postponed) #9424

New Sphinx tutorial (intersphinx, postponed) #9424

Conversation

astrojuanlu commented Jul 8, 2021 • edited

Feature or Bugfix

Purpose

Relates

ericholscher left a comment

Choose a reason for hiding this comment

astrojuanlu commented Jul 8, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tk0miya commented Jul 11, 2021

astrojuanlu commented Jul 11, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tk0miya commented Jul 12, 2021

ChaiByte commented Jul 22, 2021

astrojuanlu commented Jul 22, 2021

astrojuanlu commented Jul 22, 2021

astrojuanlu commented Jul 8, 2021 •

edited