Sphinx domain for Rust

[munircontractor]

Hi everyone,

I just published an alpha version of a Sphinx domain for Rust, along with a supporting Rust executable to generate the directives from the code. The project is https://pypi.org/project/sphinxcontrib-rust/. It is still in early stages, but has most basic functionality.

https://sphinxcontrib-rust.readthedocs.io/en/stable/index.html has the sample output from running the project on itself.

This announcement has two purposes:

1. Get help with some of the Sphinx internals. Most of this was done by digging through the Python domain's source code, so if someone wants to help me with things like formatting the text of the signatures, adding links for the types in the function signatures, etc. I'd be much obliged. As you can see from the linked docs, the formatting of the Rust directives needs work still.

2. Get some Rust users to take a look at the extension, give it a spin and provide some feedback on things like the organization of the items and the TOC.

Please let me know if anyone is interested in this. Thanks!

[chrisjsewell]

ahahaha, I wish you'd posted this about 8 hours ago; that's when I started writing a rust domain 😂

So firstly, I'm obviously not as far along as you in hooking everything up, but I did basically use the same parsing approach as you; with syn, except I link my python code to my rust code with py03, and allow for (de)serialisation of the analysed package with serde:

So there are improvements there, and I might be interested to collaborate (I wish you wouldn't be on gitlab 😄)

BUT the big issue that I don't see is solved by you or me (with syn) is meta-programming, i.e. macros:
this will not document "properly" meta-generated API objects.

rustdoc does do this, since it uses an intermediate representation generated by the compiler, i.e. after macros have been "expanded": https://rustc-dev-guide.rust-lang.org/rustdoc.html

It does this exactly here: https://github.com/rust-lang/rust/blob/25087e02b34775520856b6cc15e50f5ed0c35f50/src/librustdoc/lib.rs#L773

You can view a debug of this intermediate representation like so:

rustup run nightly cargo rustc -- -Z unpretty=hir-tree

but, to my knowledge, there is no way to hook into this from a third-party crate, without literally building the compiler from source 😒

So, that's the main question I have for you; do you see this as an issue, and can it be solved?

[munircontractor]

I can clone to GitHub if it helps, and you can fork from there. I have CI setup in GitLab so it won't be easy for me to move it here. The git interface there isn't that different IMO, but I understand preferences.

TBH I didn't know about py03 and didn't really consider FFI for this since it seemed much simpler to just output the docs from Rust and have Sphinx read them. It is similar to what autosummary does. I'll look into it. It's likely easier to share the configuration with it, than using a separate executable. But I still want to keep the file generation and directive processing separate for now. It makes developing and testing much simpler.

I wasn't planning on replicating all of rustdoc with this. I mostly want this to be good enough to be usable in projects that have Rust and other code so there's a way to generate combined docs and cross-reference between the languages. I certainly don't want to fork rustc for this. So, I think I'm ok without meta API being handled.

[chrisjsewell]

thanks for the reponse

I can clone to GitHub if it helps, and you can fork from there.

Perhaps you would be interested in allowing it to be moved to https://github.com/sphinx-extensions2, so we could colab there?
Naturally, I would grant you full access privileges, and also any others that wanted to colab

I have already working there on https://github.com/sphinx-extensions2/sphinx-autodoc2, with the similar goal of good auto-analysis, and allowing for both rst/myst docstrings, so it would also be interesting if there is any synergy there

It's likely easier to share the configuration with it, than using a separate executable

yeh and just easier for users; it would be ideal if users are just able to run the full process via sphinx, rather than an annoying two-step approach.
Again this is a goal of https://github.com/sphinx-extensions2/sphinx-autodoc2; to separate the analysis and build phases, but also allow sphinx to handle both of them

I certainly don't want to fork rustc for this. So, I think I'm ok without meta API being handled.

Oh yeh exactly, It's just an annoying limitation

[munircontractor]

yeh and just easier for users; it would be ideal if users are just able to run the full process via sphinx, rather than an annoying two-step approach.

It's still a one step approach for the user in my case. It starts a subprocess for the doc generation once the Sphinx config is available. Users just install the Sphinx extension and edit Sphinx config as necessary. I'll look more into pyO3 today and see if I can use that to read the config from Rust instead of passing it over the CLI.

I have already working there on https://github.com/sphinx-extensions2/sphinx-autodoc2, with the similar goal of good auto-analysis, and allowing for both rst/myst docstrings, so it would also be interesting if there is any synergy there

How are you handling the mixed docstrings? I wasn't considering on supporting that with Rust. Either all docstrings are rst or all are md.

I'd love to collab as well. Let me see what it'll take for me to move over to Github. We can keep discussing things here or in a more private channel till then.