Ease running on just one file

[AlexSzatmary]

It isn't easy to figure out how to efficiently rebuild a single example. This use case is important because people developing gallery entries are focused on a single file and have to preview it many times. Also, writing a gallery entry can be a good way for beginners to enter a project but beginners are likely to struggle—but making this experience better will help projects gain contributors. A tweak to the docs would have made things a lot easier for me, and some adjustment to how Sphinx Gallery gets configured might make a better process automatic.

For context, I'm working on the legend guide for Matplotlib, which is a Sphinx Gallery file. I found it difficult to generate the html for that file and only that file.

The Sphinx Gallery docs helpfully point out that you can build a single example

sphinx-build -D sphinx_gallery_conf.filename_pattern=plot_specific_example\.py ...

but I found setting include_patterns to also be essential for that goal.

I added the following to conf.py:

include_patterns = ["index.rst",
"users/explain/axes/legend_guide.rst",
]
suppress_warnings = ["toc.not_readable", "ref.doc", "ref.obj", "ref.func", "ref.meth",
"ref.class"]
gallery_dirs = ["users/explain/axes"]

and run with python -msphinx -b html -D sphinx_gallery_conf.filename_pattern=../galleries/users_explain/axes/legend_guide.py . build/html

Each of those tweaks to conf.py was important.

• include_patterns was needed to stop Sphinx from spending a lot of time on unrelated files. Without having the rst file being included, the html never gets built.
• suppress_warnings is nice to add because, while it's understandable that building only one file would trigger a lot of warnings, it's helpful to turn them off so that the relevant ones stand out.
• gallery_dirs drops the build time for a single file by a few seconds, which is nice.

1. I'd like the docs to give these hints on how to build a single file and I would be willing to make that change.
2. I think it may be possible for the build process to be modified. Sphinx Gallery could detect if only a few files are being updated, and tweak the variables noted in response.
3. Alternatively, projects such as Matplotlib could add another build target in their Makefiles, which would trigger the relevant logic within their own conf.py . Sphinx Gallery could make it easier for projects to adopt this workflow by offering a template.

[larsoner]

+1 for doc updates talking about how to configure sphinx to only build a few files (and ignore warnings). In some sense this is outside the scope of sphinx-gallery as it's a more general Sphinx concept but it'll probably help people.

I think it may be possible for the build process to be modified. Sphinx Gallery could detect if only a few files are being updated, and tweak the variables noted in response.

FWIW typically for "update a single example" I think the workflow people typically use is:

1. Do a build with -Dsphinx_gallery_conf.filename_pattern=.... This will take 5 minutes or whatever it takes to build the project doc while building a single example.
2. Modify the example
3. Rebuild with the same command as in (1), and it should be very fast (on the order of seconds not minutes) because of how Sphinx and sphinx-gallery treat file modifications

Then you don't have to worry about ignores etc. because just about everything else should work. So -1 on SG trying to be smarter about detecting changes -- I think we already do the standard thing and only rebuild files that have been modified. If you don't modify anything then running Sphinx twice (for a correctly configured project) the second run should be a no-op (end in just a few seconds).

[AlexSzatmary]

I think I'm following the workflow you described but my doc rebuilds normally take pretty long (I just timed 10 min for a from-scratch build for all project docs, and 3 min for a rebuild). I'm not doing anything fancy, just "make html". Not that I profiled it but most of the build time seems to not be spent on the galleries. Builds take a few minutes even when I haven't modified a file. Maybe there's a problem with how Matplotlib has doc builds configured with sphinx but it seems that others more experienced than me have long doc rebuild times on Matplotlib. If the Matplotlib docs should only take a few seconds to rebuild, that's out of scope for sphinx-gallery.

I think that the following point is in scope for sphinx-gallery. I have now observed that sphinx_gallery_conf.filename_pattern=plot_specific_example\.py ... does not affect build time because sphinx-gallery is good at only acting on changed files. The conf.py changes that I mentioned do make a big difference (like, 4 s build time). The existing description of sphinx_gallery_conf.filename_pattern that I linked to had led me down the wrong path on how to quickly build a single sphinx-gallery file (as in, it showed me how to cut the sphinx-gallery build time, but didn't make much difference for the total build time). As someone new to sphinx and sphinx-gallery, it took me a while to figure out that while I was editing legend_guide.py in one path, I needed to tell sphinx to build not that file but a .rst file in a totally different path.

Because the part of the sphinx-gallery docs that I cited addresses how to use -D to pass an option for a single build, not how to build a single file, I think it's best to use a different example for -D and to add a section on how to build a single gallery.

Thanks for helping me figure this out!

[larsoner]

Builds take a few minutes even when I haven't modified a file. Maybe there's a problem with how Matplotlib has doc builds configured with sphinx but it seems that others more experienced than me have long doc rebuild times on Matplotlib. If the Matplotlib docs should only take a few seconds to rebuild, that's out of scope for sphinx-gallery.

Yeah in theory running sphinx-build twice should be very quick assuming everything is configured correctly -- like the second build shouldn't take more than a handful of seconds (not minutes). But IIRC Sphinx has done some tinkering lately with how things are detected as out of date so maybe something broke there.

Because the part of the sphinx-gallery docs that I cited addresses how to use -D to pass an option for a single build, not how to build a single file, I think it's best to use a different example for -D and to add a section on how to build a single gallery.

I'm not sure I totally follow, but it might be easier to give this doc change PR a shot to explain what you mean anyway!