"Broken" gallery examples should still link to the code

[jni]

I'm not sure whether this is a bug or a feature request or me not understanding our build or configuration, but please close if it's invalid. 🙏

Over on the napari gallery, we recently noticed a broken example, which results in the "Broken" thumbnail:

You can see a recent Circle-CI build here.

My issue is that even though the hover works and turns into a link cursor, the thumbnail now links to nothing at all, which makes the gallery itself feel broken. Additionally, when exploring the issue it would be nice to actually quickly look at the code right there, instead of having to navigate to the right spot in the repo.

And, in this case, since the code runs normally when run as a Python script but not when run in the gallery (this is our fault, not sphinx-gallery's 😅), I had to go hunt down the traceback in the build logs.

My proposal is that sphinx-gallery could render a page containing the full script, together with the captured traceback from the build at the bottom.

Is this something that could be considered?

Thanks!

[larsoner]

If you look at https://sphinx-gallery.github.io/stable/auto_examples/index.html#no-image-output-examples it links to (by a clickable "BROKEN" image) this url:

https://sphinx-gallery.github.io/stable/auto_examples/no_output/plot_raise.html#sphx-glr-auto-examples-no-output-plot-raise-py

which has the traceback. There is even a second example with a SyntaxError

https://sphinx-gallery.github.io/stable/auto_examples/no_output/plot_syntaxerror.html#sphx-glr-auto-examples-no-output-plot-syntaxerror-py

So the machinery is there to produce the sorts of outputs you're looking for. So why does it work for us and not for you? I'm guessing it could be because these failures are listed in our expected_failing_examples:

sphinx-gallery/doc/conf.py

Lines 452 to 454 in f0523c2

	"expected_failing_examples": [
	"../examples/no_output/plot_raise.py",
	"../examples/no_output/plot_syntaxerror.py",

If that is indeed the case, then I don't see any reason why we shouldn't also render the unexpectedly failing examples as you suggest. PR welcome to try adding this. (If there is a good reason not to render the unexpectedly failing examples in some cases, we can add a new render_unexpected_failing_examples=False for example for backward compat that we can use to add the new behavior for Napari and others, but hopefully we won't need to and can just use effectively True always!)

[jni]

Thanks @larsoner! Very insightful and thorough response. I'll wait for others to chime in on "If there is a good reason not to render the unexpectedly failing examples in some cases", then I might try my hand at that PR.