drop dependency to pandoc ?

[12rambau]

I recently switch from nb-myst to nbspinx and I'm facing an environment issue every time I want to build my documentation in isolated env or new machines: the dependency to pandoc.

For example in pydata-sphinx-theme CI/CD I'm forced to rely on a custom github action to make sure the installation proceed on every available platform.

I don't know how complicated this might be but would it be possible to remove this dependency and rely on python only tools ?
Note that I will be happy to help but before I dive in I would like to discuss feasability.

[mgeier]

Yes, this was the plan all along (see #36), it just hasn't happened yet ...

Currently, Markdown cells are converted to reST (and transformed a bit with the Pandoc machinery) and integrated into a reST template which intermingles them with code cells.

I would like to change this to something like this:

We iterate over all cells while "directly" creating the docutils in-memory representation (without going the detour via reST). In case of code cells, this would be done within nbsphinx, and in case of Markdown cells, an external Python library should be used.

For this, we would need a Python library with a very specific API: It would need to take the text of a single Jupyter-flavored Markdown cell and a "partial" docutils document. It would then have to insert the content from the Markdown text, potentially extending already existing sections from previous Markdown cells.

This would also allow selecting between multiple Markdown parsers, see also #762.

Does that sound reasonable?
Would you have any other suggestions?

I will be happy to help

That's great!

The first thing would be to check if such a Python library already exists.
I am aware of multiple parsers for (different flavors of) Markdown that generate a docutils representation, but I don't know about the aforementioned "partial" functionality. And the library should be extensible enough to add some nbsphinx features like thumbnail galleries etc.