Unknown directive type: 'exercise' (Jupyterbook + mystmd confusion)

[scottyhq]

Hello! Despite nice documentation on compatibility and history between mystmd and jupyterbook (https://mystmd.org/guide/background) I am confused and must be missing something about how to get jupyterbook to recognize directives documented in this repository.

My documentation journey is as follows:

1. Started with reading about the exercise directive which seems to be a standard part of MyST markdown https://mystmd.org/guide/exercises so I'd like to try it out in an existing jupyterbook

2. This page https://jupyterbook.org/en/stable/content/myst.html suggests that Jupyterbook will happily interpret these directives:

Whether you write your book’s content in Jupyter notebooks (.ipynb) or in regular Markdown files (.md), you’ll write in the same flavour of MyST Markdown. Jupyter Book will know how to parse both of them.

3. But trying to add the example exercise directive to a .md file results in WARNING: Unknown directive type: 'exercise' [myst.directive_unknown] and indeed the html output is not rendered.

It's straightforward to reproduce this by modifying the template book to include the example exercise directive

jupyter-book create mynewbook/ 
cd mynewbook   
# Add the first example exercise https://mystmd.org/guide/exercises to markdown.md 
jb build .

jb --version

Jupyter Book      : 1.0.0
External ToC      : 1.0.1
MyST-Parser       : 2.0.0
MyST-NB           : 1.0.0
Sphinx Book Theme : 1.1.2
Jupyter-Cache     : 1.0.0
NbClient          : 0.8.0

[rowanc1]

The exercise library is not bundled with Jupyter book and can be found here as well as some getting started notes:
https://ebp-sphinx-exercise.readthedocs.io/en/latest/#getting-started

You will have to install it and add it to your extensions! Hope that helps!

[rowanc1]

Correct! The mystmd package does not use sphinx, and as such can't use that extension ecosystem directly. We are in the process of porting some of the highly used extensions to mystmd (exercise, proof, etc.). The docs for that project are on https://mystmd.org. You can read a bit more about the background of the mystmd project and how it relates to JupyterBook here, which might help:
https://mystmd.org/guide/background#history-of-myst-markdown-and-mystmd

@mmcky might have a bit more thoughts on the sphinx compatibility, he pinged me after seeing this issue!

For the things that are extensions in JupyterBook - as that is currently built on sphinx, there are many sphinx extensions that could be used. The jupyterbook.org docs are hopefully helpful at outlining what is and isn't included.

Let me know if that is helpful - if you need a demo of the differences and help deciding which project to choose (the content is compatible) I would be up for a quick chat to help out, would love to learn more about what you are trying to do with xarray/pangeo!
https://calendly.com/rowan-curvenote/30min?back=1&month=2024-02&date=2024-02-22

[mmcky]

With regards to exercise admonitions -- the myst syntax itself is fairly consistent between sphinx-exercise (which mystmd took some inspiration from) and mystmd. So there shouldn't be any changes in the MyST based source files -- but you will need to configure jupyter-book to understand the admonitions by adding the sphinx-exercise package.

https://ebp-sphinx-exercise.readthedocs.io/en/latest/

Installation: https://ebp-sphinx-exercise.readthedocs.io/en/latest/#getting-started

Hope that helps.

[mmcky]

thanks re: executablebooks/sphinx-exercise#64 (comment) will need to take a closer look and see if we can update the package.

[scottyhq]

Thank you both for the helpful notes and all the work on these tools as they are a fantastic resource for education and scientific publishing!

I'm summarizing my mental model below, apologies if it seems redundant and correct me if I'm wrong, but I'm trying to clarify the difference in content jupyterbook and mystmd can parse rather than the differences in the underlying build systems (e.g. sphinx):

• mystmd is a set of command line tools that can parse documents adhering to the myst spec (https://mystmd.org/spec/) plus additional directives and roles defined in the mystmd authoring guide (e.g. https://mystmd.org/guide/exercises).

• jupyterbook is an entirely distinct set of command line tools that uses a different parser https://github.com/executablebooks/MyST-Parser to interpret docs following the myst spec plus additional directives and roles that must be defined via extensions https://myst-parser.readthedocs.io/en/latest/syntax/optional.html#

[rowanc1]

Yes that is all correct. Our goal is to make the base jupyterbook and mystmd experience the same experience, which opens up opportunities like jupyterlab-myst to be editing the content directly from a JupyterHub (or any other web-based app like vscode, overleaf, etc.). mystmd also opens up other possibilities, especially in scientific writing and publishing, and is being adopted by publishers like AGU (https://agu.org) and scipy proceedings.

The current major outstanding differences in content (and ui) between mystmd and jupyterbook are listed here: #189