Parametrised markdown code blocks?

[ilg-ul]

I'm trying to understand how I can use variables in various markdown constructs, and I noticed that they work in some cases, like headers, lists, and explicit html code blocks:

export const world = 'World'

Hello {world}!

# Hello {world}!

- Hello {world}!

One <code>Hello {world}!</code> Two

However I could not make them work in markdown code blocks:

export const world = 'World'

Three `Hello {world}!` Four

    ```bash
    Five
    Hello {world}!
    Six
    ```

Am I missing something, or this is not yet implemented?

[wooorm]

It is indeed not implemented.
It works like other markdown things: you can use expressions where you can use other things, too. For example, you can use emphasis in certain places but not others. Expressions are the same.

[ilg-ul]

I see... Not very encouraging :-(

The question is not theoretical, I am considering to migrate several sites from Jekyll (Liquid templates) to Docusaurus (MDX), and most of the pages (hundreds of them) have code blocks like this:

    ```console
    $ ~/.local/xPacks/arm-none-eabi-gcc/xpack-arm-none-eabi-gcc-{{ page.version }}-{{ page.xpack-subversion }}/bin/arm-none-eabi-gcc --version
    arm-none-eabi-gcc (xPack GNU Arm Embedded GCC x86_64) {{ page.version }} {{ page.version-timestamp }}
    ```

I know I can rewrite this to html using <pre>...</pre>, but this also requires adding </br> at the end of each line, which makes things unrealistically tedious, especially for long blocks (like file system tree outputs, with tens of lines).

Any other suggestions how to approach this migration? The Liquid template engine performs only textual substitutions, and this works pretty well. Unfortunately, with MDX, such textual substitutions seem not possible. :-(

[wooorm]

You want to switch from one language to another.
That’s fine.
But there will be differences.

[remcohaszing]

Code may often contain syntax that would conflict with such interpolation. For example, consider the following MDX:

This is a destructuring example in JavaScript:

```js
const { name } = person
```

I think it would come as a total surprise if name gets interpolated.

[remcohaszing]

To help you get started, it probably looks something like this:

import { Code, Root } from 'mdast';
import { Plugin, Transformer } from 'unified';
import { visit } from 'unist-util-visit';

import { yourSubstitutionImplementation } from './some-module.js'

const remarkLiquid: Plugin<[], Root> = () => (ast) => {
  visit(ast, 'code', (node: Code) => {
    node.value = yourSubstitutionImplementation(node.value)
  })
}

export default remarkLiquid;

[ilg-ul]

I think this is a good starting point, thank you.

Is node.value a string with the entire .md file?

Invoking the Liquid template engine is trivial, but I also have to parse the frontmatter, to get the substitution variables and pass them to the Liquid engine.

[wooorm]

No, the above code looks for code nodes, whose value field is just the text inside them, not the entire file again.

If you have frontmatter, you need to set up remark to understand that frontmatter. With https://github.com/remarkjs/remark-frontmatter.

You also need to parse that frontmatter and expose it to plugins. vfile-matter can help.

• To learn more about the AST used in remark, called mdast, see: https://github.com/syntax-tree/mdast
• To learn about the file format used in remark, called vfile, see: https://github.com/vfile/vfile

---

MDX expressions are not liquid variables. You now have two solutions to a problem (variables?) that do similar but different things. I think you will run into problems when you try and mix the two in your files.

[ilg-ul]

the above code looks for code nodes, whose value field is just the text inside them, not the entire file again.

Ah, this might not be exactly what I expected. The Liquid substitution is expected to be performed on the entire original file, and the result later parsed for nodes.

Can this be done?

MDX expressions are not liquid variables. You now have two solutions to a problem (variables?) that do similar but different things. I think you will run into problems when you try and mix the two in your files.

I definitely don't have your oversight, it might be so, but, if the substitution is done on the entire file, perhaps not necessarily, since these are two separate steps.

With my limited understanding, I think that a pre-processing step with the Liquid engine, followed by the existing MDX logic, would be a very powerful combination, and would greatly simplify the migration from Jekyll.

Without this, most (if not all) of my pages require lots of small edits, generally rewrites in the more complicated JSX syntax of the initial simple MD.

BTW, Liquid allows much more than simple substitutions, it has lots of tags that can implement quite elaborate logic.

For example here is another excerpt from another page which I have no idea how to convert from Liquid/Jekyll to MDX/Docusaurus:

{% for post in site.posts %}{% if post.categories contains "releases" %}{% if post.categories contains "docus-mock" %}
* [{{ post.title }}]({{ site.baseurl }}{{ post.url }}) [(download)]({{ post.download_url }}){% endif %}{% endif %}{% endfor %}

These two lines generate a bulleted list of links to some blog pages, filtered by a category field.

If I can find a way to pre-process it, the result is plain markdown, it shouldn't be a problem.

[wooorm]

The Liquid substitution is expected to be performed on the entire original file, and the result later parsed for nodes.

Understandable! The original question was specifically about code, hence why the solution proposed was also about code.

Can this be done?

Yes, you can use liquid before passing things to MDX.
However, you are proposing to use two solutions for the same problem together.
In Docusaurus.
a) Docusaurus doesn’t support this (out of the box?)
b) You could also use one solution: either liquid, or MDX
c) Expressions and liquid variables conflict. For {a}, is that MDX or Liquid? If you use Liquid, you can’t use much of MDX

BTW, Liquid allows much more than simple substitutions, it has lots of tags that can implement quite elaborate logic.

The same is true for MDX.

Without this, most (if not all) of my pages require lots of small edits, generally rewrites in the more complicated JSX syntax of the initial simple MD.

This brings back my earlier comparison of using two programming languages together.

For example here is another excerpt from another page which I have no idea how to convert from Liquid/Jekyll to MDX/Docusaurus:

How to do this depends on your framework, particularly, React, and the language, as in JavaScript.
Something along these lines:

<ol>
  {
    site.posts
      .filter((post) => post.categories.includes('releases') && post.categories.includes('docus-mock'))
      .map((post) => {
        <li key={post.url}>
          <a href={site.baseurl + post.url}>{post.title}</a>{' '}
          <a href={post.download_url}>(download)</a>
        </li>
      }
    })
  }
</ol>

If I can find a way to pre-process it, the result is plain markdown, it shouldn't be a problem.

MDX is not plain markdown.
You can use plain markdown.
But you can also use JSX and expressions and import/exports.

[ilg-ul]

The original question was specifically about code, hence why the solution proposed was also about code.

Right, the original question was about code because most of my pages include lots of code and manually rewriting them from markdown to JSX would make the biggest part of the migration effort.

Also, since Docusaurus did not yet complete the migration to MDX 2, with MDX 1 I also have problems with lists, links, etc, practically the migration requires to fully rewrite every page, there is little to be reused. :-(

I'll probably postpone everything until Docusaurus 3 is out, with MDX 2 and full ESM support, and then I'll re-evaluate everything. If I can automate the migration with some reasonably complex sed scripts, I'll try to convert the pages to MDX. If not, I have to decide if a Liquid pre-processing step is realistic or not.

b) You could also use one solution: either liquid, or MDX

For a new site I'd definitely go for MDX, but unfortunately I have hundreds of pages with liquid substitutions, and I'm trying to find a solution to make the migration less painful.

To be exact, I have 20 smaller sites (each with 20-30 pages) and 2 large sites, each with 100+ pages.

c) Expressions and liquid variables conflict. For {a}, is that MDX or Liquid?

{a} is ignored by Liquid. The Liquid engine uses {{ expression }} for substitutions and {% tag %} for additional logic. So if the pre-processing step is done first, these constructs are removed from the text, and MDX can parse the resulting content with its usual syntax. If, for any reason, the page needs to pass something with double braces to MDX, Liquid has the {% raw %}...{% endraw %} tags, to escape anything.

If you use Liquid, you can’t use much of MDX

I don't understand this. Why MDX cannot parse the page after Liquid does the textual substitutions?

By the way, you have a (somehow) similar syntax problem, in JSX you use {x} for interpolation, but if you switch to JS (like you did in the examples above) you have to use ${ expression } inside template strings.

---

I am convinced that React/MDX/Docusaurus/etc represent a much more powerful solution, and I'd really like to do everything in the node/npm ecosystem, but unfortunately I do not have unlimited resources for this.

FYI, in my desire to stay in the same ecosystem with my embedded system projects too, I created xpm, and the xPack project.

Anyway, regardless of my desire to get rid of Ruby and Jekyll, the Liquid template engine remains a very nice thing, and the JS implementation is really great.

[wooorm]

Why MDX cannot parse the page after Liquid does the textual substitutions?

It can. I made a mistake: for some reason I thought liquid would look for one brace too ({x}).

[ilg-ul]

Ok, no problem. Let's hope for the best, that the Liquid pre-processing will not be needed.

[ilg-ul]

I mentioned earlier that I also have difficulties to migrate lists and/or links to MDX.

Any idea why the following:

export const version = '1.2.3'

- [![Github All Releases](https://img.shields.io/github/downloads/xpack-dev-tools/docus-mock-xpack/v{ version }/total.svg)](https://github.com/xpack-dev-tools/docus-mock-xpack/releases/v{ version }/)

is rendered (by the playground) as:

[![Github All Releases](https://img.shields.io/github/downloads/xpack-dev-tools/docus-mock-xpack/v1.2.3/total.svg)](https://github.com/xpack-dev-tools/docus-mock-xpack/releases/v1.2.3/)

and not as an image with a link, like GitHub does for the same content (the urls are fictive):

---> <---

I tried to enable the remark-gfm, but the result is worse, the second interpolation fails:

this release [![Github All Releases](https://img.shields.io/github/downloads/xpack-dev-tools/docus-mock-xpack/v1.2.3/total.svg)]([https://github.com/xpack-dev-tools/docus-mock-xpack/releases/v{](https://github.com/xpack-dev-tools/docus-mock-xpack/releases/v%7B) version }/)

Is this the expected behaviour?

[wooorm]

This is expected behavior. The text you show is not “valid” markdown: it is not an image in markdown.
Because there are spaces in the URL (around the word version). The CommonMark spec does not allow spaces in URLs in this position.

Presumably, you are trying to use interpolation (liquid or JavaScript) there?
In that case, that is not supported. My first comment answers why: https://github.com/orgs/mdx-js/discussions/2288#discussioncomment-5696483.

[ilg-ul]

The CommonMark spec does not allow spaces in URLs in this position.

Aha! This was really useful, thank you!

GitHub is less strict with these spaces. Is there a way (configuration option, plugin, etc) to switch to GitHub behaviour?

[wooorm]

How do you mean GH is less strict? GH follows CommonMark:

![a](https://img.shields.io/github/downloads/xpack-dev-tools/docus-mock-xpack/v{ version }/total.svg)


[ilg-ul]

Indeed. My bad, sorry. I'll remove all spaces.

[ilg-ul]

I think that now things are bit more clear. After Docusaurus will add support for MDX 2, hopefully most of my content will be accepted.

The only remaining major issue is with code blocks, which require the <pre><code className="language-console">{ ... }</code></pre> rewrite, (functional but not a joy, given that most of my pages have such blocks with shell commands or console outputs).

In Docusaurus I saw that it is possible to pass a lot of options to code blocks, like line numbers to highlight, etc.

I was wondering if it would be possible to add a new option to enable interpolation inside code blocs, something like:

    ```console enableInterpolation
    Five
    Hello {world}!
    Six
    ```

I think that this would solve the problem with a simple solution while preserving the markdown syntax, avoiding thus the rewrite to html, which is more tedious and also less readable.

[ilg-ul]

remco’s example from earlier ...

You mean a plugin that identifies that the node is a code block, checks if the enableInterpolation is present, and performs the substitutions?

I am not sure where you would want world to come from.

An export or a frontmatter definition. Does it make any difference?

I personally would recommend using JSX (<pre>...</pre>)

Yeah, just that there are hundreds of them... And I have to change not only the back ticks into <pre>..., I also have to add $ in front of braces, and this requires more than what some simple sed expressions can do.

[wooorm]

You mean a plugin

Yep

An export or a frontmatter definition. Does it make any difference?

It might not make a difference. The reason I ask is that I don’t know exactly what you want.

I also have to add $ in front of braces

That should not be needed? Why do you think that is needed?

[ilg-ul]

The reason I ask is that I don’t know exactly what you want.

Hmmm, if after more than 20 messages I was not able to describe the problem, my communication skills should be really bad...

I provided an excerpt from my existing markdown code in this comment, and you suggested a solution in the next comment.

From that I got the idea to use a JS string template inside JSX html code.

What I need is a solution to migrate my content to Docusaurus, with a not so complicated rewriting.

BTW, your code has two small issues: because the apostrof is on the first line, the string starts with an empty line; also the string arm-none-eabi-gcc-{{ page.version }} present somewhere in the middle of the block should be changed to arm-none-eabi-gcc-${ props.page.version }.

to add $ in front of braces
That should not be needed? Why do you think that is needed?

Because the JS syntax for inserting expressions in a string template is ${expression}.

[remcohaszing]

You seem to be coming from a text templating language. It converts text to other text based on interpolation. Any text can be interpolated anywhere in the document, because the content doesn’t have semantic meaning to the templating engine.

MDX is a programming language that gets compiled to JavaScript. It has certain rules of what is valid syntax, and where JavaScript expressions are allowed.

You can write dynamic links for instance in MDX, but you’ll have to use JSX syntax for that, not interpolation:

{/* This is a link to { props.link } */}
[Click me]({ props.link })

{/* This is a link to whatever link is passed as the link prop */}
<a href={props.link}>Click me</a>

Likewise, code blocks can’t be interpolated. However, you can use JSX and string expressions.

<pre><code>{`Hello ${props.name}`}</code></pre>

MDX supports plugins that can transform the code, similar to Babel.

You later told us the variables are defined in frontmatter. The frontmatter data can be accessed using the remark-frontmatter and remark-mdx-frontmatter plugins. So you could write something like:

<pre><code>{`Hello ${frontmatter.name}`}</code></pre>

Alternatively you can access the frontmatter data in your own remark plugin (this also depends on remark-frontmatter).

import { Code, Root } from 'mdast'
import { Plugin, Transformer } from 'unified'
import { visit } from 'unist-util-visit'
import { parse } from 'yaml'

import { yourSubstitutionImplementation } from './some-module.js'

const remarkLiquid: Plugin<[], Root> = () => (ast) => {
  const yaml = ast.children.find((child) => child.type === 'yaml')

  if (!yaml) {
    return
  }

  const frontmatter = parse(yaml.value)

  visit(ast, 'code', (node: Code) => {
    node.value = yourSubstitutionImplementation(node.value, frontmatter)
  })
}

export default remarkLiquid;

You can extend this to any node type you want, i.e. you could process links or inline code the same way.

[wooorm]

Hmmm, if after more than 20 messages I was not able to describe the problem, my communication skills should be really bad...

This is the XY problem: https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem/66378#66378.

The overarching question you asked was about Liquid in code blocks, and then there were several smaller questions.
Now you put your root question in words: “What I need is a solution to migrate my content to Docusaurus, with a not so complicated rewriting.”

What I need is a solution to migrate my content to Docusaurus, with a not so complicated rewriting.

This might be better asked to the Docusaurus project.
But here comes my earlier point back, also mentioned by Remco above: Liquid and MDX are very different. Just as different as comparing JavaScript to HTML. “[C]omplicated rewriting” is subjective, but yeah, if you want to do this, you need to rewrite things.

[ilg-ul]

@remcohaszing, I think that now I understand your proposal. It does not pre-process the entire file, as usual for Liquid, but only the code block content. For my use case this is probably enough and would allow to use most of the existing markdown without rewrites.

Thank you very much! I selected your message as the answer.

@wooorm, thank you for your patience.