How to prevent some characters from getting encoded?

[babakfp]

Hi

Code

import { unified } from "unified"
import remarkParse from "remark-parse"
import remarkRehype from "remark-rehype"
import rehypeStringify from "rehype-stringify"

export const transformer = async (markdown: string) => {
    const processor = unified()

    processor.use(remarkParse)

    processor.use(remarkRehype, {
        allowDangerousHtml: true,
    })

    processor.use(rehypeStringify, {
        allowDangerousCharacters: true,
        allowDangerousHtml: true,
    })

    processor.use(() => {
        return (node) => {
            console.log(JSON.stringify(node, null, 4)) // No characters are not encoded
        }
    })

    const result = await processor.process(markdown)

    console.log(result.value) // Some characters are encoded (and I don't know why!)

    return result.value
}

Input

`bg-radial-<color>`

```js
{
    /* <plugin-name> */
}
```

Output

<p><code>bg-radial-&#x3C;color></code></p>
<pre><code class="language-js">{
    /* &#x3C;plugin-name> */
}
</code></pre>

[ChristianMurphy]

Welcome @babakfp! 👋
Sorry you ran into some confusion.

Rehype needs to encode the < in <plugin-name> because you used it in a code block, so it is plaintext.
If it were not encoded, it would turn into a custom element (and an invalid one as it has no closing tag).
That behavior of turning known plaintext into HTML would be highly unexpected and insecure.

[babakfp]

I tried a different approach, but this time the issue was that "rehype-stringify" was adding its serialization on top of mine, which broke things.
I would like to have more control over things. There is this library that does what I'm looking for, however, it's not been updated for 5 years.

[ChristianMurphy]

@babakfp that doesn't answer the main question, what is the problem you are trying to solve?

Rehype's goal is to produce valid HTML.
The example of your desired result is invalid and therefore a non goal for the project.

That said we still may be able to help you solve your issue.
Are you trying to compile for a non-HTML language? Or something else entirely?

[babakfp]

I'm attempting to serialize certain characters in Svelte syntax to prevent Svelte from interpreting the code within the block. However, the methods I've tried for serialization may be incorrect:

One approach involves defining an array of character replacements:

const entries = [
    // [/</g, "&lt;"],
    // [/>/g, "&gt;"],
    [/{/g, "&#123;"],
    [/}/g, "&#125;"],
]

Another method involves a function to sanitize strings:

export const sanitizeString = (string: string) => {
    let result = ""
    for (const char of string) {
        // Convert each character to its Unicode representation
        const unicodeChar = char.charCodeAt(0).toString(16).toUpperCase()
        // Concatenate the Unicode representation to the result
        result += "&#x" + unicodeChar + ";"
        console.log(`"${result}"`)
    }
    return result
}

However, neither of these approaches seems to work, as the serialized string undergoes unexpected changes when processed by the rehype stringify plugin. For instance, { (representing {) transforms into {.

[ChristianMurphy]

Svelte syntax to prevent Svelte from interpreting the code within the block

Two thoughts:

1. You may want to ask Svelte about this? It feels like that is an issue if it is intrepreting plain text when it should not be.
2. If you want tooling specifically designed for Svelte, you may want to checkout https://mdsvex.pngwn.io/

[ChristianMurphy]

Another option worth noting, you could try https://github.com/rehypejs/rehype-react as a renderer.
It also supports JSX compatible runtimes like Svelte.

[babakfp]

I think I solved it.

constants.ts:

export const STRINGIFY_ENTITIES_DEFAULT_DANGEROUS_CHARACTERS = [
    '"',
    "&",
    "'",
    "<",
    ">",
    "`",
] as const

export const STRINGIFY_ENTITIES_DEFAULT_SVELTE_DANGEROUS_CHARACTERS = [
    "{",
    "}",
] as const

rehype-sanitize-md-code.ts:

import { stringifyEntities } from "stringify-entities"
import { visit } from "unist-util-visit"
import type { Plugin } from "unified"

import {
    STRINGIFY_ENTITIES_DEFAULT_DANGEROUS_CHARACTERS,
    STRINGIFY_ENTITIES_DEFAULT_SVELTE_DANGEROUS_CHARACTERS,
} from "./constants.js"

export const rehypeSanitizeMdCode: Plugin = () => {
    return (tree) => {
        visit(tree, "element", (node) => {
            // @ts-ignore
            if (node.tagName === "code") {
                visit(node, "text", (childNode) => {
                    // @ts-ignore
                    childNode.type = "raw"

                    // @ts-ignore
                    childNode.value = stringifyEntities(
                        // @ts-ignore
                        childNode.value,
                        {
                            subset: [
                                ...STRINGIFY_ENTITIES_DEFAULT_DANGEROUS_CHARACTERS,
                                ...STRINGIFY_ENTITIES_DEFAULT_SVELTE_DANGEROUS_CHARACTERS,
                            ],
                        }
                    )
                })
            }
        })
    }
}

---

Thanks for helping 🌸.

[ChristianMurphy]

I'd be very cautious mucking with escapes.
There is a fair amount of risk of opening an app up to unescaped insecure content and causing unexpected errors from characters going through.

I'd encourage you @babakfp and future readers to consider the alternatives noted in https://github.com/orgs/rehypejs/discussions/167#discussioncomment-8815787 and https://github.com/orgs/rehypejs/discussions/167#discussioncomment-8823022