How to avoid a full-width space or   after a line break is deleted using remark-parse and remark-rehype

[k1350]

Hi, when I insert a full-width space or   after a line break (2 spaces) and converted markdown to html, the full-width space or   seems to be deleted.

Sample: https://github.com/k1350/remark_sample

import { unified } from 'unified'
import remarkParse from 'remark-parse'
import remarkRehype from 'remark-rehype'
import rehypeStringify from 'rehype-stringify'

const text = `
foo  
　baz

foo  
 baz

foo  
 baz

foo  
 baz
`;

async function main() {
    const html = await unified()
        .use(remarkParse)
        .use(remarkRehype)
        .use(rehypeStringify)
        .process(text);

    console.log(String(html));
}

main();

result

<p>foo<br>
baz</p>
<p>foo<br>
baz</p>
<p>foo<br>
baz</p>
<p>foo<br>
baz</p>

I have looked into CommonMark spec and tested this pattern at CommonMark official playground.
https://spec.commonmark.org/0.30/#hard-line-breaks
https://spec.commonmark.org/dingus/?text=foo%20%20%0A%E3%80%80baz%0A%0Afoo%20%20%0A%20baz%0A%0Afoo%20%20%0A%26emsp%3Bbaz%0A%0Afoo%20%20%0A%26nbsp%3Bbaz

I think a full-width space or   after a line break should not be deleted.

I am not familiar with remark, so I could not determine if my usage is bad or if it is a bug in remark.
If my usage is bad, I want to know how to avoid a full-width space or   after a line break is deleted.

[wooorm]

They are not deleted. They are still there.

$ node example.js 
<p>foo
　baz</p>
<p>foo
baz</p>
<p>foo
 baz</p>
<p>foo
 baz</p>

[k1350]

@wooorm Thank you for your reply. It seems there is no line break (2 spaces or \) after foo. I think <br> should appear in the result.
If you don't mind, could you try this example?

import { unified } from 'unified'
import remarkParse from 'remark-parse'
import remarkRehype from 'remark-rehype'
import rehypeStringify from 'rehype-stringify'

const text = `
foo\\
　baz

foo\\ 
 baz

foo\\
 baz

foo\\
 baz
`;

async function main() {
    const html = await unified()
        .use(remarkParse)
        .use(remarkRehype)
        .use(rehypeStringify)
        .process(text);

    console.log(String(html));
}

main();

My result is following.

<p>foo<br>
baz</p>
<p>foo<br>
baz</p>
<p>foo<br>
baz</p>
<p>foo<br>
baz</p>

CommonMark playground:
https://spec.commonmark.org/dingus/?text=foo%5C%0A%E3%80%80baz%0A%0Afoo%5C%0A%20baz%0A%0Afoo%5C%0A%26emsp%3Bbaz%0A%0Afoo%5C%0A%26nbsp%3Bbaz

[wooorm]

Ah! That seems to be a problem in how we turn markdown into HTML, and clean left over whitespace: https://github.com/syntax-tree/mdast-util-to-hast/blob/6d7683d8fcd8bb4a850fe1f9f44c11b37d899c5f/lib/state.js#L290-L311.
It’s stripping all whitespace around breaks, but should only strip spaces and tabs I think!

[wooorm]

Released in 13.0.1.
Note that I’m currently going through a wave of majors, so it’ll take a bit for the code to end up in remark-rehype!

[k1350]

@wooorm Thank you very much for taking the time to review the issue and fix it despite your busy schedule!
I will wait for updating dependencies of remark-rehype.