Parse markdown without trimming spaces

[fgarcia]

I want to parse a markdown file and detect quick outlines, something like this

some outline
  first
    deep
  second

Sadly the parser trims the leading spaces. So I get this (no empty space after \n)

{
  type: 'text',
  value: 'some outline\nfirst\ndeep\nsecond',
  position: [Object]
}

Somehow I can circumvent this adding a non significant empty space to the text before parsing

.split('\n')
.map(x => {
  return x.replace(/^\s(.*\w)/g, ' $1')
})
.join('\n')

And that gives me exactly what I wanted

{
  type: 'text',
  value: 'some outline\n  first\n    deep\n  second',
  position: [Object]
}

Basically I want a simple syntax extension where the leading white space has some meaning. Is there a way to prevent auto-trimming in the mdast? I come up with a solution but I do not want to depend in a hack that might break in the future

[wooorm]

detect quick outlines

What do you mean detect? What are “quick outlines”?

Sadly the parser trims the leading spaces. So I get this (no empty space after \n)

This is how markdown works. Leading and trailing spaces are dropped.

Somehow I can circumvent this adding a non significant empty space to the text before parsing

this is also how markdown works. Leading and trailing spaces are dropped, but other spaces are not, meaning between non-space character. A   (non-breaking space) is such a character.

Basically I want a simple syntax extension where the leading white space has some meaning. Is there a way to prevent auto-trimming in the mdast?

That isn’t markdown. It doesn’t work in other places where markdown works. There might be some hacky ways around it with out pluggable parser, but it won’t work in other places.

I come up with a solution but I do not want to depend in a hack that might break in the future

I think your solution is a great solution. It works everywhere that supports CommonMark/GFM. It won’t change in the future.

[fgarcia]

That isn’t markdown. It doesn’t work in other places where markdown works. There might be some hacky ways around it with out pluggable parser, but it won’t work in other places.

I mostly thought plug-ins were exactly for that: To add new features and/or meaning on top of valid Markdown text. Trimming the leading space in the AST and not in the HTML (during rehype stringify) takes away some of the flexibility of what could be done with markdown. It is even a bit aggressive if I want to modify markdown and write it back without destroying the user indentation. I expected markdown to tolerate and ignore the indentation, not forcing a sanitation or consider them invalid. Maybe I was to lenient on this 🙁

I think your solution is a great solution. It works everywhere that supports CommonMark/GFM. It won’t change in the future.

That is huge deal and solves my problem!

I really love remarkjs a lot and all the features I can implement with it. I would be sad if I had to use it as a "read-only" tool because the mandatory sanitations are too intrusive on top of the markdown refactoring/linting I wanted to do. Thanks a ton for the quick answer!

[ChristianMurphy]

I mostly thought plug-ins were exactly for that: To add new features and/or meaning on top of valid Markdown text.

Parser plugins can do this!
There is more info on parser extensions in micromark (the tokenizer remark builds on) https://github.com/micromark/micromark#extending-markdown

Trimming the leading space in the AST and not in the HTML (during rehype stringify) takes away some of the flexibility of what could be done with markdown.

To the contrary, this is the reason AST exist.
AST is short for Abstract Syntax Tree, the Abstract part focuses on simplifying working on structure, intentionally glossing over or normalizing stylistic parts of the language, like spaces which don't change how the document will be displayed (https://en.wikipedia.org/wiki/Abstract_syntax_tree)

What you are describing is a Concrete Syntax Tree (https://en.wikipedia.org/wiki/Parse_tree), which could be built on top of micromark (syntax-tree/mdast#36 (comment)), but would have a completely different structure and way of working from remark or rehype.