Add lexer for Typst

[miliog]

Is there an existing issue for this?

• I have searched the existing issues

What is the missing lexer?

Typst is a new markup language and I've noticed that chroma doesn't support it.

The syntax documentation can be found here:
https://typst.app/docs/reference/syntax/

Links to existing syntax definitions

Github Linguist uses the following:
https://github.com/michidk/typst-grammar/blob/master/grammars/tmlanguage.json

[miliog]

I converted the new pygments lexer (credits go to @jvoisin) to the Chroma XML format using python3 _tools/pygments2chroma_xml.py pygments.lexers.typst.TypstLexer > lexers/embedded/typst.xml. See miliog@d05a092.

Only the line

<rule pattern="(_|^|+|-|/|*|-&gt;|&lt;-|!=|==)\b"><token type="Operator"/></rule>

was broken and an error was shown while running the test. To fix it, I escaped the special characters +, -, /, and * by preceding them with a backslash (\):

<rule pattern="(_|\^|\+|\-|\/|\*|&gt;|&lt;|!=|==)\b"><token type="Operator"/></rule>

However, there are still a few problems remaining, as shown below:

My example works just fine in pygments according to https://pygments.org/demo/:

[jonaspleyer]

FYI I have looked at this problem and put some incremental work into the typst lexer at https://github.com/pygments/pygments It will probably be the easiest if we first develop there and then simply copy the lexer over just as you did before. Happy for feedback on my recent PR over there:
pygments/pygments#2656