Syntax highlighting support for various languages (tree-sitter grammars list)

[Tiggilyboo]

Having looked through the majority of the issues, I would like to collectively address the state of the tree sitter language support currently in Lapce.

Here follows current support:

• Agda
• Angular
• AutoIt
• Bash PR: Add bash tree-sitter support #282 add syntax highlighting support for Bash and Zig #690
• C PR: Add tree-sitter C support #274
• C++ PR: Add Cpp tree-sitter support #276
• C# PR: Add C# tree-sitter language #273, Add tree-sitter Dockerfile, C# & Nix, normalize language names #1104
• Carbon
• Crystal
• CSS PR: Add CSS tree-sitter language #315
• D PR: initial support for D syntax highlighting #1335
• Dart PR: Adding support for the dart language (help needed) #1153
• Dockerfile PR: Add tree-sitter Dockerfile, C# & Nix, normalize language names #1104
• Elixir PR: Add elixir language support #317
• Elm PR: [TreeSitter] Add elm language support #625
• Fish PR: feat: add Svelte & LaTeX #1161
• Fluent
• Fortran
• Glimmer PR: Add various tree-sitter languages #656
• Go bf79a35
• Haml
• Haskell PR: Add tree-sitter langs Haskell, CodeQL, and Swift #636
• Haxe PR: Add various tree-sitter languages #656
• HCL (Terraform) PR: Add various tree-sitter languages #656
• HTML PR: Add HTML tree-sitter support #428
• Java PRs: Add Java syntax #266 Add Java syntax #429
• Javascript bf2e495
• JSON PR: Add tree-sitter JSON support #318, Bump json tree-sitter support to 0.20 #280
• Julia PR: Add tree-sitter Julia support #762
• Kotlin PR: Kotlin tree-sitter support #1299
• LaTeX PR: feat: add Svelte & LaTeX #1161
• Lean3
• Lean4
• Lua PR: Lua support #1375
• Markdown PR: Add markdown tree-sitter language #320
• NIX PR: feat: add tree-sitter-nix #798, Add tree-sitter Dockerfile, C# & Nix, normalize language names #1104
• OCAML PR: Add various tree-sitter languages #656
• Pascal
• Php PR: Add PHP syntax highlighting #246
• Python bf2e495
• QL PR: Add tree-sitter langs Haskell, CodeQL, and Swift #636
• R PR: Add tree-sitter support for R #1370
• RenPy
• Rhai
• Ruby PR: [TreeSitter] Add ruby language support #430
• Rust
• Scala
• Scheme PR: Add Scheme tree-sitter support #1416
• SCSS PR: Add various tree-sitter languages #656
• SQL | PR: feat(tree-sitter): add sql language #1472
• Svelte PR: feat: add Svelte & LaTeX #1161
• Swift PR: Add tree-sitter langs Haskell, CodeQL, and Swift #636
• TOML
• Typescript bf2e495
• Vala
• Verilog
• Vue PR: add vue highlight support #1326
• WGSL PR: feat: add tree-sitter-wgsl #807
• XML | PR: feat(tree-sitter): Add XML support #1554
• Zig PR: add syntax highlighting support for Bash and Zig #690

I would like to get a few more of these up and running, I'll see what others remain which are more low hanging fruit (Already on version 0.20 of tree-sitter and already expose the highlight query).

One other thing I would like to address is whether or not we wish to put a feature flag on these? Not sure how large all of these + build time impact this would have for lapce. Something to think about...

Cheers, I am really enjoying this editor!

[bugadani]

Looking at how slowly a simple version change goes through in the upstream grammar repos, I wonder if we should temporarily use the forks instead. We aren't building to publish on crates.io so git dependencies should be just fine. We can track here which packages use the upstream versions and which ones use the forks. This would allow us to add support faster, as well as fix issues without being blocked on low traffic crates.

[VixieTSQ]

The current swift grammar linked is abandoned. Update it with this one: https://github.com/alex-pinkus/tree-sitter-swift
now waiting on this pr for upstream: alex-pinkus/tree-sitter-swift#194

[VixieTSQ]

I've just made prs to a lot of these. Here you are!

We're waiting on:
Verilog
Swift can be implemented right now! Is up to date and highlights exposed. You must use the crate on crates.io. Implemented!
SCSS
Scala
QL can be implemented right now! Is up to date and highlights exposed. Implemented!
OCaml
HCL
Haxe
Haskell can be implemented right now! Is up to date and highlights exposed. Implemented!
Glimmer
Fluent

[ghishadow]

@VixieTSQ for HCL updated repo is https://github.com/MichaHoffmann/tree-sitter-hcl

[VixieTSQ]

@ghishadow
list updated that one doesn't have it's bindings generated. I'll send a PR. But it also doesn't have a highlights file...

[ghishadow]

@ghishadow list updated that one doesn't have it's bindings generated. I'll send a PR. But it also doesn't have a highlights file...

you can copy from https://github.com/nvim-treesitter/nvim-treesitter/tree/master/queries/hcl or https://github.com/helix-editor/helix/tree/master/runtime/queries/hcl I think with attribution.

Looks like HTML syntax highlighting is also not working as HIGHLIGHTS_QUERY is commented in https://github.com/tree-sitter/tree-sitter-html/blob/master/bindings/rust/lib.rs#L38

[VixieTSQ]

#636 has been merged! go ahead and check those langs off, @Tiggilyboo

[bugadani]

I've done that :)

[SimY4]

Tree sitter Scala: https://github.com/tree-sitter/tree-sitter-scala

[urbit-pilled]

Tree sitter hoon: https://github.com/urbit-pilled/tree-sitter-hoon

[baldram]

Here is the Scala related PR #2280 not mentioned in the #272 (comment). It has been waiting for four months, so we'll need to resolve some minor conflicts and it's ready for another review probably then.

[panekj]

We're not going to add any additional grammar to lapce for now until external grammar store is implemented.

[baldram]

@panekj Thank you for the clarification.
Can you elaborate more what is "external grammar store" that blocks it? Which issue to track?

Is there a way to enable syntax highlighting locally, from within the editor, without adding it to lapce-core/Cargo.toml through a PR?

[panekj]

Can you elaborate more what is "external grammar store" that blocks it?

Place to get additional grammars from, something like what plugins currently are except we can't distribute grammars through plugin store

Which issue to track?

There isn't any

Is there a way to enable syntax highlighting locally, from within the editor, without adding it to lapce-core/Cargo.toml through a PR?

No

[aryalaadi]

AT&T asm?

[JLCareglio]

I'm currently working on implementing astro - can this be added to the list?

Idem 👀

[maxpetts]

I'm currently working on implementing astro - can this be added to the list?

Idem 👀

#1874

[FlixCoder]

Typst:
https://github.com/uben0/tree-sitter-typst

(or https://github.com/frozolotl/tree-sitter-typst )

[lnxyz]

tree sitter nim:
https://github.com/alaviss/tree-sitter-nim

[framlingham]

I would love to have AutoIt or AHK (v2) syntax highlighting if possible. I might have to have a look at tree-sitter.

[coolcoder613eb]

Assembler, NASM?

[volkertb]

Question: both Dockerfile and bash are checked, but in the nightly build as of 2023-12-31, neither Dockerfiles nor .sh files are getting any hightlighting. (The .sh file with which I just tried this has a #!/usr/bin/env bash shebang, so it's actually a Bash script, not just a generic POSIX shell script.)

Is there anything we need to do to get highlighting for such file types to work?

XML file highlighting also isn't working, at least not "out of the box".

C, C++ and yaml files do appear to have highlighting "out of the box".

[panekj]

Highlighting changed due to dynamic grammar loading and some languages might be broken for now

[volkertb]

Got it. Thanks for clarifying. 👍

[volkertb]

Maybe reset the checkmarks then, to avoid confusion?

[volkertb]

Highlighting for Markdown (.md) does currently work, at least in the most recent nightly build. ☑️

[volkertb]

Highlighting for DOS/Windows batch (.bat) files currently does not work, at least in the most recent nightly build. ❌

[Hellfar]

Highlighting for Ruby (.rb) files currently does not work, at least in the v0.3.1. ❌

[kylegoetz]

Highlighting changed due to dynamic grammar loading and some languages might be broken for now

Does this mean that we are able to externally load grammars now? I had written a TSG for Unison and was going to file a PR and that same week, it was announced that Lapce would not incorporate any more TSGs until an "external store" was available (unsure what that means, like a central Lapce-hosted repository of TSGs?), and so I bailed on Lapce as an IDE because it officially could not support my language. I stepped away from Lapce since fall of 2023 and decided to check back in.

[panekj]

Does this mean that we are able to externally load grammars now?

Yes, but adding new languages is not possible without modifying Lapce and building it