[feature] debugger REPL?

[vergenzt]

⭐ Suggestion

It'd be really nice to be able to have some sort of interactive REPL where the program pauses on each node traversed and drops into a simple REPL console that lets you print out attributes of the current node and figure out why a rule is or isn't matching.

💻 Use Cases

• When building a complex rule, it's a bit tough to debug right now. I'm currently resorting to commenting out more and more of the rule until I get something that matches, and inspecting the JSON output of that part that matches. Then repeat for each subpart of the rule.
• I'd love to be able to pass a --debug flag to sg scan or sg run that drops me into a primitive REPL to inspect the current parsing state.
• Some example commands could be:
  • [m]etavars - print metavars that have matched so far
  • [pn]/print-node - print current node in AST form
  • [pr]/print-rule - print current (sub)rule attempting to be matched
  • [s]ource - print source code for current node's range
  • [u]p - go up in the rule matching frame (to show node / source for parent rule(s))
  • [d]own - go down in the rule matching frame (unless at bottom)
  • [k] - go up in the AST frame (while staying in the same rule matching frame; AST frame resets upon rule frame movement)
  • [j] - go down in the AST frame (while staying in the same rule matching frame)
  • [n]ext - continue with match to next sibling node
  • [i]nto - continue with match into first child node (or next sibling node if no children)
  • [o]ut - continue with match, skipping remaining sibling nodes
  • [c]ontinue - continue with match until next breakpoint or end
  • [q]uit/exit - exit the command
• perhaps an additional field could be added to the rule schema called breakpoint: true to trigger spawning this REPL?

[HerringtonDarkholme]

thanks for the suggestion! debugger repl looks like a very interesting topic and I will investigate it later.

At the moment, ast-grep's top priority is to ship fix support in napi and python API.
Better debugging experience will be explored later.

[vergenzt]

Oooh! That's exciting! Is there an issue and/or branch I can follow with updates on fix support in the programmatic APIs? I've thought about switching to API usage for my use case but the lack of easy fix support (without having to manually transform ranges) has tipped the balance in favor of YAML for now. I'd love to help test out programmatic fix support!

[HerringtonDarkholme]

Hi @vergenzt, the branch has not been created yet and the API is under design. I will create an issue to track the progress of it.
Thanks for your support 👍

[vergenzt]

How would you feel about me attempting this debugger feature on my own in parallel? I'm new to Rust so it would take me a while, but I'm curious whether you're open to reviewing progress / a design proposal.

[vergenzt]

A few thoughts:

• I'm seeing it looks like sg scan and sg run both spawn multiple threads, one per file? So I'm guessing we need a way to force single-threaded one-at-a-time mode if breakpoints are enabled. (Perhaps via a --debug flag? And maybe if that flag is not passed then breakpoints are skipped with a warning to stderr.)

• I also see that rules don't run one-by-one but appear to be keyed by kind (when present) and run simultaneously. I'm still thinking through how to represent that in debug mode... maybe we just don't do this and instead run rules one by one when --debug is true? Would that be easy to accomplish? IMO performance needn't really be an issue in debug mode.

[HerringtonDarkholme]

Hi! Thanks for your input!

I'm more leaning toward to have a debugger on the web playground. https://ast-grep.github.io/playground.html

Reasoning:

1. building a debugger is pretty hard and requires ncurses or other TUI package
2. setting a breakpoint is hard (gdb/lldb is a little bit too arcane magic to me)

Pros for building debugger on playground

1. easier to build UI since it is in web
2. setting breakpoint is trivial (clicking on node )
3. it will not bloat shipped binary

[vergenzt]

I didn't think it would be a proper gdb-level breakpoint, just a "breakpoint" as in a place where we tell the code to stop and await user input (i.e. it could just output a prompt token, wait for an input command, respond to that input command, and repeat). But I hear ya that a web debugger makes more sense - I hadn't thought of that since I haven't been able to use the web debugger much yet as I'm using a custom language.

[HerringtonDarkholme]

How do you think about making the playground load WASM file?

[vergenzt]

How do you think about making the playground load WASM file?

That would be awesome! Great idea.