Adds graph debug documentation to book #379

afreeland · 2024-02-19T17:04:04Z

I ran across this comment while having some issues with regex support. Thought this was beneficial information and wanted to try and include it in the book so it was a little more visible to people that may be trying to debug and understand how things are working

jeertmans

Hey! Thank you some much for your contribution! I left a few review comments that I'd like you to address before we can merge :-)

Feel free to accept or reject my suggestions!

jeertmans · 2024-02-19T19:46:27Z

book/src/debug-graph.md

+# Debug Graph
+
+It may be beneficial during debugging to be able to visualize how Logos has generated its graph. 


Maybe add some introduction to the fact that Logos construct a graph internally to generate its code. Here, this comes a bit out of nowhere :-)

jeertmans · 2024-02-19T19:47:49Z

book/src/SUMMARY.md

@@ -10,6 +10,7 @@
 + [Using `Extras`](./extras.md)
 + [Using callbacks](./callbacks.md)
 + [Common regular expressions](./common-regex.md)
+ [Debug Graph](./debug-graph.md)


I'd maybe add just a Debugging section, and, in the debug.md file, have one section talking about the graph. So we can later add some other sections, e.g., to talk about regex-syntax's HIR or else.

jeertmans · 2024-02-19T19:49:50Z

book/src/debug-graph.md

+}
+```
+
+With our graph debugging enabled, we see the following output:


Suggested change

With our graph debugging enabled, we see the following output:

Logos actually constructs a graph that contains the logic for matching tokens:

jeertmans · 2024-02-19T20:00:13Z

book/src/debug-graph.md

+}
+```
+
+Lets look at `@9` for the character `.` we see that it will jump `=>` to `2`.  We can then follow that by looking at `@2` which resolves to our `::Period` token. It will then continue to look for any matches in the case there is potential continuation after the `.` character. In the _input_ we provided there is not, since it is the end of our input.


Suggested change

Lets look at `@9` for the character `.` we see that it will jump `=>` to `2`. We can then follow that by looking at `@2` which resolves to our `::Period` token. It will then continue to look for any matches in the case there is potential continuation after the `.` character. In the _input_ we provided there is not, since it is the end of our input.

This graph can help us understand how our patterns are matched, and maybe understand why we have a bug at some point.

Let's look at the last node, `9`, for the character `.`. We see that it will jump (`=>`) to `2`. We can then follow that by looking at `2` which resolves to our `::Period` token. It will then continue to look for any matches in the case there is potential continuation after the `.` character. In the _input_ we provided, there is not, since it is the end of our input.

Note that I have to read a bit more of the code to feel confident about how Graph actually works.

Also, this may be a bit counter-intuitive to take the example of ., where this is the last bit of our input string.

The thing I was to know (I guess the information is somewhere in the source code) is whether we always enter the graph via its last entry, or not.

Totally agree about it being counter-intuitive about the . example. I used the . since it seemed a little more straight forward on the graph to understand. Could probably smooth over that some if I reworked some of the wording to lead into a bit more, instead of BLAM, understand this lol

jeertmans · 2024-02-19T20:02:18Z

book/src/debug-graph.md

+
+Lets look at `@9` for the character `.` we see that it will jump `=>` to `2`.  We can then follow that by looking at `@2` which resolves to our `::Period` token. It will then continue to look for any matches in the case there is potential continuation after the `.` character. In the _input_ we provided there is not, since it is the end of our input.
+
+We also can try to identify how `fast` works by looking at `@9` that `f` jumps to `7`. This will then resolve the last letters of our word _fast_ by matching `ast` which jumps to `8`. Since our provided _input_ to the lexer does not include alphanumeric character after the word "fast" but rather whitespace, the token `::Fast` will be recognized. Then the graph will look for further potential continuation (ie `[g-z] => 4`)


Suggested change

We also can try to identify how `fast` works by looking at `@9` that `f` jumps to `7`. This will then resolve the last letters of our word _fast_ by matching `ast` which jumps to `8`. Since our provided _input_ to the lexer does not include alphanumeric character after the word "fast" but rather whitespace, the token `::Fast` will be recognized. Then the graph will look for further potential continuation (ie `[g-z] => 4`)

We also can try to identify how `fast` works by looking at `9`, first, and seeing that `f` makes it jump to `7`. This will then resolve the last letters of our word _fast_ by matching `ast` which jumps to `8`. Since our provided _input_ to the lexer does not include alphanumeric character after the word "fast" but rather whitespace, the token `::Fast` will be recognized. Then the graph will look for further potential continuation (i.e., `[g-z] => 4`)

book/src/debug-graph.md

afreeland · 2024-02-20T02:35:01Z

Yeah, sorry for the kind of rushed PR...trying to sneak this in-between family time lol

Made some improvements to language and file structure based off of your recommendations. Hopefully, a little less jarring to someone reading it.

As for the enable debugging part, I totally agree, I am new to rust but I can try to put something together that properly uses a debug-mode kind of flag. Will keep it as a separate commit or could be a follow up PR if I get a chance. Would make life much nicer if something was an easy feature bit to flip.

This adds a debug feature, similar to what Clap does, as requested by #379. The current implementation only debugs a few parts of the code, and it may be interesting to add more debug messages in the different crates.

jeertmans · 2024-02-20T09:40:45Z

Yeah, sorry for the kind of rushed PR...trying to sneak this in-between family time lol

Made some improvements to language and file structure based off of your recommendations. Hopefully, a little less jarring to someone reading it.

As for the enable debugging part, I totally agree, I am new to rust but I can try to put something together that properly uses a debug-mode kind of flag. Will keep it as a separate commit or could be a follow up PR if I get a chance. Would make life much nicer if something was an easy feature bit to flip.

No issue, I did it myself since this was an easy fix. The issue you had was that you needed to declare a debug feature in sub-crates. See #382 for more details :-)

* chore(lib): add debug feature This adds a debug feature, similar to what Clap does, as requested by #379. The current implementation only debugs a few parts of the code, and it may be interesting to add more debug messages in the different crates. * chore(lint): run cargo fmt

jeertmans

Nice work @afreeland! I left some more comments (sorry to be picky about some ^^'), but I hope you don't mind..

Note that #382 was merged and just added a new debug feature.

book/src/debugging.md

jeertmans · 2024-02-20T09:56:30Z

book/src/debugging.md

+
+We also can try to identify how the token `fast` works by looking at `@9`, first, and seeing that `f` will cause Logos to jump to `7`. This will then resolve the last letters of our word _fast_ by matching `ast` which jumps to `@8`. Since our provided _input_ to the lexer does not include alphabetic characters after the word "fast" but rather whitespace, the token `::Fast` will be recognized. Then the graph will look for further potential continuation (i.e., `[g-z] => 4`)
+
+### Enabling 


This and the following section should be one level above, so:

## Enabling ### Step 1...

Also, maybe name the steps (instead of saying step 1, step 2). But this may change as a result of #382.

I rebased and provided a simplified example that leverages your debug feature.

I also believe that "# Enabling" should be the first section.

book/src/debugging.md

Co-authored-by: João Marcos <marcospb19@hotmail.com>

jeertmans added the book Related to the book label Feb 19, 2024

jeertmans requested changes Feb 19, 2024

View reviewed changes

jeertmans mentioned this pull request Feb 20, 2024

chore(lib): add debug feature #382

Merged

jeertmans requested changes Feb 20, 2024

View reviewed changes

marcospb19 reviewed Feb 21, 2024

View reviewed changes

book/src/debugging.md Outdated Show resolved Hide resolved

afreeland force-pushed the debug-graph branch from f23af3f to 39a91fd Compare February 24, 2024 17:11

Add debug support and documentation for internal graph output

7ce0526

afreeland force-pushed the debug-graph branch from 39a91fd to 7ce0526 Compare February 24, 2024 17:17

Refine enabling feature

f342865

marcospb19 reviewed Feb 28, 2024

View reviewed changes

book/src/debugging.md Outdated Show resolved Hide resolved

Update book/src/debugging.md

ae18909

Co-authored-by: João Marcos <marcospb19@hotmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds graph debug documentation to book #379

Adds graph debug documentation to book #379

afreeland commented Feb 19, 2024

jeertmans left a comment

jeertmans Feb 19, 2024

jeertmans Feb 19, 2024

jeertmans Feb 19, 2024

jeertmans Feb 19, 2024

jeertmans Feb 19, 2024

afreeland Feb 20, 2024

jeertmans Feb 19, 2024

afreeland commented Feb 20, 2024

jeertmans commented Feb 20, 2024

jeertmans left a comment

jeertmans Feb 20, 2024

afreeland Feb 27, 2024

marcospb19 Feb 28, 2024

		# Debug Graph

		It may be beneficial during debugging to be able to visualize how Logos has generated its graph.

	With our graph debugging enabled, we see the following output:
	Logos actually constructs a graph that contains the logic for matching tokens:

-Lets look at `@9` for the character `.` we see that it will jump `=>` to `2`.  We can then follow that by looking at `@2` which resolves to our `::Period` token. It will then continue to look for any matches in the case there is potential continuation after the `.` character. In the _input_ we provided there is not, since it is the end of our input.
+This graph can help us understand how our patterns are matched, and maybe understand why we have a bug at some point.
+Let's look at the last node, `9`, for the character `.`. We see that it will jump (`=>`) to `2`.  We can then follow that by looking at `2` which resolves to our `::Period` token. It will then continue to look for any matches in the case there is potential continuation after the `.` character. In the _input_ we provided, there is not, since it is the end of our input.


		Lets look at `@9` for the character `.` we see that it will jump `=>` to `2`. We can then follow that by looking at `@2` which resolves to our `::Period` token. It will then continue to look for any matches in the case there is potential continuation after the `.` character. In the _input_ we provided there is not, since it is the end of our input.

		We also can try to identify how `fast` works by looking at `@9` that `f` jumps to `7`. This will then resolve the last letters of our word _fast_ by matching `ast` which jumps to `8`. Since our provided _input_ to the lexer does not include alphanumeric character after the word "fast" but rather whitespace, the token `::Fast` will be recognized. Then the graph will look for further potential continuation (ie `[g-z] => 4`)


		We also can try to identify how the token `fast` works by looking at `@9`, first, and seeing that `f` will cause Logos to jump to `7`. This will then resolve the last letters of our word _fast_ by matching `ast` which jumps to `@8`. Since our provided _input_ to the lexer does not include alphabetic characters after the word "fast" but rather whitespace, the token `::Fast` will be recognized. Then the graph will look for further potential continuation (i.e., `[g-z] => 4`)

		### Enabling

Adds graph debug documentation to book #379

Are you sure you want to change the base?

Adds graph debug documentation to book #379

Conversation

afreeland commented Feb 19, 2024

jeertmans left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

afreeland commented Feb 20, 2024

jeertmans commented Feb 20, 2024

jeertmans left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment