Theoretical question

[angeldollface]

I've been trying to implement an interpreter for a language of my own. I struggle with defining a module that parses and analyzes a stream of tokens. Where can I find the module for the grammar definiton? And the module that generates an AST?

[DimitrisJim]

Here's an overview of the back end https://rustpython.github.io/blog/2020/04/02/thing-explainer-parser.html, some paths might be different but the filenames are generally the same. You should probably be able to get an idea of how things generally work.

[angeldollface]

Hi @DimitrisJim !❤️ Wow, thank you so much. This really helps! I'm gonna try this out soon! I didn't expect anyone to actually answer a pretty generic question like this but your help means a lot! Can I maybe come back here if I run into any trouble? I know you must have a lot to do and no reason to help but this problem has been vexing me for the past ~2 years.🙄🙄

[angeldollface]

And I am truly sorry for only answering now. I had a lot on my plate what with end of the semester and all that jazz.🙈

[DimitrisJim]

No problem. You can back and ask again, can't guarantee any answers but it's open. You can also pop by our discord, more people would, in theory, be able to answer a question of yours.

[angeldollface]

I also did drop by the Discord and was pointed to the Rust Lang community server but that isn't an option for me at the moment because I had some trouble there long time ago and got banned and am currently applying there for readmittance.

[angeldollface]

@DimitrisJim Hey! :) So, I tried reading some of the source in the blog post that was linked here and stumbled across the enum that tries to determine what type of statement was written. My latest commit to Mandy here kind of implements that but I don't know how to proceed now. Could you maybe post a small example of how I would construct an AST from a stream of tokens that are just a struct with the token name and value? I know it might be asking a bit much but I'm really quite lost.

[youknowone]

Hi, I think explaining our rustpython code base is a discussion topic of our repository, but a generic programming language development is not. Are you looking for programming language development community?

[fanninpm]

To be fair, I've read a bunch of the Crafting Interpreters book, and I still have a few lingering questions about how RustPython does things.

[DimitrisJim]

@fanninpm you should ask them, it would be helpful to others (myself included).

@angeldollface this is getting somewhat out of scope indeed, the book linked in the previous comment would probably be the best approach since we can't really guide you through all the aspects of developing a language.

[angeldollface]

Hmm, I had a look at Crafting Interpreters. Frankly, the language is terrible (sample read) and assumes too much, for me, anyway. Also, as a student, I don't really have the time and money resources to read it. Guess, I'll just stay ignorant then. 🙃

[angeldollface]

Thanks all the same, I guess.🤷‍♀️