-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Question: Do you reimplement the typescript ast? #101
Comments
Sort of. I wrote my own code that parses TypeScript, but it discards the information about type annotations instead of creating AST nodes for them. The information isn't needed to convert TypeScript to JavaScript and it's more efficient to just treat type annotations the same as whitespace. TypeScript-only syntax such as It was pretty crazy though. I unfortunately had to add backtracking to handle certain constructs. I'll also have to keep it updated as the TypeScript project adds new type system syntax, which luckily happens pretty slowly. Here were some of the most complicated parts: esbuild/internal/parser/parser.go Lines 713 to 727 in 8d93493
esbuild/internal/parser/parser.go Lines 2347 to 2378 in 8d93493
|
Wow, that is crazy. So basically, you wrote an entire JS/TS parser from scratch, albeit a bit lossy since the resulting AST only records tokens that will actually end up in the generated js bundle. Another question: is it a non-goal of |
Yes, that's a non-goal. The AST format is currently an internal implementation detail. AST transformation passes are all merged together into as few passes as possible, which is great for speed but not for a plug-and-play architecture. The goals of this project are roughly a) speed and correctness, b) build a real bundler, and c) inspire the community to build faster tools. It's hard enough building a working bundler from scratch with all of these features. It would be a distraction from these goals to also attempt to build a novel plug-and-play architecture with abstractions that enable a large community-run project at the same time, especially since doing that often comes at the expense of performance. Perhaps once this project is more mature it can then be factored into something more like that, or another project that's more like that architecture can be started. Right now esbuild is still an experiment. |
That’s awesome. After playing around with Figma and seeing the black magic that enabled it to work in the browser, you were already a legend. This project just cements your legendary status in my mind. Thanks so much for taking the time to respond to these questions. |
Just out of curiosity. Does that means that esbuild would not throw type errors? e.g. function greetings(name: string) {
return `hello, ${name}`;
}
greetings(42); // would throw type error if compiled with tsc |
Yes, for a few reasons:
|
I've thought a bit about how one might do this because their type checker is really slow, and it definitely breaks my flow sometimes. It's especially annoying to not be able to see type errors in files you don't currently have open in their IDE, which I assume is turned off because it's too slow. A somewhat straightforward way to speed it up without the blessing of the TypeScript team and without signing up for manually porting every commit would be to come up with an automated porting process to a more efficient language. This would likely only be a moderate speedup because it's still the same single-threaded code, and you'd still have to handle a lot of JavaScript's quirks that slow it down. But you could probably cut some corners and special-case the conversion for the type of JavaScript that the TypeScript compiler team writes. There may be some things about the compiler code that using a more efficient language could speed up, even if you don't rewrite their code. For example, I think the TypeScript compiler makes pretty heavy use of megamorphic object shapes for its AST nodes, which is a somewhat of a performance anti-pattern for modern JavaScript JITs with inline caches. There are other languages that don't have that problem. A language with an optimizing compiler may also be able to do certain optimizations such as method devirtualization and function inlining that the JavaScript JIT can't do in certain cases. These performance improvements are usually hard to notice because it's death-by-a-thousand-cuts everywhere, but they can be a significant performance improvement (potentially 2x faster?). Here is some proof that wins like this exist: microsoft/TypeScript#39247. I say this with the caveat that I haven't extensively profiled the TypeScript compiler itself, but I do have extensive experience with writing a web programming language with an optimizing compiler that had to deal with similar issues. But the real way to speed up the TypeScript compiler would be to get the TypeScript compiler team itself on board and motivated about getting major performance improvements in their compiler (major enough to consider either porting to a native language or writing their TypeScript differently to make automatic porting easier). That's going to be more sustainable and could also lead to bigger wins since you could potentially make algorithmic speedups, similar to what esbuild does for JavaScript bundling. But I assume that's never going to happen because the TypeScript compiler project is very mature at this point. I'm guessing their type checking algorithm is also not straightforward to parallelize so it may not be possible to go faster on multi-core machines. So it would be an interesting project to tackle for sure! I might attempt it some day. But it's a separate project from esbuild. |
I just have to note that your megamorphic object shapes link was enlightening, educational, and highly entertaining. (eagle to mother goose unknown shape spotted operation DEOPT is a go) |
To get your blazing speeds, did you reverse engineer the typescript parser in golang? If so it seems crazy to me that this works! In either case, nice job on this library.
The text was updated successfully, but these errors were encountered: