Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DO NOT REVIEW] Perf testing module-ified TypeScript compiler, take two #50811

Closed
wants to merge 22 commits into from

Conversation

jakebailey
Copy link
Member

I can't reopen #50765 due to my rebasing, so I'm opening it again so I can test downleveling of const/let.

This step makes further commits look clearer by unindenting all of the top level namespaces preemptively.
This step makes all implicit namespace accesses explicit, e.g. "Node" turns into "ts.Node".
This step converts each file into an exported module by hoisting the namespace bodies into the global scope and transferring internal markers down onto declarations as needed.

The namespaces are reconstructed as "barrel"-style modules, which are identical to the old namespace objects in structure. These reconstructed namespaces are then imported in the newly module-ified files, making existing expressions like "ts." valid.
This step converts as many explicit accesses as possible in favor of direct imports from the modules in which things were declared. This restores the code (as much as possible) back to how it looked originally before the explicitify step, e.g. instead of "ts.Node" and "ts.Symbol", we have just "Node" and "Symbol".
Now that we are modules, there's no reason to ban multiple namespaces
per file; each file is its own scope and declaring a namespace won't
merge it into any other files.
This project is the same as the (soon added) typescript project.
If these are regular comments, then they won't appear in our d.ts files.
But, now we are relying on api-extractor to produce out final merged
d.ts files, so they need to be present in the "input" d.ts files,
meaning they have to be JSDoc comments.

These comments only work today because all of our builds load their TS
files from scratch, so they see the actual source files and their
non-JSDoc comments.

The comments also need to be attached to a declaration, not floating,
otherwise they won't be used by api-extractor, so move them if needed.
These appear to break api-extractor, and we won't have a fix until TS
starts to emit d.ts files using `import type { ... }` for imports that
aren't explicitly written in the source files.
The import expression for the ETW trace types breaks API extractor; it
outputs a named import for a non-existent name in the module rather than
emitting a default import. For now, redeclare the type locally and then
use a conditional type to verify that what we have is compatible with
the upstream type when type checking locally.
This configures the existing build tasks to use esbuild by defualt. If
using the plain files is desired, passing `--bundle=false` will build
using plain files and still produce a runnable system.
@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Sep 16, 2022
@typescript-bot
Copy link
Collaborator

Thanks for the PR! It looks like you've changed the TSServer protocol in some way. Please ensure that any changes here don't break consumers of the current TSServer API. For some extra review, we'll ping @sheetalkamat, @amcasey, @mjbvz, @minestarks for you. Feel free to loop in other consumers/maintainers if necessary

@jakebailey

This comment was marked as outdated.

@typescript-bot

This comment was marked as outdated.

@jakebailey
Copy link
Member Author

jakebailey commented Sep 16, 2022

@typescript-bot perf test this faster

This is the "baseline", i.e. just esbuild on src.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Sep 16, 2022

Heya @jakebailey, I've started to run the abridged perf test suite on this PR at f37e88f. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..50811

Metric main 50811 Delta Best Worst
Angular - node (v14.15.1, x64)
Memory used 338,923k (± 0.01%) 328,472k (± 0.01%) 🟩-10,451k (- 3.08%) 328,409k 328,506k
Parse Time 2.06s (± 0.88%) 2.06s (± 0.52%) -0.00s (- 0.05%) 2.04s 2.09s
Bind Time 0.80s (± 0.46%) 0.70s (± 0.57%) 🟩-0.10s (-12.19%) 0.69s 0.71s
Check Time 5.84s (± 0.36%) 5.35s (± 0.39%) 🟩-0.50s (- 8.47%) 5.30s 5.41s
Emit Time 6.19s (± 0.48%) 5.20s (± 0.82%) 🟩-0.98s (-15.91%) 5.10s 5.31s
Total Time 14.89s (± 0.34%) 13.31s (± 0.37%) 🟩-1.58s (-10.62%) 13.16s 13.41s
Compiler-Unions - node (v14.15.1, x64)
Memory used 190,148k (± 0.02%) 182,299k (± 0.44%) 🟩-7,849k (- 4.13%) 181,451k 185,426k
Parse Time 0.85s (± 0.58%) 0.89s (± 0.65%) +0.03s (+ 3.86%) 0.87s 0.90s
Bind Time 0.48s (± 0.62%) 0.46s (± 0.87%) 🟩-0.02s (- 4.77%) 0.45s 0.47s
Check Time 6.72s (± 0.28%) 6.22s (± 0.67%) 🟩-0.50s (- 7.40%) 6.14s 6.32s
Emit Time 2.38s (± 0.84%) 1.99s (± 0.98%) 🟩-0.39s (-16.36%) 1.95s 2.05s
Total Time 10.44s (± 0.27%) 9.56s (± 0.60%) 🟩-0.88s (- 8.44%) 9.44s 9.73s
Monaco - node (v14.15.1, x64)
Memory used 326,554k (± 0.01%) 315,704k (± 0.01%) 🟩-10,849k (- 3.32%) 315,632k 315,764k
Parse Time 1.59s (± 0.84%) 1.56s (± 0.48%) -0.03s (- 1.83%) 1.54s 1.57s
Bind Time 0.73s (± 0.80%) 0.63s (± 0.94%) 🟩-0.09s (-12.79%) 0.62s 0.65s
Check Time 5.72s (± 0.55%) 5.14s (± 0.44%) 🟩-0.58s (-10.15%) 5.09s 5.19s
Emit Time 3.31s (± 0.78%) 2.79s (± 0.71%) 🟩-0.51s (-15.57%) 2.76s 2.85s
Total Time 11.34s (± 0.34%) 10.12s (± 0.29%) 🟩-1.22s (-10.77%) 10.07s 10.21s
TFS - node (v14.15.1, x64)
Memory used 289,697k (± 0.01%) 280,802k (± 0.01%) 🟩-8,895k (- 3.07%) 280,771k 280,834k
Parse Time 1.29s (± 0.76%) 1.32s (± 0.76%) +0.03s (+ 2.55%) 1.30s 1.35s
Bind Time 0.79s (± 0.47%) 0.58s (± 0.63%) 🟩-0.20s (-25.48%) 0.58s 0.59s
Check Time 5.35s (± 0.48%) 5.03s (± 0.43%) 🟩-0.32s (- 6.00%) 4.99s 5.07s
Emit Time 3.55s (± 0.75%) 2.95s (± 0.56%) 🟩-0.60s (-16.91%) 2.92s 2.99s
Total Time 10.98s (± 0.25%) 9.89s (± 0.28%) 🟩-1.09s (- 9.95%) 9.82s 9.95s
material-ui - node (v14.15.1, x64)
Memory used 436,661k (± 0.01%) 427,373k (± 0.00%) -9,288k (- 2.13%) 427,338k 427,407k
Parse Time 1.86s (± 0.58%) 1.87s (± 0.46%) +0.01s (+ 0.48%) 1.85s 1.88s
Bind Time 0.58s (± 1.02%) 0.53s (± 0.65%) 🟩-0.06s (- 9.76%) 0.52s 0.53s
Check Time 12.89s (± 0.80%) 12.00s (± 0.48%) 🟩-0.89s (- 6.90%) 11.88s 12.18s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.33s (± 0.68%) 14.40s (± 0.40%) 🟩-0.94s (- 6.11%) 14.29s 14.58s
xstate - node (v14.15.1, x64)
Memory used 546,997k (± 0.00%) 517,236k (± 0.01%) 🟩-29,760k (- 5.44%) 517,172k 517,317k
Parse Time 2.60s (± 0.31%) 2.68s (± 0.57%) +0.08s (+ 3.11%) 2.65s 2.71s
Bind Time 0.98s (± 1.08%) 0.85s (± 0.86%) 🟩-0.13s (-13.20%) 0.83s 0.86s
Check Time 1.52s (± 0.44%) 1.47s (± 0.50%) 🟩-0.05s (- 3.42%) 1.46s 1.49s
Emit Time 0.07s (± 0.00%) 0.07s (± 4.13%) +0.00s (+ 2.86%) 0.07s 0.08s
Total Time 5.17s (± 0.37%) 5.08s (± 0.37%) -0.09s (- 1.80%) 5.05s 5.13s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-210-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 50811 10
Baseline main 10

Developer Information:

Download Benchmark

@jakebailey
Copy link
Member Author

@typescript-bot perf test this faster

Now, with let/const removed via babel.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Sep 16, 2022

Heya @jakebailey, I've started to run the abridged perf test suite on this PR at 5c3013d. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..50811

Metric main 50811 Delta Best Worst
Angular - node (v14.15.1, x64)
Memory used 338,923k (± 0.01%) 328,287k (± 0.00%) 🟩-10,636k (- 3.14%) 328,244k 328,312k
Parse Time 2.06s (± 0.88%) 1.93s (± 0.41%) 🟩-0.13s (- 6.41%) 1.91s 1.95s
Bind Time 0.80s (± 0.46%) 0.69s (± 0.71%) 🟩-0.10s (-12.81%) 0.69s 0.71s
Check Time 5.84s (± 0.36%) 5.28s (± 0.39%) 🟩-0.57s (- 9.67%) 5.25s 5.33s
Emit Time 6.19s (± 0.48%) 5.15s (± 0.56%) 🟩-1.04s (-16.81%) 5.07s 5.21s
Total Time 14.89s (± 0.34%) 13.05s (± 0.26%) 🟩-1.84s (-12.33%) 12.98s 13.15s
Compiler-Unions - node (v14.15.1, x64)
Memory used 190,148k (± 0.02%) 182,374k (± 0.39%) 🟩-7,774k (- 4.09%) 181,976k 185,246k
Parse Time 0.85s (± 0.58%) 0.81s (± 0.37%) 🟩-0.05s (- 5.39%) 0.80s 0.81s
Bind Time 0.48s (± 0.62%) 0.45s (± 0.50%) 🟩-0.03s (- 6.85%) 0.44s 0.45s
Check Time 6.72s (± 0.28%) 6.16s (± 0.50%) 🟩-0.56s (- 8.38%) 6.10s 6.22s
Emit Time 2.38s (± 0.84%) 1.96s (± 0.76%) 🟩-0.41s (-17.45%) 1.94s 2.01s
Total Time 10.44s (± 0.27%) 9.38s (± 0.32%) 🟩-1.06s (-10.16%) 9.30s 9.43s
Monaco - node (v14.15.1, x64)
Memory used 326,554k (± 0.01%) 315,437k (± 0.01%) 🟩-11,117k (- 3.40%) 315,401k 315,488k
Parse Time 1.59s (± 0.84%) 1.47s (± 0.35%) 🟩-0.11s (- 7.25%) 1.46s 1.48s
Bind Time 0.73s (± 0.80%) 0.62s (± 0.55%) 🟩-0.10s (-14.31%) 0.62s 0.63s
Check Time 5.72s (± 0.55%) 5.12s (± 0.50%) 🟩-0.60s (-10.41%) 5.06s 5.18s
Emit Time 3.31s (± 0.78%) 2.74s (± 0.42%) 🟩-0.56s (-17.02%) 2.71s 2.77s
Total Time 11.34s (± 0.34%) 9.96s (± 0.32%) 🟩-1.38s (-12.20%) 9.89s 10.04s
TFS - node (v14.15.1, x64)
Memory used 289,697k (± 0.01%) 278,434k (± 0.01%) 🟩-11,263k (- 3.89%) 278,374k 278,495k
Parse Time 1.29s (± 0.76%) 1.26s (± 1.47%) -0.03s (- 2.48%) 1.23s 1.31s
Bind Time 0.79s (± 0.47%) 0.59s (± 0.62%) 🟩-0.20s (-25.35%) 0.58s 0.59s
Check Time 5.35s (± 0.48%) 4.97s (± 0.46%) 🟩-0.38s (- 7.01%) 4.92s 5.02s
Emit Time 3.55s (± 0.75%) 2.71s (± 0.54%) 🟩-0.84s (-23.61%) 2.69s 2.75s
Total Time 10.98s (± 0.25%) 9.53s (± 0.30%) 🟩-1.45s (-13.19%) 9.49s 9.60s
material-ui - node (v14.15.1, x64)
Memory used 436,661k (± 0.01%) 427,047k (± 0.00%) -9,614k (- 2.20%) 427,006k 427,094k
Parse Time 1.86s (± 0.58%) 1.77s (± 0.42%) 🟩-0.09s (- 4.79%) 1.76s 1.79s
Bind Time 0.58s (± 1.02%) 0.53s (± 0.71%) 🟩-0.06s (-10.10%) 0.52s 0.53s
Check Time 12.89s (± 0.80%) 11.91s (± 0.35%) 🟩-0.98s (- 7.59%) 11.81s 11.99s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.33s (± 0.68%) 14.21s (± 0.32%) 🟩-1.12s (- 7.33%) 14.10s 14.31s
xstate - node (v14.15.1, x64)
Memory used 546,997k (± 0.00%) 516,843k (± 0.00%) 🟩-30,154k (- 5.51%) 516,784k 516,900k
Parse Time 2.60s (± 0.31%) 2.53s (± 0.67%) -0.07s (- 2.69%) 2.50s 2.57s
Bind Time 0.98s (± 1.08%) 0.84s (± 0.68%) 🟩-0.14s (-14.02%) 0.83s 0.85s
Check Time 1.52s (± 0.44%) 1.46s (± 0.52%) 🟩-0.06s (- 3.68%) 1.45s 1.48s
Emit Time 0.07s (± 0.00%) 0.07s (± 3.23%) -0.00s (- 1.43%) 0.06s 0.07s
Total Time 5.17s (± 0.37%) 4.91s (± 0.51%) 🟩-0.26s (- 5.13%) 4.87s 4.98s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-210-generic
Architecturex64
Available Memory16 GB
Available Memory15 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 50811 10
Baseline main 10

Developer Information:

Download Benchmark

@jakebailey
Copy link
Member Author

Closing; I've summarized this on #49332 (comment).

@jakebailey jakebailey closed this Sep 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants