Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve logic that chooses co- vs. contra-variant inferences #57909

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

Andarist
Copy link
Contributor

@Andarist Andarist commented Mar 22, 2024

To review this it might be helpful to see how this evolved over time:
#27028
#46392
#52123
#52180
#54072

The reason why this inference fails today is that isTypeSubtypeOf leads to requireOptionalProperties === true. This has such inferences:

inferredCovariantType // { query: string }
inferredContravariantType // { query?: unknown; body?: unknown; }

So the covariant inference lacks body property and thus it fails the check and the contravariant inference gets chosen at the end.

fixes #57908
fixes #58468

@typescript-bot typescript-bot added the For Uncommitted Bug PR for untriaged, rejected, closed or missing bug label Mar 22, 2024
@typescript-bot
Copy link
Collaborator

This PR doesn't have any linked issues. Please open an issue that references this PR. From there we can discuss and prioritise.

@jakebailey
Copy link
Member

@typescript-bot test top800
@typescript-bot user test this
@typescript-bot run dt

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 22, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
test top800 ✅ Started
user test this ✅ Started 👀 Results
run dt ✅ Started 👀 Results
perf test this faster ✅ Started 👀 Results

@@ -26465,10 +26465,10 @@ export function createTypeChecker(host: TypeCheckerHost): TypeChecker {
// and has inferences that would conflict. Otherwise, we prefer the contra-variant inference.
const preferCovariantType = inferredCovariantType && (!inferredContravariantType ||
!(inferredCovariantType.flags & TypeFlags.Never) &&
some(inference.contraCandidates, t => isTypeSubtypeOf(inferredCovariantType, t)) &&
some(inference.contraCandidates, t => isTypeAssignableTo(inferredCovariantType, t)) &&
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is not the correct change then at the very least a test case should be proposed that shows how the isTypeSubtypeOf is better.

When this check was originally introduced by @ahejlsberg here it was states that:

Furthermore, knowing that an error will result when the co-variant inference is not a subtype of the contra-variant inference, we now prefer the contra-variant inference because it is likely to have come from an explicit type annotation on a function. This improves our error reporting.

An alternative idea to fix this example would be to keep coAndContraRelationCheck on inferenceContext. From what I understand, the subtypeRelation in this context is mainly used when dealing with multiple overloads - a single signature case uses assignableRelation with chooseOverload. So perhaps the used relation should determine what is being used here.

@typescript-bot
Copy link
Collaborator

Hey @jakebailey, the results of running the DT tests are ready.

There were interesting changes:

Branch only errors:

Package: react
Error:

Error: 
/mnt/vss/_work/1/DefinitelyTyped/types/react/test/index.ts
  794:5  error  TypeScript@local compile error: 
Unused '@ts-expect-error' directive  @definitelytyped/expect

✖ 1 problem (1 error, 0 warnings)

    at combineErrorsAndWarnings (/mnt/vss/_work/1/DefinitelyTyped/node_modules/.pnpm/@definitelytyped+dtslint@0.2.18_typescript@5.5.0-dev.20240322/node_modules/@definitelytyped/dtslint/dist/index.js:194:28)
    at runTests (/mnt/vss/_work/1/DefinitelyTyped/node_modules/.pnpm/@definitelytyped+dtslint@0.2.18_typescript@5.5.0-dev.20240322/node_modules/@definitelytyped/dtslint/dist/index.js:186:20)

Package: styled-theming
Error:

Error: 
/mnt/vss/_work/1/DefinitelyTyped/types/styled-theming/styled-theming-tests.tsx
  72:47  error  TypeScript@local compile error: 
No overload matches this call.
  The last overload gave the following error.
    Type '"wrong variant"' is not assignable to type '"primary" | "secondary" | undefined'  @definitelytyped/expect

✖ 1 problem (1 error, 0 warnings)

    at combineErrorsAndWarnings (/mnt/vss/_work/1/DefinitelyTyped/node_modules/.pnpm/@definitelytyped+dtslint@0.2.18_typescript@5.5.0-dev.20240322/node_modules/@definitelytyped/dtslint/dist/index.js:194:28)
    at runTests (/mnt/vss/_work/1/DefinitelyTyped/node_modules/.pnpm/@definitelytyped+dtslint@0.2.18_typescript@5.5.0-dev.20240322/node_modules/@definitelytyped/dtslint/dist/index.js:186:20)

You can check the log here.

@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the user tests comparing main and refs/pull/57909/merge:

Something interesting changed - please have a look.

Details

uglify-js

/mnt/ts_downloads/_/m/uglify-js/tsconfig.json

  • [NEW] error TS2684: The 'this' context of type '(...items: never[]) => number' is not assignable to method's 'this' of type '(this: any[], ...args: any) => number'.
    • /mnt/ts_downloads/_/m/uglify-js/node_modules/uglify-js/lib/compress.js(293,17)
    • /mnt/ts_downloads/_/m/uglify-js/node_modules/uglify-js/lib/compress.js(2472,29)
    • /mnt/ts_downloads/_/m/uglify-js/node_modules/uglify-js/lib/compress.js(7469,47)
    • /mnt/ts_downloads/_/m/uglify-js/node_modules/uglify-js/lib/compress.js(14005,13)
  • [NEW] error TS2684: The 'this' context of type '(...items: never[]) => number' is not assignable to method's 'this' of type '(this: any, ...args: any) => number'.
    • /mnt/ts_downloads/_/m/uglify-js/node_modules/uglify-js/lib/compress.js(1867,13)
    • /mnt/ts_downloads/_/m/uglify-js/node_modules/uglify-js/lib/compress.js(7298,64)
    • /mnt/ts_downloads/_/m/uglify-js/node_modules/uglify-js/lib/compress.js(7981,33)
    • /mnt/ts_downloads/_/m/uglify-js/node_modules/uglify-js/lib/compress.js(8012,29)
    • /mnt/ts_downloads/_/m/uglify-js/node_modules/uglify-js/lib/compress.js(14002,17)
    • /mnt/ts_downloads/_/m/uglify-js/node_modules/uglify-js/lib/parse.js(1863,17)

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,529k (± 0.00%) 295,524k (± 0.00%) ~ 295,506k 295,544k p=0.423 n=6
Parse Time 2.66s (± 0.31%) 2.66s (± 0.28%) ~ 2.65s 2.67s p=0.209 n=6
Bind Time 0.83s (± 0.66%) 0.82s (± 0.99%) ~ 0.81s 0.83s p=0.859 n=6
Check Time 8.20s (± 0.37%) 8.20s (± 0.15%) ~ 8.18s 8.21s p=0.809 n=6
Emit Time 7.04s (± 0.20%) 7.09s (± 0.88%) ~ 7.00s 7.18s p=0.107 n=6
Total Time 18.72s (± 0.16%) 18.76s (± 0.32%) ~ 18.69s 18.86s p=0.225 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 191,964k (± 0.06%) 193,733k (± 0.95%) ~ 191,878k 195,497k p=0.128 n=6
Parse Time 1.36s (± 0.89%) 1.36s (± 1.21%) ~ 1.33s 1.38s p=1.000 n=6
Bind Time 0.72s (± 0.00%) 0.72s (± 0.00%) ~ 0.72s 0.72s p=1.000 n=6
Check Time 9.56s (± 0.60%) 9.49s (± 0.37%) -0.08s (- 0.80%) 9.44s 9.54s p=0.045 n=6
Emit Time 2.62s (± 0.51%) 2.63s (± 0.64%) ~ 2.61s 2.65s p=0.227 n=6
Total Time 14.26s (± 0.40%) 14.19s (± 0.39%) ~ 14.11s 14.28s p=0.090 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,372k (± 0.01%) 347,374k (± 0.01%) ~ 347,324k 347,407k p=0.873 n=6
Parse Time 2.47s (± 0.33%) 2.48s (± 0.40%) ~ 2.47s 2.49s p=0.498 n=6
Bind Time 0.93s (± 0.44%) 0.92s (± 0.59%) ~ 0.92s 0.93s p=0.282 n=6
Check Time 7.00s (± 0.41%) 7.02s (± 0.49%) ~ 6.97s 7.07s p=0.465 n=6
Emit Time 4.08s (± 0.37%) 4.07s (± 0.40%) ~ 4.04s 4.08s p=0.157 n=6
Total Time 14.49s (± 0.26%) 14.49s (± 0.23%) ~ 14.45s 14.55s p=0.872 n=6
TFS - node (v18.15.0, x64)
Memory used 302,726k (± 0.02%) 302,708k (± 0.01%) ~ 302,678k 302,760k p=0.575 n=6
Parse Time 2.41s (± 0.68%) 2.39s (± 1.72%) ~ 2.34s 2.44s p=0.571 n=6
Bind Time 1.19s (± 0.53%) 1.19s (± 1.02%) ~ 1.17s 1.20s p=0.673 n=6
Check Time 7.44s (± 0.56%) 7.45s (± 0.46%) ~ 7.41s 7.50s p=0.376 n=6
Emit Time 4.28s (± 0.41%) 4.29s (± 0.58%) ~ 4.26s 4.33s p=1.000 n=6
Total Time 15.33s (± 0.34%) 15.32s (± 0.48%) ~ 15.24s 15.42s p=0.688 n=6
material-ui - node (v18.15.0, x64)
Memory used 509,876k (± 0.01%) 509,895k (± 0.00%) ~ 509,875k 509,907k p=0.125 n=6
Parse Time 2.64s (± 0.37%) 2.65s (± 0.65%) ~ 2.63s 2.67s p=0.215 n=6
Bind Time 0.99s (± 0.99%) 0.98s (± 0.56%) ~ 0.98s 0.99s p=0.322 n=6
Check Time 17.23s (± 0.22%) 17.23s (± 0.37%) ~ 17.15s 17.32s p=1.000 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.86s (± 0.19%) 20.87s (± 0.29%) ~ 20.81s 20.98s p=0.746 n=6
mui-docs - node (v18.15.0, x64)
Memory used 1,737,544k (± 0.00%) 1,737,524k (± 0.00%) ~ 1,737,491k 1,737,554k p=0.230 n=6
Parse Time 7.78s (± 0.38%) 7.80s (± 0.61%) ~ 7.74s 7.88s p=0.373 n=6
Bind Time 2.80s (± 0.32%) 2.81s (± 0.83%) ~ 2.78s 2.84s p=0.806 n=6
Check Time 66.38s (± 0.34%) 66.49s (± 0.45%) ~ 66.11s 66.91s p=0.521 n=6
Emit Time 0.15s (± 3.53%) 0.16s (± 3.29%) ~ 0.15s 0.16s p=0.640 n=6
Total Time 77.12s (± 0.32%) 77.25s (± 0.34%) ~ 76.98s 77.63s p=0.471 n=6
self-build-src - node (v18.15.0, x64)
Memory used 2,394,217k (± 0.02%) 2,393,952k (± 0.04%) ~ 2,392,566k 2,394,786k p=0.471 n=6
Parse Time 6.07s (± 0.90%) 6.07s (± 0.68%) ~ 6.02s 6.11s p=0.688 n=6
Bind Time 2.26s (± 0.52%) 2.24s (± 1.68%) ~ 2.17s 2.28s p=0.415 n=6
Check Time 39.48s (± 0.29%) 39.61s (± 0.22%) +0.14s (+ 0.34%) 39.48s 39.73s p=0.045 n=6
Emit Time 3.12s (± 0.48%) 3.15s (± 1.61%) ~ 3.08s 3.21s p=0.520 n=6
Total Time 50.94s (± 0.22%) 51.08s (± 0.20%) +0.14s (+ 0.27%) 50.89s 51.20s p=0.045 n=6
self-compiler - node (v18.15.0, x64)
Memory used 415,185k (± 0.01%) 415,193k (± 0.01%) ~ 415,144k 415,226k p=0.936 n=6
Parse Time 3.43s (± 0.96%) 3.42s (± 0.75%) ~ 3.39s 3.45s p=0.807 n=6
Bind Time 1.29s (± 0.76%) 1.29s (± 0.82%) ~ 1.27s 1.30s p=0.547 n=6
Check Time 18.02s (± 0.35%) 18.01s (± 0.43%) ~ 17.89s 18.11s p=0.872 n=6
Emit Time 1.31s (± 1.12%) 1.32s (± 0.88%) ~ 1.31s 1.34s p=0.324 n=6
Total Time 24.06s (± 0.35%) 24.04s (± 0.30%) ~ 23.94s 24.16s p=0.806 n=6
vscode - node (v18.15.0, x64)
Memory used 2,889,942k (± 0.00%) 2,889,412k (± 0.00%) -531k (- 0.02%) 2,889,325k 2,889,479k p=0.005 n=6
Parse Time 12.95s (± 0.27%) 12.94s (± 0.27%) ~ 12.91s 13.00s p=0.683 n=6
Bind Time 4.13s (± 0.82%) 4.14s (± 0.64%) ~ 4.10s 4.17s p=0.746 n=6
Check Time 71.13s (± 0.24%) 71.45s (± 0.31%) +0.32s (+ 0.45%) 71.14s 71.74s p=0.045 n=6
Emit Time 19.50s (± 0.88%) 19.40s (± 0.79%) ~ 19.18s 19.60s p=0.336 n=6
Total Time 107.71s (± 0.35%) 107.93s (± 0.32%) ~ 107.57s 108.40s p=0.378 n=6
webpack - node (v18.15.0, x64)
Memory used 408,193k (± 0.03%) 408,153k (± 0.01%) ~ 408,080k 408,221k p=0.810 n=6
Parse Time 3.89s (± 0.72%) 3.90s (± 0.67%) ~ 3.86s 3.94s p=0.623 n=6
Bind Time 1.67s (± 0.49%) 1.69s (± 0.89%) +0.02s (+ 1.20%) 1.67s 1.70s p=0.031 n=6
Check Time 16.80s (± 0.32%) 16.72s (± 0.20%) -0.08s (- 0.46%) 16.67s 16.77s p=0.037 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.36s (± 0.18%) 22.31s (± 0.20%) ~ 22.25s 22.36s p=0.077 n=6
xstate - node (v18.15.0, x64)
Memory used 512,982k (± 0.01%) 513,035k (± 0.02%) ~ 512,919k 513,174k p=0.471 n=6
Parse Time 3.96s (± 0.43%) 3.94s (± 0.42%) ~ 3.92s 3.97s p=0.052 n=6
Bind Time 1.85s (± 0.90%) 1.85s (± 1.15%) ~ 1.82s 1.87s p=1.000 n=6
Check Time 3.37s (± 0.55%) 3.37s (± 0.24%) ~ 3.36s 3.38s p=0.870 n=6
Emit Time 0.08s (± 6.19%) 0.09s (± 6.44%) ~ 0.08s 0.09s p=0.640 n=6
Total Time 9.28s (± 0.19%) 9.25s (± 0.33%) ~ 9.22s 9.30s p=0.090 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@jakebailey
Copy link
Member

@typescript-bot pack this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 22, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
pack this ✅ Started ✅ Results

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 22, 2024

Hey @jakebailey, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/160700/artifacts?artifactName=tgz&fileId=09532E0BDDB3EF1D0C0185CBFA531EE829D75833400D06C510FCB93118E30D0C02&fileName=/typescript-5.5.0-insiders.20240322.tgz"
    }
}

and then running npm install.


There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/pr-build@5.5.0-pr-57909-8".;

@typescript-bot typescript-bot added For Backlog Bug PRs that fix a backlog bug and removed For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Mar 23, 2024
@Andarist
Copy link
Contributor Author

@jakebailey could you rerun top800, dt and perf suites? a new playground would also be appreciated :)

@jakebailey
Copy link
Member

@typescript-bot test top800
@typescript-bot user test this
@typescript-bot run dt

@typescript-bot perf test this faster
@typescript-bot pack this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 23, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
test top800 ✅ Started ✅ Results
user test this ✅ Started ✅ Results
run dt ✅ Started 👀 Results
perf test this faster ✅ Started 👀 Results
pack this ✅ Started ✅ Results

@typescript-bot
Copy link
Collaborator

typescript-bot commented Mar 23, 2024

Hey @jakebailey, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/160702/artifacts?artifactName=tgz&fileId=C476EBF980531EB5F87264AF4D7EFD709FC10481D708D1A72266850B714544B902&fileName=/typescript-5.5.0-insiders.20240323.tgz"
    }
}

and then running npm install.


There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/pr-build@5.5.0-pr-57909-12".;

@typescript-bot
Copy link
Collaborator

Hey @jakebailey, the results of running the DT tests are ready.

There were interesting changes:

Branch only errors:

Package: styled-theming
Error:

Error: 
/mnt/vss/_work/1/DefinitelyTyped/types/styled-theming/styled-theming-tests.tsx
  72:47  error  TypeScript@local compile error: 
No overload matches this call.
  The last overload gave the following error.
    Type '"wrong variant"' is not assignable to type '"primary" | "secondary" | undefined'  @definitelytyped/expect

✖ 1 problem (1 error, 0 warnings)

    at combineErrorsAndWarnings (/mnt/vss/_work/1/DefinitelyTyped/node_modules/.pnpm/@definitelytyped+dtslint@0.2.18_typescript@5.5.0-dev.20240323/node_modules/@definitelytyped/dtslint/dist/index.js:194:28)
    at runTests (/mnt/vss/_work/1/DefinitelyTyped/node_modules/.pnpm/@definitelytyped+dtslint@0.2.18_typescript@5.5.0-dev.20240323/node_modules/@definitelytyped/dtslint/dist/index.js:186:20)

You can check the log here.

@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the user tests comparing main and refs/pull/57909/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Angular - node (v18.15.0, x64)
Memory used 295,530k (± 0.01%) 295,537k (± 0.01%) ~ 295,512k 295,555k p=1.000 n=6
Parse Time 2.66s (± 0.37%) 2.66s (± 0.28%) ~ 2.65s 2.67s p=0.652 n=6
Bind Time 0.82s (± 0.66%) 0.83s (± 0.49%) ~ 0.83s 0.84s p=0.054 n=6
Check Time 8.20s (± 0.32%) 8.22s (± 0.60%) ~ 8.17s 8.29s p=0.872 n=6
Emit Time 7.07s (± 0.35%) 7.05s (± 0.64%) ~ 7.02s 7.14s p=0.222 n=6
Total Time 18.76s (± 0.19%) 18.76s (± 0.33%) ~ 18.68s 18.84s p=1.000 n=6
Compiler-Unions - node (v18.15.0, x64)
Memory used 192,468k (± 0.73%) 194,259k (± 0.94%) ~ 191,886k 195,602k p=0.093 n=6
Parse Time 1.37s (± 0.97%) 1.36s (± 1.52%) ~ 1.33s 1.39s p=0.250 n=6
Bind Time 0.72s (± 0.00%) 0.72s (± 0.00%) ~ 0.72s 0.72s p=1.000 n=6
Check Time 9.51s (± 0.66%) 9.52s (± 0.57%) ~ 9.43s 9.59s p=1.000 n=6
Emit Time 2.63s (± 0.95%) 2.62s (± 0.96%) ~ 2.58s 2.64s p=0.461 n=6
Total Time 14.23s (± 0.53%) 14.21s (± 0.41%) ~ 14.11s 14.26s p=0.573 n=6
Monaco - node (v18.15.0, x64)
Memory used 347,373k (± 0.01%) 347,383k (± 0.00%) ~ 347,358k 347,402k p=0.575 n=6
Parse Time 2.48s (± 0.33%) 2.48s (± 0.30%) ~ 2.47s 2.49s p=0.729 n=6
Bind Time 0.93s (± 0.88%) 0.93s (± 0.56%) ~ 0.92s 0.93s p=0.929 n=6
Check Time 7.00s (± 0.49%) 7.00s (± 0.38%) ~ 6.97s 7.04s p=0.686 n=6
Emit Time 4.07s (± 0.34%) 4.07s (± 0.26%) ~ 4.05s 4.08s p=1.000 n=6
Total Time 14.48s (± 0.28%) 14.48s (± 0.18%) ~ 14.45s 14.52s p=0.808 n=6
TFS - node (v18.15.0, x64)
Memory used 302,725k (± 0.01%) 302,715k (± 0.02%) ~ 302,659k 302,798k p=0.575 n=6
Parse Time 2.40s (± 1.41%) 2.51s (± 8.80%) ~ 2.39s 2.96s p=0.260 n=6
Bind Time 1.21s (± 0.68%) 1.24s (±10.13%) ~ 1.18s 1.50s p=0.230 n=6
Check Time 7.46s (± 0.50%) 7.75s (± 9.17%) ~ 7.43s 9.20s p=0.574 n=6
Emit Time 4.29s (± 0.54%) 4.45s (± 9.53%) ~ 4.25s 5.31s p=0.571 n=6
Total Time 15.35s (± 0.38%) 15.95s (± 9.29%) ~ 15.32s 18.97s p=0.748 n=6
material-ui - node (v18.15.0, x64)
Memory used 509,900k (± 0.00%) 509,884k (± 0.00%) ~ 509,859k 509,917k p=0.128 n=6
Parse Time 2.66s (± 0.63%) 2.65s (± 0.70%) ~ 2.62s 2.67s p=0.325 n=6
Bind Time 0.99s (± 1.05%) 0.99s (± 0.82%) ~ 0.98s 1.00s p=0.270 n=6
Check Time 17.22s (± 0.44%) 17.23s (± 0.53%) ~ 17.17s 17.41s p=0.936 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 20.86s (± 0.41%) 20.87s (± 0.48%) ~ 20.81s 21.07s p=0.936 n=6
mui-docs - node (v18.15.0, x64)
Memory used 1,737,562k (± 0.00%) 1,737,442k (± 0.00%) -120k (- 0.01%) 1,737,418k 1,737,479k p=0.005 n=6
Parse Time 7.77s (± 0.27%) 7.79s (± 0.86%) ~ 7.73s 7.92s p=0.746 n=6
Bind Time 2.80s (± 0.27%) 2.80s (± 0.42%) ~ 2.79s 2.82s p=0.933 n=6
Check Time 66.67s (± 0.41%) 66.46s (± 0.27%) ~ 66.19s 66.65s p=0.229 n=6
Emit Time 0.15s (± 2.69%) 0.15s (± 3.36%) ~ 0.15s 0.16s p=0.595 n=6
Total Time 77.40s (± 0.34%) 77.21s (± 0.22%) ~ 76.94s 77.36s p=0.173 n=6
self-build-src - node (v18.15.0, x64)
Memory used 2,393,895k (± 0.05%) 2,393,977k (± 0.02%) ~ 2,393,272k 2,394,910k p=0.471 n=6
Parse Time 6.10s (± 0.95%) 6.05s (± 1.13%) ~ 5.96s 6.14s p=0.261 n=6
Bind Time 2.26s (± 0.78%) 2.25s (± 1.21%) ~ 2.20s 2.27s p=1.000 n=6
Check Time 39.45s (± 0.41%) 39.49s (± 0.41%) ~ 39.38s 39.81s p=0.810 n=6
Emit Time 3.10s (± 2.12%) 3.11s (± 1.39%) ~ 3.04s 3.17s p=0.936 n=6
Total Time 50.90s (± 0.36%) 50.93s (± 0.40%) ~ 50.74s 51.30s p=0.936 n=6
self-compiler - node (v18.15.0, x64)
Memory used 415,116k (± 0.01%) 415,122k (± 0.01%) ~ 415,083k 415,153k p=0.936 n=6
Parse Time 4.19s (± 0.63%) 4.20s (± 0.53%) ~ 4.17s 4.23s p=0.864 n=6
Bind Time 1.58s (± 1.36%) 1.60s (± 1.62%) ~ 1.56s 1.63s p=0.421 n=6
Check Time 22.33s (± 0.38%) 22.27s (± 0.47%) ~ 22.16s 22.46s p=0.228 n=6
Emit Time 1.65s (± 0.99%) 1.66s (± 0.62%) ~ 1.65s 1.67s p=0.242 n=6
Total Time 29.76s (± 0.35%) 29.72s (± 0.32%) ~ 29.62s 29.88s p=0.575 n=6
vscode - node (v18.15.0, x64)
Memory used 2,891,049k (± 0.00%) 2,891,279k (± 0.00%) +230k (+ 0.01%) 2,891,226k 2,891,343k p=0.005 n=6
Parse Time 12.93s (± 0.35%) 12.92s (± 0.26%) ~ 12.88s 12.98s p=1.000 n=6
Bind Time 4.13s (± 0.54%) 4.13s (± 0.32%) ~ 4.12s 4.15s p=0.807 n=6
Check Time 71.47s (± 0.29%) 71.67s (± 0.34%) ~ 71.32s 71.99s p=0.230 n=6
Emit Time 19.43s (± 0.95%) 19.41s (± 0.84%) ~ 19.25s 19.64s p=0.872 n=6
Total Time 107.96s (± 0.27%) 108.12s (± 0.24%) ~ 107.85s 108.59s p=0.575 n=6
webpack - node (v18.15.0, x64)
Memory used 408,129k (± 0.01%) 408,163k (± 0.01%) ~ 408,104k 408,215k p=0.173 n=6
Parse Time 3.90s (± 0.70%) 3.89s (± 0.23%) ~ 3.88s 3.90s p=0.934 n=6
Bind Time 1.68s (± 0.58%) 1.67s (± 0.76%) ~ 1.65s 1.69s p=0.116 n=6
Check Time 16.78s (± 0.24%) 16.74s (± 0.20%) ~ 16.69s 16.77s p=0.086 n=6
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) ~ 0.00s 0.00s p=1.000 n=6
Total Time 22.36s (± 0.24%) 22.31s (± 0.19%) ~ 22.25s 22.36s p=0.064 n=6
xstate - node (v18.15.0, x64)
Memory used 513,044k (± 0.02%) 513,043k (± 0.02%) ~ 512,923k 513,149k p=0.936 n=6
Parse Time 3.95s (± 0.55%) 3.94s (± 0.30%) ~ 3.92s 3.95s p=0.747 n=6
Bind Time 1.84s (± 0.95%) 1.85s (± 0.99%) ~ 1.83s 1.87s p=0.508 n=6
Check Time 3.37s (± 0.99%) 3.37s (± 0.55%) ~ 3.35s 3.40s p=0.515 n=6
Emit Time 0.09s (± 6.44%) 0.08s (± 4.99%) ~ 0.08s 0.09s p=0.282 n=6
Total Time 9.24s (± 0.61%) 9.25s (± 0.38%) ~ 9.19s 9.29s p=0.688 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Angular - node (v18.15.0, x64)
  • Compiler-Unions - node (v18.15.0, x64)
  • Monaco - node (v18.15.0, x64)
  • TFS - node (v18.15.0, x64)
  • material-ui - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the top 800 repos comparing main and refs/pull/57909/merge:

Everything looks good!

@Andarist
Copy link
Contributor Author

Andarist commented Mar 23, 2024

The only reported error here is actually desired! See the comment here. I didn't look into this one before because of that and because it just involves multiple libraries with complicated type. I plan to reduce it to a test case now and add it here.

EDIT:// From this initial repro: TS playground to this TS playground. It's pretty lengthy but I already have problems with removing from it just about anything.

@sandersn sandersn added this to Not started in PR Backlog Apr 2, 2024
@jakebailey
Copy link
Member

@typescript-bot test it

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 9, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
test top400 ✅ Started ✅ Results
user test this ✅ Started ✅ Results
run dt ✅ Started 👀 Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

Hey @jakebailey, the results of running the DT tests are ready.

There were interesting changes:

Branch only errors:

Package: styled-theming
Error:

Error: 
/mnt/vss/_work/1/DefinitelyTyped/types/styled-theming/styled-theming-tests.tsx
  72:47  error  TypeScript@local compile error: 
No overload matches this call.
  The last overload gave the following error.
    Type '"wrong variant"' is not assignable to type '"primary" | "secondary" | undefined'  @definitelytyped/expect

✖ 1 problem (1 error, 0 warnings)

    at combineErrorsAndWarnings (/mnt/vss/_work/1/DefinitelyTyped/node_modules/.pnpm/@definitelytyped+dtslint@0.2.20_typescript@5.5.0-dev.20240509/node_modules/@definitelytyped/dtslint/dist/index.js:194:28)
    at runTests (/mnt/vss/_work/1/DefinitelyTyped/node_modules/.pnpm/@definitelytyped+dtslint@0.2.20_typescript@5.5.0-dev.20240509/node_modules/@definitelytyped/dtslint/dist/index.js:186:20)

You can check the log here.

@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the user tests comparing main and refs/pull/57909/merge:

Everything looks good!

@Andarist
Copy link
Contributor Author

Andarist commented May 9, 2024

Still just one error :p and I have commented on it already here

@typescript-bot
Copy link
Collaborator

@jakebailey
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,154 62,154 ~ ~ ~ p=1.000 n=6
Types 50,248 50,248 ~ ~ ~ p=1.000 n=6
Memory used 192,795k (± 0.76%) 192,759k (± 0.78%) ~ 192,109k 195,825k p=0.378 n=6
Parse Time 1.29s (± 1.74%) 1.30s (± 1.16%) ~ 1.28s 1.31s p=0.864 n=6
Bind Time 0.72s 0.72s ~ ~ ~ p=1.000 n=6
Check Time 9.52s (± 0.32%) 9.51s (± 0.19%) ~ 9.49s 9.54s p=0.419 n=6
Emit Time 2.65s (± 0.19%) 2.64s (± 0.31%) -0.01s (- 0.38%) 2.63s 2.65s p=0.050 n=6
Total Time 14.19s (± 0.36%) 14.16s (± 0.13%) ~ 14.13s 14.18s p=0.294 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,110 944,120 +10 (+ 0.00%) ~ ~ p=0.001 n=6
Types 407,141 407,148 +7 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 1,221,999k (± 0.00%) 1,222,003k (± 0.00%) ~ 1,221,955k 1,222,042k p=1.000 n=6
Parse Time 6.80s (± 0.71%) 6.81s (± 0.66%) ~ 6.75s 6.87s p=0.810 n=6
Bind Time 1.88s (± 0.48%) 1.88s (± 0.88%) ~ 1.85s 1.89s p=0.803 n=6
Check Time 31.09s (± 0.34%) 31.15s (± 0.53%) ~ 31.03s 31.47s p=0.630 n=6
Emit Time 14.72s (± 0.66%) 14.78s (± 0.29%) ~ 14.72s 14.84s p=0.377 n=6
Total Time 54.49s (± 0.17%) 54.61s (± 0.34%) ~ 54.42s 54.93s p=0.173 n=6
mui-docs - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 1,961,290 1,961,274 -16 (- 0.00%) ~ ~ p=0.001 n=6
Types 696,905 696,860 -45 (- 0.01%) ~ ~ p=0.001 n=6
Memory used 1,778,074k (± 0.00%) 1,778,019k (± 0.00%) -56k (- 0.00%) 1,778,006k 1,778,038k p=0.005 n=6
Parse Time 6.77s (± 0.38%) 6.78s (± 0.24%) ~ 6.76s 6.81s p=0.332 n=6
Bind Time 2.31s (± 1.11%) 2.31s (± 0.22%) ~ 2.31s 2.32s p=0.928 n=6
Check Time 57.01s (± 0.32%) 56.97s (± 0.34%) ~ 56.74s 57.28s p=0.575 n=6
Emit Time 0.14s 0.14s ~ ~ ~ p=1.000 n=6
Total Time 66.23s (± 0.29%) 66.20s (± 0.28%) ~ 65.97s 66.49s p=0.575 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,220,247 1,220,280 +33 (+ 0.00%) ~ ~ p=0.001 n=6
Types 258,932 258,941 +9 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 2,334,478k (± 0.04%) 2,334,608k (± 0.03%) ~ 2,333,545k 2,335,230k p=0.936 n=6
Parse Time 4.99s (± 1.79%) 4.96s (± 1.57%) ~ 4.85s 5.07s p=0.810 n=6
Bind Time 1.86s (± 0.63%) 1.88s (± 1.00%) ~ 1.86s 1.91s p=0.187 n=6
Check Time 33.48s (± 0.32%) 33.37s (± 0.32%) ~ 33.26s 33.54s p=0.109 n=6
Emit Time 2.61s (± 2.40%) 2.64s (± 1.82%) ~ 2.55s 2.68s p=0.228 n=6
Total Time 42.94s (± 0.46%) 42.86s (± 0.41%) ~ 42.72s 43.20s p=0.630 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,220,247 1,220,280 +33 (+ 0.00%) ~ ~ p=0.001 n=6
Types 258,932 258,941 +9 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 2,410,887k (± 0.02%) 2,411,704k (± 0.03%) +817k (+ 0.03%) 2,410,948k 2,412,348k p=0.045 n=6
Parse Time 7.68s (± 0.84%) 7.73s (± 1.46%) ~ 7.58s 7.91s p=0.378 n=6
Bind Time 2.50s (± 1.87%) 2.50s (± 0.93%) ~ 2.46s 2.53s p=0.575 n=6
Check Time 49.26s (± 0.48%) 49.38s (± 0.34%) ~ 49.07s 49.57s p=0.336 n=6
Emit Time 3.85s (± 3.34%) 3.81s (± 1.33%) ~ 3.74s 3.86s p=0.810 n=6
Total Time 63.28s (± 0.39%) 63.42s (± 0.43%) ~ 62.89s 63.64s p=0.173 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 256,476 256,479 +3 (+ 0.00%) ~ ~ p=0.001 n=6
Types 104,239 104,238 -1 (- 0.00%) ~ ~ p=0.001 n=6
Memory used 425,330k (± 0.00%) 425,345k (± 0.01%) ~ 425,313k 425,384k p=0.471 n=6
Parse Time 3.38s (± 0.16%) 3.35s (± 0.82%) ~ 3.32s 3.39s p=0.250 n=6
Bind Time 1.30s (± 0.90%) 1.31s (± 0.89%) ~ 1.29s 1.32s p=0.185 n=6
Check Time 17.79s (± 0.23%) 17.75s (± 0.58%) ~ 17.56s 17.86s p=0.630 n=6
Emit Time 1.35s (± 0.89%) 1.36s (± 2.18%) ~ 1.33s 1.41s p=1.000 n=6
Total Time 23.82s (± 0.21%) 23.78s (± 0.48%) ~ 23.58s 23.91s p=0.521 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,575 224,575 ~ ~ ~ p=1.000 n=6
Types 93,785 93,785 ~ ~ ~ p=1.000 n=6
Memory used 369,747k (± 0.02%) 369,852k (± 0.03%) ~ 369,702k 370,016k p=0.093 n=6
Parse Time 2.83s (± 0.86%) 2.82s (± 1.47%) ~ 2.77s 2.88s p=0.686 n=6
Bind Time 1.58s (± 0.66%) 1.60s (± 0.94%) ~ 1.57s 1.61s p=0.142 n=6
Check Time 15.57s (± 0.36%) 15.57s (± 0.42%) ~ 15.47s 15.67s p=0.571 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.98s (± 0.41%) 19.99s (± 0.41%) ~ 19.88s 20.13s p=0.687 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,809,653 2,809,890 +237 (+ 0.01%) ~ ~ p=0.001 n=6
Types 953,438 953,461 +23 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 2,981,399k (± 0.00%) 2,981,559k (± 0.00%) ~ 2,981,474k 2,981,687k p=0.054 n=6
Parse Time 13.69s (± 0.41%) 13.72s (± 0.61%) ~ 13.60s 13.83s p=0.575 n=6
Bind Time 4.10s (± 2.02%) 4.14s (± 2.62%) ~ 4.06s 4.28s p=0.570 n=6
Check Time 72.99s (± 2.66%) 72.30s (± 0.56%) ~ 71.85s 73.03s p=0.575 n=6
Emit Time 22.97s (± 7.08%) 23.57s (± 0.88%) ~ 23.37s 23.84s p=0.936 n=6
Total Time 113.75s (± 0.44%) 113.72s (± 0.49%) ~ 113.04s 114.32s p=1.000 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 265,866 265,866 ~ ~ ~ p=1.000 n=6
Types 108,424 108,424 ~ ~ ~ p=1.000 n=6
Memory used 410,525k (± 0.02%) 410,556k (± 0.01%) ~ 410,483k 410,640k p=0.575 n=6
Parse Time 4.75s (± 0.74%) 4.73s (± 0.80%) ~ 4.68s 4.79s p=0.198 n=6
Bind Time 2.07s (± 0.61%) 2.05s (± 1.33%) ~ 2.01s 2.08s p=0.285 n=6
Check Time 20.89s (± 0.18%) 20.91s (± 0.51%) ~ 20.73s 21.03s p=0.471 n=6
Emit Time 0.00s (±244.70%) 0.00s ~ ~ ~ p=0.405 n=6
Total Time 27.72s (± 0.16%) 27.69s (± 0.50%) ~ 27.49s 27.85s p=0.872 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 524,576 524,841 +265 (+ 0.05%) ~ ~ p=0.001 n=6
Types 178,847 178,899 +52 (+ 0.03%) ~ ~ p=0.001 n=6
Memory used 462,531k (± 0.01%) 462,674k (± 0.02%) +144k (+ 0.03%) 462,549k 462,824k p=0.031 n=6
Parse Time 3.89s (± 0.41%) 3.89s (± 0.25%) ~ 3.88s 3.90s p=1.000 n=6
Bind Time 1.45s (± 1.40%) 1.44s (± 1.16%) ~ 1.42s 1.46s p=0.367 n=6
Check Time 22.46s (± 0.63%) 22.41s (± 0.56%) ~ 22.27s 22.57s p=0.521 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 27.80s (± 0.49%) 27.74s (± 0.42%) ~ 27.62s 27.89s p=0.423 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@jakebailey Here are the results of running the top 400 repos comparing main and refs/pull/57909/merge:

Everything looks good!

@jakebailey
Copy link
Member

Indeed, just rechecking since this is old and also we have new perf benchmarks / stats.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
For Backlog Bug PRs that fix a backlog bug
Projects
PR Backlog
  
Not started
3 participants