Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix symbol merging of augmentations to pattern ambient modules #31078

Merged
merged 4 commits into from Apr 30, 2019

Conversation

andrewbranch
Copy link
Member

Fixes #30752

@@ -923,7 +924,9 @@ namespace ts {
if (!target.exports) target.exports = createSymbolTable();
mergeSymbolTable(target.exports, source.exports);
}
recordMergedSymbol(target, source);
if (!unidirectional) {
recordMergedSymbol(target, source);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sandersn you were right that the merged symbols essentially go both ways (target and source don't share a mergeId, but target gets there by cloneSymbol(resolvedTarget) and being mutated; then source gets added here.) Introducing a unidirectonal option to prevent adding a mergeId to source accomplishes your suggestion.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm. I've been burned pretty badly in the past by trying to extend the architecture of symbol merging. Merging is where state touches the checker, which is otherwise stateless:
Symbols from the binder are created when loading a file and only re-created when that file is editted. But merging mutates those symbols. It's important to only do that once, and tricky to make sure that happens as checkers are re-created over and over again and run merge* each time. I'd rather not change the architecture if it's not necessary.

Can you instead investigate copying constituent symbols from mainModule.exports and moduleAugmentation.exports to a fresh symbol created from cloneSymbol(moduleAugmentation) (or maybe it's called createSymbol)? This method is more likely to break symbol sharing, so you'll need to add tests for find-all-refs and rename-symbol to make they work correctly.

The patternAmbientModuleAugmentation map is still probably needed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I follow—moduleAgumentation (the target) isn't transient, so mergeSymbol clones it rather than mutates it, and mainModule (the source) doesn't get mutated because it's the source. I think the only thing getting mutated here is the mergedSymbols array, which is the problem—I need to avoid calling recordMergedSymbol with these two symbols, because you should never be able to get from mainModule to moduleAugmentation via getMergedSymbol.

Copy link
Member

@sandersn sandersn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One big suggestion, and one horrifying test case.

@@ -923,7 +924,9 @@ namespace ts {
if (!target.exports) target.exports = createSymbolTable();
mergeSymbolTable(target.exports, source.exports);
}
recordMergedSymbol(target, source);
if (!unidirectional) {
recordMergedSymbol(target, source);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm. I've been burned pretty badly in the past by trying to extend the architecture of symbol merging. Merging is where state touches the checker, which is otherwise stateless:
Symbols from the binder are created when loading a file and only re-created when that file is editted. But merging mutates those symbols. It's important to only do that once, and tricky to make sure that happens as checkers are re-created over and over again and run merge* each time. I'd rather not change the architecture if it's not necessary.

Can you instead investigate copying constituent symbols from mainModule.exports and moduleAugmentation.exports to a fresh symbol created from cloneSymbol(moduleAugmentation) (or maybe it's called createSymbol)? This method is more likely to break symbol sharing, so you'll need to add tests for find-all-refs and rename-symbol to make they work correctly.

The patternAmbientModuleAugmentation map is still probably needed.


==== tests/cases/conformance/ambient/types.ts (0 errors) ====
declare module "*.foo" {
let everywhere: string;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here's a terrifying test case

declare module "*.foo" {
  export interface OhNo { star: string }
}
declare module "a.foo" {
  export interface OhNo { a: string }
}
import { OhNo } from "b.foo"
declare let ohno: OhNo;
ohno.a // oh no

If we can give an error here, we should. Or we could try to recursively do what this PR does.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😱 OH NO

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

Doing the "unidirectional" thing recursively fixed this immediately. It feels like I'm on the right track, so ping me if you get a chance to elaborate on your concerns about symbol mutation, and hopefully I can address them in a way that continues to work well recursively like this.

@andrewbranch
Copy link
Member Author

@typescript-bot perf test this please and thank you

@typescript-bot
Copy link
Collaborator

typescript-bot commented Apr 24, 2019

Heya @andrewbranch, I've started to run the perf test suite on this PR at 973c3ca. You can monitor the build here. It should now contribute to this PR's status checks.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@andrewbranch
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..31078

Metric master 31078 Delta Best Worst
Angular - node (v6.5.0, x64)
Memory used 341,063k (± 0.43%) 340,173k (± 0.36%) -891k (- 0.26%) 339,331k 343,431k
Parse Time 1.63s (± 0.61%) 1.61s (± 1.48%) -0.02s (- 1.10%) 1.55s 1.66s
Bind Time 1.54s (± 0.97%) 1.58s (± 2.82%) +0.05s (+ 2.99%) 1.51s 1.70s
Check Time 4.86s (± 0.49%) 4.78s (± 0.59%) -0.08s (- 1.61%) 4.72s 4.84s
Emit Time 6.37s (± 1.72%) 6.30s (± 1.54%) -0.07s (- 1.10%) 6.13s 6.50s
Total Time 14.40s (± 0.82%) 14.28s (± 0.80%) -0.12s (- 0.85%) 14.00s 14.49s
Monaco - node (v6.5.0, x64)
Memory used 364,356k (± 0.09%) 364,326k (± 0.09%) -31k (- 0.01%) 364,102k 365,659k
Parse Time 1.27s (± 0.57%) 1.27s (± 0.74%) -0.00s (- 0.08%) 1.25s 1.29s
Bind Time 1.52s (± 0.57%) 1.51s (± 0.56%) -0.01s (- 0.46%) 1.49s 1.53s
Check Time 5.68s (± 2.76%) 5.59s (± 2.47%) -0.09s (- 1.59%) 5.04s 5.71s
Emit Time 3.59s (± 2.96%) 3.53s (± 2.56%) -0.06s (- 1.64%) 3.23s 3.66s
Total Time 12.05s (± 2.10%) 11.90s (± 1.83%) -0.15s (- 1.26%) 11.06s 12.12s
TFS - node (v6.5.0, x64)
Memory used 321,067k (± 0.00%) 321,072k (± 0.01%) +5k (+ 0.00%) 320,928k 321,112k
Parse Time 0.98s (± 0.46%) 0.98s (± 0.35%) -0.00s (- 0.10%) 0.97s 0.98s
Bind Time 1.27s (± 0.97%) 1.27s (± 0.60%) +0.00s (+ 0.24%) 1.26s 1.29s
Check Time 4.36s (± 0.59%) 4.28s (± 0.53%) -0.09s (- 1.97%) 4.24s 4.34s
Emit Time 3.13s (± 0.90%) 3.10s (± 2.01%) -0.03s (- 1.09%) 2.85s 3.15s
Total Time 9.75s (± 0.32%) 9.63s (± 0.80%) -0.11s (- 1.17%) 9.34s 9.74s
Angular - node (v6.5.0, x86)
Memory used 191,016k (± 0.00%) 191,054k (± 0.01%) +38k (+ 0.02%) 191,026k 191,093k
Parse Time 1.47s (± 0.91%) 1.47s (± 0.82%) 0.00s ( 0.00%) 1.45s 1.51s
Bind Time 1.50s (± 0.99%) 1.50s (± 1.15%) +0.00s (+ 0.07%) 1.48s 1.54s
Check Time 4.76s (± 1.18%) 4.69s (± 1.85%) -0.07s (- 1.37%) 4.58s 5.03s
Emit Time 6.23s (± 1.66%) 6.09s (± 1.45%) -0.14s (- 2.30%) 5.94s 6.35s
Total Time 13.95s (± 0.91%) 13.74s (± 1.25%) -0.21s (- 1.48%) 13.53s 14.37s
Monaco - node (v6.5.0, x86)
Memory used 204,356k (± 0.01%) 204,357k (± 0.01%) +1k (+ 0.00%) 204,326k 204,391k
Parse Time 1.26s (± 0.56%) 1.26s (± 0.51%) +0.00s (+ 0.08%) 1.25s 1.28s
Bind Time 1.55s (± 0.68%) 1.55s (± 0.73%) -0.00s (- 0.26%) 1.53s 1.57s
Check Time 4.64s (± 1.59%) 4.58s (± 0.47%) -0.06s (- 1.36%) 4.53s 4.62s
Emit Time 3.13s (± 4.48%) 3.19s (± 0.91%) +0.05s (+ 1.72%) 3.14s 3.25s
Total Time 10.59s (± 1.00%) 10.58s (± 0.46%) -0.01s (- 0.07%) 10.48s 10.70s
TFS - node (v6.5.0, x86)
Memory used 180,330k (± 0.01%) 180,349k (± 0.01%) +19k (+ 0.01%) 180,318k 180,390k
Parse Time 0.99s (± 0.84%) 0.98s (± 0.49%) -0.01s (- 0.91%) 0.97s 0.99s
Bind Time 1.31s (± 0.84%) 1.32s (± 1.11%) +0.00s (+ 0.23%) 1.29s 1.35s
Check Time 3.92s (± 0.38%) 3.85s (± 0.66%) -0.07s (- 1.76%) 3.79s 3.91s
Emit Time 2.56s (± 0.62%) 2.55s (± 0.77%) -0.01s (- 0.47%) 2.51s 2.61s
Total Time 8.78s (± 0.32%) 8.69s (± 0.38%) -0.09s (- 0.99%) 8.62s 8.76s
Angular - node (v8.9.0, x64)
Memory used 330,708k (± 0.02%) 330,762k (± 0.01%) +54k (+ 0.02%) 330,625k 330,830k
Parse Time 1.80s (± 0.58%) 1.79s (± 0.31%) -0.01s (- 0.55%) 1.78s 1.80s
Bind Time 1.36s (± 1.30%) 1.36s (± 0.56%) -0.00s (- 0.22%) 1.34s 1.37s
Check Time 4.64s (± 1.83%) 4.55s (± 1.16%) -0.08s (- 1.83%) 4.49s 4.70s
Emit Time 6.07s (± 2.63%) 6.13s (± 2.54%) +0.07s (+ 1.09%) 5.65s 6.29s
Total Time 13.86s (± 0.85%) 13.83s (± 0.93%) -0.03s (- 0.22%) 13.41s 14.02s
Monaco - node (v8.9.0, x64)
Memory used 358,474k (± 0.02%) 358,446k (± 0.02%) -28k (- 0.01%) 358,327k 358,561k
Parse Time 1.45s (± 0.47%) 1.45s (± 0.46%) +0.00s (+ 0.28%) 1.43s 1.46s
Bind Time 1.52s (± 0.88%) 1.53s (± 1.03%) +0.02s (+ 1.06%) 1.51s 1.58s
Check Time 4.86s (± 1.71%) 4.71s (± 1.43%) -0.15s (- 3.00%) 4.63s 4.97s
Emit Time 3.01s (± 5.55%) 3.25s (± 3.03%) +0.24s (+ 8.05%) 2.86s 3.33s
Total Time 10.83s (± 0.87%) 10.95s (± 0.46%) +0.12s (+ 1.07%) 10.79s 11.01s
TFS - node (v8.9.0, x64)
Memory used 313,833k (± 0.02%) 313,862k (± 0.01%) +29k (+ 0.01%) 313,780k 313,952k
Parse Time 1.15s (± 0.66%) 1.14s (± 0.43%) -0.00s (- 0.17%) 1.13s 1.15s
Bind Time 1.24s (± 1.77%) 1.23s (± 1.01%) -0.00s (- 0.40%) 1.21s 1.27s
Check Time 4.21s (± 1.02%) 4.23s (± 0.37%) +0.02s (+ 0.48%) 4.18s 4.25s
Emit Time 3.12s (± 0.68%) 3.13s (± 0.42%) +0.01s (+ 0.32%) 3.10s 3.16s
Total Time 9.71s (± 0.65%) 9.73s (± 0.27%) +0.03s (+ 0.27%) 9.68s 9.78s
Angular - node (v8.9.0, x86)
Memory used 187,126k (± 0.02%) 187,159k (± 0.02%) +33k (+ 0.02%) 187,088k 187,214k
Parse Time 1.74s (± 0.48%) 1.75s (± 0.79%) +0.00s (+ 0.11%) 1.72s 1.79s
Bind Time 1.52s (± 0.44%) 1.53s (± 0.31%) +0.00s (+ 0.26%) 1.52s 1.54s
Check Time 4.28s (± 0.64%) 4.25s (± 0.66%) -0.03s (- 0.79%) 4.17s 4.30s
Emit Time 5.69s (± 0.80%) 5.64s (± 1.09%) -0.05s (- 0.95%) 5.54s 5.81s
Total Time 13.24s (± 0.36%) 13.16s (± 0.66%) -0.08s (- 0.60%) 12.97s 13.41s
Monaco - node (v8.9.0, x86)
Memory used 199,834k (± 0.02%) 199,841k (± 0.01%) +7k (+ 0.00%) 199,786k 199,886k
Parse Time 1.50s (± 0.53%) 1.51s (± 0.66%) +0.00s (+ 0.33%) 1.49s 1.54s
Bind Time 1.39s (± 0.50%) 1.40s (± 0.63%) +0.01s (+ 0.43%) 1.38s 1.42s
Check Time 4.62s (± 0.47%) 4.55s (± 0.57%) -0.07s (- 1.58%) 4.49s 4.60s
Emit Time 3.09s (± 1.21%) 3.09s (± 0.39%) -0.01s (- 0.19%) 3.06s 3.11s
Total Time 10.61s (± 0.47%) 10.54s (± 0.24%) -0.07s (- 0.66%) 10.48s 10.58s
TFS - node (v8.9.0, x86)
Memory used 175,898k (± 0.02%) 175,933k (± 0.01%) +36k (+ 0.02%) 175,875k 175,997k
Parse Time 1.20s (± 1.02%) 1.21s (± 0.68%) +0.01s (+ 0.67%) 1.19s 1.22s
Bind Time 1.24s (± 0.71%) 1.23s (± 0.90%) -0.00s (- 0.32%) 1.22s 1.27s
Check Time 4.01s (± 0.99%) 4.07s (± 0.36%) +0.06s (+ 1.42%) 4.04s 4.10s
Emit Time 2.74s (± 1.06%) 2.77s (± 1.65%) +0.03s (+ 0.91%) 2.70s 2.89s
Total Time 9.19s (± 0.76%) 9.28s (± 0.47%) +0.09s (+ 0.95%) 9.21s 9.40s
Angular - node (v9.0.0, x64)
Memory used 330,409k (± 0.02%) 330,481k (± 0.02%) +72k (+ 0.02%) 330,334k 330,640k
Parse Time 1.64s (± 0.38%) 1.64s (± 0.27%) -0.00s (- 0.06%) 1.63s 1.65s
Bind Time 1.33s (± 0.56%) 1.34s (± 0.50%) +0.01s (+ 0.75%) 1.33s 1.36s
Check Time 4.32s (± 0.50%) 4.32s (± 0.54%) -0.00s (- 0.07%) 4.27s 4.38s
Emit Time 5.72s (± 1.06%) 5.78s (± 1.53%) +0.07s (+ 1.19%) 5.66s 5.99s
Total Time 13.01s (± 0.52%) 13.08s (± 0.78%) +0.07s (+ 0.55%) 12.92s 13.29s
Monaco - node (v9.0.0, x64)
Memory used 358,185k (± 0.01%) 358,167k (± 0.01%) -17k (- 0.00%) 358,109k 358,246k
Parse Time 1.29s (± 0.36%) 1.30s (± 0.38%) +0.01s (+ 0.47%) 1.29s 1.31s
Bind Time 1.51s (± 0.65%) 1.51s (± 0.48%) -0.00s (- 0.26%) 1.50s 1.53s
Check Time 4.69s (± 0.54%) 4.64s (± 0.41%) -0.05s (- 1.09%) 4.61s 4.68s
Emit Time 3.22s (± 0.59%) 3.22s (± 0.55%) +0.01s (+ 0.25%) 3.19s 3.27s
Total Time 10.71s (± 0.41%) 10.67s (± 0.28%) -0.04s (- 0.35%) 10.61s 10.74s
TFS - node (v9.0.0, x64)
Memory used 313,766k (± 0.01%) 313,755k (± 0.01%) -10k (- 0.00%) 313,695k 313,909k
Parse Time 1.01s (± 0.59%) 1.01s (± 0.44%) -0.00s (- 0.30%) 1.00s 1.02s
Bind Time 1.20s (± 0.74%) 1.21s (± 0.39%) +0.00s (+ 0.25%) 1.20s 1.22s
Check Time 4.25s (± 1.65%) 4.16s (± 1.76%) -0.10s (- 2.28%) 4.04s 4.29s
Emit Time 2.98s (± 3.28%) 3.02s (± 1.96%) +0.04s (+ 1.41%) 2.90s 3.13s
Total Time 9.45s (± 0.36%) 9.39s (± 0.31%) -0.05s (- 0.57%) 9.33s 9.46s
Angular - node (v9.0.0, x86)
Memory used 187,335k (± 0.03%) 187,336k (± 0.02%) +1k (+ 0.00%) 187,250k 187,386k
Parse Time 1.56s (± 0.68%) 1.56s (± 0.38%) +0.00s (+ 0.06%) 1.55s 1.57s
Bind Time 1.51s (± 0.72%) 1.51s (± 0.46%) -0.00s (- 0.07%) 1.50s 1.53s
Check Time 4.04s (± 0.99%) 3.98s (± 0.47%) -0.06s (- 1.44%) 3.92s 4.02s
Emit Time 5.36s (± 0.64%) 5.33s (± 0.45%) -0.03s (- 0.49%) 5.29s 5.38s
Total Time 12.46s (± 0.58%) 12.38s (± 0.24%) -0.08s (- 0.63%) 12.32s 12.47s
Monaco - node (v9.0.0, x86)
Memory used 199,885k (± 0.01%) 199,934k (± 0.02%) +49k (+ 0.02%) 199,877k 200,007k
Parse Time 1.32s (± 0.44%) 1.32s (± 0.49%) -0.00s (- 0.08%) 1.31s 1.34s
Bind Time 1.36s (± 0.56%) 1.37s (± 0.70%) +0.00s (+ 0.15%) 1.35s 1.40s
Check Time 4.46s (± 0.34%) 4.43s (± 0.46%) -0.03s (- 0.65%) 4.37s 4.48s
Emit Time 3.02s (± 0.65%) 3.02s (± 0.88%) -0.00s (- 0.03%) 2.96s 3.07s
Total Time 10.16s (± 0.30%) 10.13s (± 0.26%) -0.03s (- 0.30%) 10.07s 10.19s
TFS - node (v9.0.0, x86)
Memory used 176,010k (± 0.02%) 176,058k (± 0.02%) +48k (+ 0.03%) 175,950k 176,129k
Parse Time 1.03s (± 0.66%) 1.04s (± 0.64%) +0.00s (+ 0.29%) 1.02s 1.05s
Bind Time 1.22s (± 1.13%) 1.22s (± 0.69%) +0.00s (+ 0.25%) 1.20s 1.24s
Check Time 3.90s (± 0.50%) 3.84s (± 0.65%) -0.06s (- 1.49%) 3.77s 3.88s
Emit Time 2.70s (± 0.69%) 2.72s (± 0.97%) +0.02s (+ 0.82%) 2.67s 2.79s
Total Time 8.85s (± 0.43%) 8.81s (± 0.46%) -0.03s (- 0.36%) 8.71s 8.91s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-142-generic
Architecturex64
Available Memory16 GB
Available Memory1 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v6.5.0, x64)
  • node (v6.5.0, x86)
  • node (v8.9.0, x64)
  • node (v8.9.0, x86)
  • node (v9.0.0, x64)
  • node (v9.0.0, x86)
Scenarios
  • Angular - node (v6.5.0, x64)
  • Angular - node (v6.5.0, x86)
  • Angular - node (v8.9.0, x64)
  • Angular - node (v8.9.0, x86)
  • Angular - node (v9.0.0, x64)
  • Angular - node (v9.0.0, x86)
  • Monaco - node (v6.5.0, x64)
  • Monaco - node (v6.5.0, x86)
  • Monaco - node (v8.9.0, x64)
  • Monaco - node (v8.9.0, x86)
  • Monaco - node (v9.0.0, x64)
  • Monaco - node (v9.0.0, x86)
  • TFS - node (v6.5.0, x64)
  • TFS - node (v6.5.0, x86)
  • TFS - node (v8.9.0, x64)
  • TFS - node (v8.9.0, x86)
  • TFS - node (v9.0.0, x64)
  • TFS - node (v9.0.0, x86)
Benchmark Name Iterations
Current 31078 10
Baseline master 10

@andrewbranch
Copy link
Member Author

@sandersn the perf test is complaining about this one:

‌Monaco - node (v8.9.0, x86) ‌ ‌│‌
────────────┬────────────────────┬────────────────────┬──────────────────┬──────────┬──────────
Emit Time   │    3.01s (± 5.55%) │    3.25s (± 3.03%) │ +0.24s (+ 8.05%) │    2.86s │    3.33s

+8% delta but ±5.55% variance measuring master 🤔 I can ignore this, right?

@sandersn
Copy link
Member

Practically, yes, because it's a minor change in a component you didn't touch. @rbuckton put together those stats and can say whether that notation means it's technically statistically significant.

@andrewbranch andrewbranch merged commit 90d3acf into microsoft:master Apr 30, 2019
@andrewbranch andrewbranch deleted the bug/30752 branch April 30, 2019 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Strange merging behavior of module augmentation and pattern ambient module
3 participants