Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix leaked alias symbol independent type params in printback #42211

Closed

Conversation

weswigham
Copy link
Member

@weswigham weswigham commented Jan 4, 2021

When we measure the variance of an alias, an Independent type parameter is an unused type parameter - for whatever reason, it has no effect on the comparability of the resulting type. What this often means is that that type parameter also does not play into the construction of the the target type at all. That, in turn, means that due to interning, you can have multiple seemingly unique references to a type, which all point at a single underlying type - usually the one first manufactured within the type alias declaration itself. That underlying type, in turn, has an alias from when it was first created, and that alias's type arguments may contain type parameters only available within the scope of the original alias declaration. Normally, this would disqualify use of the alias as a reference external to that declaration, but we failed to detect such a case as our assumption is that if we are doing printback using a type parameter, that it will be made available in scope (otherwise, how did we get it?). Since this type parameter comes from far afield, we can't safely do printback using it, however, in this fix, rather than issuing an error in such a circumstance (which would be unfortunate and hard to fix for users), I detect when this situation arises by checking the variance and visibility of the alias symbol type arguments, and substitute an unreachable Independent type argument with unknown, as it will not affect the resulting type's shape, thus creating a functioning declaration file.

This approach required checking the variance of some type aliases where we previously had not bothered (usually simply because the test in question had not been written to require it), and in turn exposed a crash-y problem we had calculating variance while handling recursive references to unions and intersections within type aliases (and recursive normalization thereof - see the types in recursivelyExpandingUnionNoStackoverflow as an example of a type which we crashed when fetching the variance on it). To fix that, #37910 (which just got merged into master) mostly handles it, however the size limit it originally imposed has to be removed, as we need the deeply nested type guarding part of recursiveTypeRelatedTo even for the smallest unions, as recursive type references can create situations where these can infinitely expand upon normalization. In addition, I've added an identity for unions and intersections with aliases into getRecursionIdentity to improve how quickly we recognize these structures as deeply nested (the time it took to calculate variances in recursiveTypeReferences2 proved that needed).

TL;DR: We now print

declare const nested: PublicWrap<unknown, {}>;

rather than the erroneous

declare const nested: PublicWrap<T, {}>;

where PublicWrap's first type parameter is Independent.

Fixes #42079
Fixes #44727

…ards and introducing better deep nesting heuristic for unions and intersections
@typescript-bot typescript-bot added Author: Team For Milestone Bug PRs that fix a bug with a specific milestone labels Jan 4, 2021
@weswigham
Copy link
Member Author

@typescript-bot perf test this - since this affects caches, this should cause some change to perf, but I can't predict if it'll be good or bad in real world scenarios; the caching is noncritial to fixing this, but making the extra caching conditional would complicate this change a bit, so I'd like to avoid it if possible.

Also:
@typescript-bot test this
@typescript-bot run dt
@typescript-bot user test this
since this affects Independent type parameter printback, this could change some baselines which may capture our current behavior. I've minimized how often it occurs, so hopefully there's not many changes beyond some duplicate elaborations going away from the expanded caching.

And @typescript-bot pack this for good measure.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 4, 2021

Heya @weswigham, I've started to run the extended test suite on this PR at 3fe794d. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 4, 2021

Heya @weswigham, I've started to run the tarball bundle task on this PR at 3fe794d. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 4, 2021

Heya @weswigham, I've started to run the perf test suite on this PR at 3fe794d. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 4, 2021

Heya @weswigham, I've started to run the parallelized community code test suite on this PR at 3fe794d. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 4, 2021

Heya @weswigham, I've started to run the parallelized Definitely Typed test suite on this PR at 3fe794d. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 4, 2021

Hey @weswigham, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/92595/artifacts?artifactName=tgz&fileId=22D269592FF948AC1C76C42D4B4EE6F22175A778874614673A41A2F1203482CC02&fileName=/typescript-4.2.0-insiders.20210104.tgz"
    }
}

and then running npm install.


There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/pr-build@4.2.0-pr-42211-6".;

@typescript-bot
Copy link
Collaborator

The user suite test run you requested has finished and failed. I've opened a PR with the baseline diff from master.

@weswigham
Copy link
Member Author

The user baselines all look great excepting a few new excessive stack warnings when calculating variance for some types in ts-toolbelt. I'm wondering if those errors could also have been triggered prior to this PR with some slight modifications to force variance analysis of the same types. In any case, I'll look at it.

@typescript-bot
Copy link
Collaborator

@weswigham
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..42211

Metric master 42211 Delta Best Worst
Angular - node (v10.16.3, x64)
Memory used 345,625k (± 0.02%) 346,033k (± 0.03%) +408k (+ 0.12%) 345,726k 346,222k
Parse Time 1.97s (± 0.48%) 1.98s (± 0.38%) +0.01s (+ 0.51%) 1.97s 2.00s
Bind Time 0.82s (± 0.70%) 0.83s (± 1.08%) +0.00s (+ 0.36%) 0.81s 0.84s
Check Time 4.99s (± 0.40%) 4.98s (± 0.39%) -0.01s (- 0.18%) 4.93s 5.02s
Emit Time 5.34s (± 0.83%) 5.33s (± 0.47%) -0.01s (- 0.09%) 5.28s 5.38s
Total Time 13.13s (± 0.43%) 13.12s (± 0.35%) -0.00s (- 0.02%) 13.01s 13.20s
Compiler-Unions - node (v10.16.3, x64)
Memory used 205,784k (± 0.07%) 214,605k (± 0.03%) +8,822k (+ 4.29%) 214,532k 214,814k
Parse Time 0.79s (± 0.84%) 0.79s (± 0.98%) +0.00s (+ 0.13%) 0.77s 0.81s
Bind Time 0.51s (± 1.64%) 0.51s (± 1.72%) +0.00s (+ 0.99%) 0.49s 0.53s
Check Time 12.17s (± 0.82%) 10.61s (± 0.85%) 🟩-1.56s (-12.82%) 10.45s 10.93s
Emit Time 2.32s (± 1.08%) 2.36s (± 0.91%) +0.04s (+ 1.59%) 2.28s 2.38s
Total Time 15.79s (± 0.71%) 14.28s (± 0.64%) 🟩-1.51s (- 9.58%) 14.14s 14.58s
Monaco - node (v10.16.3, x64)
Memory used 354,937k (± 0.02%) 355,176k (± 0.01%) +239k (+ 0.07%) 355,104k 355,242k
Parse Time 1.60s (± 0.50%) 1.60s (± 0.56%) +0.01s (+ 0.44%) 1.59s 1.62s
Bind Time 0.73s (± 0.96%) 0.73s (± 0.89%) -0.00s (- 0.41%) 0.71s 0.74s
Check Time 5.11s (± 0.50%) 5.13s (± 0.51%) +0.02s (+ 0.35%) 5.08s 5.19s
Emit Time 2.82s (± 0.83%) 2.80s (± 0.71%) -0.02s (- 0.53%) 2.75s 2.84s
Total Time 10.26s (± 0.47%) 10.27s (± 0.31%) +0.01s (+ 0.07%) 10.16s 10.30s
TFS - node (v10.16.3, x64)
Memory used 307,985k (± 0.02%) 307,991k (± 0.02%) +6k (+ 0.00%) 307,866k 308,126k
Parse Time 1.23s (± 0.81%) 1.24s (± 0.66%) +0.01s (+ 0.57%) 1.22s 1.26s
Bind Time 0.68s (± 1.03%) 0.68s (± 0.87%) +0.01s (+ 0.88%) 0.67s 0.70s
Check Time 4.57s (± 0.49%) 4.60s (± 0.43%) +0.03s (+ 0.61%) 4.56s 4.66s
Emit Time 2.92s (± 0.76%) 2.96s (± 0.54%) +0.04s (+ 1.27%) 2.92s 3.00s
Total Time 9.40s (± 0.35%) 9.48s (± 0.37%) +0.08s (+ 0.85%) 9.40s 9.58s
material-ui - node (v10.16.3, x64)
Memory used 490,253k (± 0.02%) 496,883k (± 0.01%) +6,630k (+ 1.35%) 496,744k 497,089k
Parse Time 2.07s (± 0.83%) 2.08s (± 0.72%) +0.01s (+ 0.34%) 2.05s 2.12s
Bind Time 0.65s (± 1.08%) 0.65s (± 1.25%) -0.00s (- 0.15%) 0.64s 0.68s
Check Time 13.64s (± 0.62%) 13.67s (± 0.66%) +0.03s (+ 0.21%) 13.47s 13.85s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 16.36s (± 0.53%) 16.39s (± 0.54%) +0.04s (+ 0.21%) 16.20s 16.56s
Angular - node (v12.1.0, x64)
Memory used 323,138k (± 0.02%) 323,535k (± 0.03%) +397k (+ 0.12%) 323,353k 323,787k
Parse Time 1.97s (± 0.56%) 1.97s (± 0.59%) -0.00s (- 0.10%) 1.94s 2.00s
Bind Time 0.81s (± 0.55%) 0.81s (± 0.72%) -0.00s (- 0.12%) 0.80s 0.82s
Check Time 4.89s (± 1.14%) 4.88s (± 0.43%) -0.01s (- 0.25%) 4.84s 4.92s
Emit Time 5.50s (± 0.76%) 5.50s (± 0.64%) +0.00s (+ 0.02%) 5.46s 5.63s
Total Time 13.18s (± 0.41%) 13.16s (± 0.38%) -0.01s (- 0.11%) 13.07s 13.31s
Compiler-Unions - node (v12.1.0, x64)
Memory used 191,898k (± 0.07%) 199,595k (± 0.09%) +7,697k (+ 4.01%) 199,043k 199,831k
Parse Time 0.78s (± 1.23%) 0.77s (± 0.75%) -0.00s (- 0.26%) 0.76s 0.78s
Bind Time 0.50s (± 0.96%) 0.50s (± 0.73%) -0.00s (- 0.20%) 0.49s 0.50s
Check Time 10.88s (± 0.63%) 9.64s (± 0.83%) 🟩-1.23s (-11.35%) 9.51s 9.89s
Emit Time 2.36s (± 1.10%) 2.38s (± 1.20%) +0.02s (+ 0.81%) 2.34s 2.45s
Total Time 14.51s (± 0.53%) 13.29s (± 0.59%) 🟩-1.22s (- 8.40%) 13.14s 13.52s
Monaco - node (v12.1.0, x64)
Memory used 337,042k (± 0.02%) 337,288k (± 0.03%) +246k (+ 0.07%) 337,106k 337,690k
Parse Time 1.58s (± 0.73%) 1.58s (± 0.42%) -0.00s (- 0.13%) 1.57s 1.60s
Bind Time 0.71s (± 0.94%) 0.71s (± 1.33%) +0.00s (+ 0.56%) 0.70s 0.75s
Check Time 4.92s (± 0.55%) 4.93s (± 0.54%) +0.01s (+ 0.16%) 4.87s 4.99s
Emit Time 2.88s (± 0.93%) 2.88s (± 0.69%) +0.00s (+ 0.07%) 2.84s 2.92s
Total Time 10.09s (± 0.48%) 10.10s (± 0.43%) +0.01s (+ 0.09%) 10.02s 10.19s
TFS - node (v12.1.0, x64)
Memory used 292,222k (± 0.01%) 292,263k (± 0.04%) +41k (+ 0.01%) 292,094k 292,644k
Parse Time 1.25s (± 0.72%) 1.25s (± 0.54%) +0.00s (+ 0.24%) 1.24s 1.27s
Bind Time 0.66s (± 1.05%) 0.65s (± 1.04%) -0.00s (- 0.15%) 0.65s 0.68s
Check Time 4.51s (± 0.74%) 4.54s (± 0.57%) +0.03s (+ 0.67%) 4.47s 4.58s
Emit Time 2.96s (± 0.61%) 2.98s (± 1.79%) +0.02s (+ 0.71%) 2.90s 3.17s
Total Time 9.38s (± 0.56%) 9.43s (± 0.64%) +0.05s (+ 0.54%) 9.26s 9.58s
material-ui - node (v12.1.0, x64)
Memory used 468,035k (± 0.06%) 473,334k (± 0.08%) +5,299k (+ 1.13%) 472,439k 473,771k
Parse Time 2.07s (± 0.68%) 2.09s (± 0.64%) +0.01s (+ 0.53%) 2.05s 2.11s
Bind Time 0.64s (± 0.96%) 0.64s (± 0.73%) -0.00s (- 0.16%) 0.63s 0.65s
Check Time 12.14s (± 0.78%) 12.20s (± 0.52%) +0.06s (+ 0.49%) 12.05s 12.33s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 14.86s (± 0.69%) 14.93s (± 0.39%) +0.07s (+ 0.46%) 14.80s 15.04s
Angular - node (v8.9.0, x64)
Memory used 348,215k (± 0.02%) 348,758k (± 0.02%) +543k (+ 0.16%) 348,619k 348,917k
Parse Time 2.51s (± 0.39%) 2.52s (± 0.40%) +0.01s (+ 0.20%) 2.50s 2.54s
Bind Time 0.86s (± 0.87%) 0.86s (± 1.00%) +0.00s (+ 0.58%) 0.84s 0.88s
Check Time 5.59s (± 0.54%) 5.60s (± 0.70%) +0.01s (+ 0.21%) 5.50s 5.68s
Emit Time 6.32s (± 1.31%) 6.42s (± 1.31%) +0.10s (+ 1.57%) 6.14s 6.61s
Total Time 15.28s (± 0.48%) 15.39s (± 0.64%) +0.11s (+ 0.75%) 15.17s 15.62s
Compiler-Unions - node (v8.9.0, x64)
Memory used 213,568k (± 0.02%) 222,392k (± 0.01%) +8,824k (+ 4.13%) 222,308k 222,459k
Parse Time 0.95s (± 0.42%) 0.96s (± 0.77%) +0.01s (+ 1.16%) 0.94s 0.97s
Bind Time 0.57s (± 0.83%) 0.57s (± 1.32%) +0.00s (+ 0.17%) 0.56s 0.59s
Check Time 15.05s (± 0.71%) 13.17s (± 0.98%) 🟩-1.88s (-12.48%) 12.94s 13.51s
Emit Time 2.77s (± 1.77%) 2.86s (± 0.69%) +0.09s (+ 3.18%) 2.82s 2.91s
Total Time 19.34s (± 0.55%) 17.56s (± 0.76%) 🟩-1.78s (- 9.22%) 17.33s 17.94s
Monaco - node (v8.9.0, x64)
Memory used 358,849k (± 0.01%) 359,132k (± 0.02%) +283k (+ 0.08%) 359,036k 359,303k
Parse Time 1.93s (± 0.43%) 1.93s (± 0.32%) +0.00s (+ 0.16%) 1.91s 1.94s
Bind Time 0.91s (± 0.57%) 0.91s (± 0.61%) +0.00s (+ 0.11%) 0.90s 0.92s
Check Time 5.65s (± 0.51%) 5.68s (± 0.70%) +0.02s (+ 0.41%) 5.59s 5.78s
Emit Time 3.43s (± 0.48%) 3.42s (± 0.64%) -0.00s (- 0.03%) 3.38s 3.49s
Total Time 11.92s (± 0.37%) 11.95s (± 0.40%) +0.03s (+ 0.27%) 11.87s 12.11s
TFS - node (v8.9.0, x64)
Memory used 310,595k (± 0.02%) 310,632k (± 0.02%) +37k (+ 0.01%) 310,559k 310,853k
Parse Time 1.57s (± 0.48%) 1.57s (± 0.49%) -0.01s (- 0.32%) 1.55s 1.59s
Bind Time 0.69s (± 0.71%) 0.69s (± 0.65%) -0.01s (- 0.86%) 0.68s 0.70s
Check Time 5.37s (± 0.57%) 5.37s (± 0.78%) +0.01s (+ 0.11%) 5.28s 5.48s
Emit Time 2.97s (± 0.96%) 2.99s (± 0.70%) +0.02s (+ 0.61%) 2.95s 3.03s
Total Time 10.60s (± 0.49%) 10.62s (± 0.63%) +0.02s (+ 0.14%) 10.47s 10.78s
material-ui - node (v8.9.0, x64)
Memory used 497,585k (± 0.01%) 504,031k (± 0.01%) +6,446k (+ 1.30%) 503,930k 504,153k
Parse Time 2.50s (± 0.38%) 2.50s (± 0.59%) +0.00s (+ 0.12%) 2.47s 2.53s
Bind Time 0.81s (± 1.71%) 0.81s (± 0.82%) +0.00s (+ 0.62%) 0.80s 0.82s
Check Time 18.32s (± 0.56%) 18.06s (± 0.82%) -0.26s (- 1.41%) 17.76s 18.28s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 21.62s (± 0.53%) 21.37s (± 0.73%) -0.25s (- 1.17%) 21.03s 21.62s
Angular - node (v8.9.0, x86)
Memory used 199,903k (± 0.02%) 200,200k (± 0.02%) +297k (+ 0.15%) 200,056k 200,297k
Parse Time 2.43s (± 0.63%) 2.45s (± 0.69%) +0.01s (+ 0.53%) 2.41s 2.48s
Bind Time 1.01s (± 1.11%) 1.01s (± 0.59%) +0.00s (+ 0.10%) 0.99s 1.02s
Check Time 5.02s (± 0.52%) 5.04s (± 0.27%) +0.02s (+ 0.38%) 5.02s 5.08s
Emit Time 6.11s (± 0.84%) 6.11s (± 0.90%) +0.00s (+ 0.07%) 6.00s 6.24s
Total Time 14.57s (± 0.44%) 14.61s (± 0.40%) +0.04s (+ 0.24%) 14.49s 14.73s
Compiler-Unions - node (v8.9.0, x86)
Memory used 128,592k (± 0.03%) 133,662k (± 0.02%) +5,070k (+ 3.94%) 133,617k 133,709k
Parse Time 0.97s (± 0.78%) 0.96s (± 0.60%) -0.01s (- 0.62%) 0.95s 0.98s
Bind Time 0.50s (± 1.15%) 0.50s (± 1.12%) -0.00s (- 0.40%) 0.48s 0.51s
Check Time 13.98s (± 0.73%) 12.28s (± 0.36%) 🟩-1.70s (-12.17%) 12.17s 12.36s
Emit Time 2.63s (± 2.09%) 2.68s (± 1.25%) +0.05s (+ 1.75%) 2.59s 2.74s
Total Time 18.07s (± 0.65%) 16.41s (± 0.33%) 🟩-1.67s (- 9.21%) 16.33s 16.56s
Monaco - node (v8.9.0, x86)
Memory used 203,321k (± 0.02%) 203,500k (± 0.01%) +179k (+ 0.09%) 203,445k 203,563k
Parse Time 1.98s (± 1.16%) 1.98s (± 0.60%) -0.00s (- 0.10%) 1.95s 2.00s
Bind Time 0.72s (± 1.21%) 0.72s (± 0.86%) +0.00s (+ 0.14%) 0.71s 0.74s
Check Time 5.75s (± 1.81%) 5.61s (± 1.91%) -0.13s (- 2.35%) 5.47s 5.88s
Emit Time 2.86s (± 3.96%) 3.03s (± 3.47%) +0.17s (+ 5.77%) 2.76s 3.16s
Total Time 11.31s (± 0.57%) 11.33s (± 0.37%) +0.03s (+ 0.25%) 11.24s 11.43s
TFS - node (v8.9.0, x86)
Memory used 177,703k (± 0.02%) 177,737k (± 0.03%) +34k (+ 0.02%) 177,647k 177,868k
Parse Time 1.60s (± 1.11%) 1.61s (± 1.23%) +0.01s (+ 0.62%) 1.58s 1.66s
Bind Time 0.65s (± 0.75%) 0.65s (± 0.69%) -0.01s (- 0.92%) 0.64s 0.66s
Check Time 4.87s (± 0.38%) 4.91s (± 0.77%) +0.04s (+ 0.76%) 4.84s 5.02s
Emit Time 2.84s (± 1.57%) 2.86s (± 1.25%) +0.02s (+ 0.67%) 2.80s 2.97s
Total Time 9.97s (± 0.60%) 10.03s (± 0.79%) +0.06s (+ 0.60%) 9.91s 10.24s
material-ui - node (v8.9.0, x86)
Memory used 280,370k (± 0.01%) 283,874k (± 0.01%) +3,504k (+ 1.25%) 283,798k 283,928k
Parse Time 2.56s (± 0.93%) 2.57s (± 0.94%) +0.00s (+ 0.16%) 2.53s 2.62s
Bind Time 0.71s (± 5.33%) 0.76s (± 7.03%) +0.05s (+ 6.62%) 0.68s 0.89s
Check Time 16.62s (± 0.60%) 16.66s (± 0.93%) +0.04s (+ 0.26%) 16.51s 17.24s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 19.89s (± 0.41%) 19.99s (± 1.02%) +0.09s (+ 0.47%) 19.77s 20.75s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-197-generic
Architecturex64
Available Memory16 GB
Available Memory9 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v8.9.0, x64)
  • node (v8.9.0, x86)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v8.9.0, x64)
  • Angular - node (v8.9.0, x86)
  • Compiler-Unions - node (v10.16.3, x64)
  • Compiler-Unions - node (v12.1.0, x64)
  • Compiler-Unions - node (v8.9.0, x64)
  • Compiler-Unions - node (v8.9.0, x86)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v8.9.0, x64)
  • Monaco - node (v8.9.0, x86)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v8.9.0, x64)
  • TFS - node (v8.9.0, x86)
  • material-ui - node (v10.16.3, x64)
  • material-ui - node (v12.1.0, x64)
  • material-ui - node (v8.9.0, x64)
  • material-ui - node (v8.9.0, x86)
Benchmark Name Iterations
Current 42211 10
Baseline master 10

@weswigham
Copy link
Member Author

With #37910 merged, this should no longer have an outsized performance benefit (since it did the same thing with respect to expanding caching). I'll ask @typescript-bot perf test this again so we can see if removing the size limit really does make perf worse. If it does, I'll need to rejigger it to still do recursive checks but skip the cache for small unions; but I'd like to avoid that complexity if at all possible.

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 5, 2021

Heya @weswigham, I've started to run the perf test suite on this PR at 7879343. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@weswigham
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..42211

Metric master 42211 Delta Best Worst
Angular - node (v10.16.3, x64)
Memory used 345,729k (± 0.02%) 345,951k (± 0.02%) +222k (+ 0.06%) 345,759k 346,062k
Parse Time 1.99s (± 0.53%) 1.98s (± 0.86%) -0.01s (- 0.50%) 1.95s 2.02s
Bind Time 0.82s (± 0.85%) 0.82s (± 0.60%) -0.00s (- 0.61%) 0.81s 0.83s
Check Time 5.01s (± 0.63%) 4.99s (± 0.46%) -0.03s (- 0.50%) 4.92s 5.02s
Emit Time 5.39s (± 0.57%) 5.34s (± 0.76%) -0.05s (- 0.95%) 5.25s 5.41s
Total Time 13.21s (± 0.32%) 13.12s (± 0.45%) -0.09s (- 0.71%) 13.00s 13.25s
Compiler-Unions - node (v10.16.3, x64)
Memory used 213,079k (± 0.08%) 214,850k (± 0.06%) +1,771k (+ 0.83%) 214,323k 214,976k
Parse Time 0.79s (± 1.03%) 0.79s (± 0.89%) -0.00s (- 0.50%) 0.78s 0.81s
Bind Time 0.50s (± 1.40%) 0.50s (± 1.34%) -0.00s (- 0.80%) 0.48s 0.51s
Check Time 10.42s (± 1.12%) 10.27s (± 0.73%) -0.14s (- 1.36%) 10.08s 10.44s
Emit Time 2.38s (± 1.00%) 2.36s (± 0.83%) -0.02s (- 0.84%) 2.33s 2.42s
Total Time 14.09s (± 0.79%) 13.92s (± 0.63%) -0.17s (- 1.20%) 13.70s 14.12s
Monaco - node (v10.16.3, x64)
Memory used 355,012k (± 0.02%) 355,142k (± 0.03%) +130k (+ 0.04%) 354,896k 355,424k
Parse Time 1.60s (± 0.54%) 1.60s (± 0.66%) +0.00s (+ 0.12%) 1.58s 1.63s
Bind Time 0.73s (± 0.61%) 0.73s (± 1.00%) 0.00s ( 0.00%) 0.72s 0.75s
Check Time 5.15s (± 0.57%) 5.12s (± 0.61%) -0.03s (- 0.62%) 5.05s 5.21s
Emit Time 2.81s (± 0.68%) 2.80s (± 0.60%) -0.01s (- 0.39%) 2.76s 2.83s
Total Time 10.30s (± 0.44%) 10.25s (± 0.49%) -0.05s (- 0.44%) 10.13s 10.36s
TFS - node (v10.16.3, x64)
Memory used 308,022k (± 0.03%) 308,080k (± 0.02%) +58k (+ 0.02%) 308,012k 308,273k
Parse Time 1.24s (± 0.66%) 1.24s (± 0.55%) -0.00s (- 0.08%) 1.22s 1.25s
Bind Time 0.69s (± 0.65%) 0.68s (± 0.95%) -0.01s (- 0.87%) 0.67s 0.70s
Check Time 4.60s (± 0.49%) 4.61s (± 0.50%) +0.01s (+ 0.20%) 4.55s 4.65s
Emit Time 2.95s (± 1.11%) 2.95s (± 0.58%) -0.00s (- 0.03%) 2.91s 2.99s
Total Time 9.47s (± 0.47%) 9.47s (± 0.41%) +0.00s (+ 0.01%) 9.40s 9.56s
material-ui - node (v10.16.3, x64)
Memory used 493,456k (± 0.01%) 496,422k (± 0.01%) +2,967k (+ 0.60%) 496,289k 496,548k
Parse Time 2.08s (± 0.49%) 2.07s (± 0.53%) -0.01s (- 0.39%) 2.04s 2.09s
Bind Time 0.65s (± 1.16%) 0.64s (± 0.76%) -0.01s (- 1.53%) 0.64s 0.66s
Check Time 13.64s (± 0.90%) 13.64s (± 0.58%) -0.01s (- 0.05%) 13.47s 13.86s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 16.38s (± 0.75%) 16.35s (± 0.51%) -0.03s (- 0.18%) 16.15s 16.57s
Angular - node (v12.1.0, x64)
Memory used 323,253k (± 0.11%) 323,428k (± 0.03%) +175k (+ 0.05%) 323,250k 323,748k
Parse Time 1.98s (± 0.73%) 1.98s (± 1.01%) -0.01s (- 0.30%) 1.95s 2.04s
Bind Time 0.82s (± 0.89%) 0.81s (± 0.69%) -0.01s (- 1.22%) 0.80s 0.82s
Check Time 4.88s (± 0.51%) 4.90s (± 0.45%) +0.01s (+ 0.23%) 4.85s 4.95s
Emit Time 5.49s (± 0.61%) 5.52s (± 0.54%) +0.03s (+ 0.47%) 5.47s 5.62s
Total Time 13.18s (± 0.26%) 13.20s (± 0.24%) +0.02s (+ 0.15%) 13.13s 13.26s
Compiler-Unions - node (v12.1.0, x64)
Memory used 198,607k (± 0.05%) 199,940k (± 0.06%) +1,333k (+ 0.67%) 199,510k 200,161k
Parse Time 0.78s (± 1.13%) 0.78s (± 0.99%) +0.00s (+ 0.13%) 0.76s 0.80s
Bind Time 0.50s (± 0.94%) 0.50s (± 0.94%) 0.00s ( 0.00%) 0.49s 0.51s
Check Time 9.56s (± 1.05%) 9.46s (± 0.84%) -0.10s (- 1.04%) 9.33s 9.66s
Emit Time 2.35s (± 1.32%) 2.37s (± 1.25%) +0.02s (+ 0.77%) 2.27s 2.41s
Total Time 13.19s (± 0.92%) 13.11s (± 0.80%) -0.08s (- 0.58%) 12.90s 13.37s
Monaco - node (v12.1.0, x64)
Memory used 337,115k (± 0.02%) 337,213k (± 0.01%) +98k (+ 0.03%) 337,114k 337,316k
Parse Time 1.59s (± 0.36%) 1.58s (± 0.92%) -0.01s (- 0.38%) 1.55s 1.61s
Bind Time 0.71s (± 1.36%) 0.70s (± 0.53%) -0.01s (- 0.84%) 0.70s 0.71s
Check Time 4.92s (± 0.36%) 4.90s (± 0.49%) -0.02s (- 0.35%) 4.86s 4.98s
Emit Time 2.87s (± 0.78%) 2.88s (± 0.85%) +0.01s (+ 0.21%) 2.82s 2.95s
Total Time 10.09s (± 0.38%) 10.07s (± 0.43%) -0.02s (- 0.21%) 10.00s 10.20s
TFS - node (v12.1.0, x64)
Memory used 292,215k (± 0.01%) 292,351k (± 0.02%) +137k (+ 0.05%) 292,203k 292,476k
Parse Time 1.25s (± 0.80%) 1.26s (± 0.46%) +0.00s (+ 0.24%) 1.24s 1.27s
Bind Time 0.66s (± 1.33%) 0.65s (± 1.02%) -0.01s (- 1.06%) 0.65s 0.68s
Check Time 4.54s (± 0.23%) 4.52s (± 0.68%) -0.02s (- 0.46%) 4.46s 4.60s
Emit Time 2.96s (± 0.84%) 2.94s (± 0.98%) -0.02s (- 0.54%) 2.88s 3.02s
Total Time 9.41s (± 0.34%) 9.37s (± 0.55%) -0.04s (- 0.47%) 9.29s 9.53s
material-ui - node (v12.1.0, x64)
Memory used 470,657k (± 0.01%) 473,268k (± 0.02%) +2,612k (+ 0.55%) 473,165k 473,511k
Parse Time 2.08s (± 0.86%) 2.07s (± 0.43%) -0.00s (- 0.19%) 2.05s 2.10s
Bind Time 0.64s (± 0.47%) 0.64s (± 0.70%) +0.00s (+ 0.00%) 0.63s 0.65s
Check Time 12.15s (± 0.82%) 12.06s (± 0.81%) -0.09s (- 0.72%) 11.88s 12.36s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 14.87s (± 0.79%) 14.78s (± 0.69%) -0.09s (- 0.63%) 14.56s 15.07s
Angular - node (v8.9.0, x64)
Memory used 348,421k (± 0.01%) 348,628k (± 0.02%) +207k (+ 0.06%) 348,438k 348,760k
Parse Time 2.51s (± 0.45%) 2.51s (± 0.19%) -0.00s (- 0.16%) 2.50s 2.52s
Bind Time 0.86s (± 0.42%) 0.86s (± 1.06%) -0.01s (- 0.58%) 0.84s 0.88s
Check Time 5.63s (± 0.62%) 5.62s (± 0.64%) -0.01s (- 0.27%) 5.54s 5.67s
Emit Time 6.36s (± 0.96%) 6.36s (± 1.31%) -0.01s (- 0.11%) 6.18s 6.55s
Total Time 15.37s (± 0.50%) 15.34s (± 0.61%) -0.03s (- 0.19%) 15.15s 15.58s
Compiler-Unions - node (v8.9.0, x64)
Memory used 220,978k (± 0.02%) 222,753k (± 0.02%) +1,775k (+ 0.80%) 222,677k 222,863k
Parse Time 0.96s (± 0.88%) 0.95s (± 0.70%) -0.01s (- 0.63%) 0.94s 0.97s
Bind Time 0.58s (± 1.44%) 0.57s (± 0.63%) -0.00s (- 0.35%) 0.57s 0.58s
Check Time 13.15s (± 1.01%) 12.89s (± 0.70%) -0.26s (- 1.96%) 12.77s 13.14s
Emit Time 2.82s (± 0.62%) 2.73s (± 2.11%) -0.08s (- 2.95%) 2.60s 2.81s
Total Time 17.50s (± 0.82%) 17.15s (± 0.74%) -0.35s (- 1.98%) 16.91s 17.44s
Monaco - node (v8.9.0, x64)
Memory used 358,943k (± 0.02%) 359,061k (± 0.02%) +118k (+ 0.03%) 358,969k 359,236k
Parse Time 1.93s (± 0.25%) 1.92s (± 0.35%) -0.01s (- 0.67%) 1.90s 1.93s
Bind Time 0.91s (± 0.49%) 0.91s (± 0.52%) -0.00s (- 0.11%) 0.90s 0.92s
Check Time 5.71s (± 0.56%) 5.72s (± 0.50%) +0.01s (+ 0.21%) 5.67s 5.79s
Emit Time 3.42s (± 0.32%) 3.43s (± 0.41%) +0.00s (+ 0.06%) 3.40s 3.46s
Total Time 11.97s (± 0.30%) 11.97s (± 0.21%) -0.00s (- 0.03%) 11.91s 12.02s
TFS - node (v8.9.0, x64)
Memory used 310,593k (± 0.02%) 310,760k (± 0.02%) +168k (+ 0.05%) 310,667k 310,891k
Parse Time 1.56s (± 0.38%) 1.57s (± 0.81%) +0.01s (+ 0.51%) 1.55s 1.61s
Bind Time 0.69s (± 0.97%) 0.69s (± 1.06%) +0.00s (+ 0.15%) 0.67s 0.70s
Check Time 5.37s (± 0.56%) 5.36s (± 0.46%) -0.00s (- 0.02%) 5.32s 5.43s
Emit Time 2.96s (± 0.77%) 2.99s (± 0.64%) +0.02s (+ 0.81%) 2.95s 3.04s
Total Time 10.58s (± 0.48%) 10.61s (± 0.36%) +0.03s (+ 0.26%) 10.55s 10.71s
material-ui - node (v8.9.0, x64)
Memory used 500,676k (± 0.01%) 503,642k (± 0.01%) +2,966k (+ 0.59%) 503,517k 503,756k
Parse Time 2.50s (± 0.50%) 2.50s (± 0.50%) +0.00s (+ 0.12%) 2.48s 2.53s
Bind Time 0.81s (± 1.00%) 0.81s (± 1.53%) +0.00s (+ 0.37%) 0.78s 0.83s
Check Time 18.24s (± 1.27%) 18.07s (± 0.84%) -0.17s (- 0.93%) 17.77s 18.45s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 21.55s (± 1.05%) 21.39s (± 0.76%) -0.16s (- 0.77%) 21.08s 21.80s
Angular - node (v8.9.0, x86)
Memory used 199,984k (± 0.03%) 200,150k (± 0.02%) +167k (+ 0.08%) 200,054k 200,246k
Parse Time 2.45s (± 0.49%) 2.44s (± 0.94%) -0.01s (- 0.45%) 2.40s 2.51s
Bind Time 1.00s (± 0.75%) 1.00s (± 1.10%) -0.00s (- 0.40%) 0.97s 1.02s
Check Time 5.06s (± 0.71%) 5.06s (± 0.50%) -0.00s (- 0.08%) 5.02s 5.13s
Emit Time 6.15s (± 0.99%) 6.10s (± 0.76%) -0.05s (- 0.85%) 5.96s 6.18s
Total Time 14.66s (± 0.58%) 14.59s (± 0.45%) -0.07s (- 0.48%) 14.45s 14.75s
Compiler-Unions - node (v8.9.0, x86)
Memory used 132,653k (± 0.02%) 133,671k (± 0.02%) +1,018k (+ 0.77%) 133,573k 133,735k
Parse Time 0.96s (± 0.51%) 0.96s (± 0.58%) -0.00s (- 0.21%) 0.95s 0.98s
Bind Time 0.50s (± 1.76%) 0.50s (± 0.96%) -0.00s (- 0.60%) 0.49s 0.51s
Check Time 12.05s (± 0.50%) 11.80s (± 0.60%) -0.25s (- 2.05%) 11.66s 11.98s
Emit Time 2.64s (± 1.19%) 2.73s (± 0.95%) +0.08s (+ 3.14%) 2.67s 2.78s
Total Time 16.15s (± 0.54%) 15.98s (± 0.50%) -0.17s (- 1.03%) 15.85s 16.17s
Monaco - node (v8.9.0, x86)
Memory used 203,406k (± 0.02%) 203,422k (± 0.02%) +16k (+ 0.01%) 203,328k 203,486k
Parse Time 1.99s (± 1.34%) 1.97s (± 0.43%) -0.02s (- 0.86%) 1.96s 1.99s
Bind Time 0.72s (± 0.92%) 0.72s (± 0.95%) -0.00s (- 0.56%) 0.71s 0.74s
Check Time 5.75s (± 1.93%) 5.63s (± 1.69%) -0.13s (- 2.22%) 5.49s 5.86s
Emit Time 2.88s (± 4.43%) 2.98s (± 4.39%) +0.09s (+ 3.26%) 2.72s 3.14s
Total Time 11.34s (± 0.45%) 11.29s (± 0.50%) -0.06s (- 0.50%) 11.18s 11.41s
TFS - node (v8.9.0, x86)
Memory used 177,809k (± 0.02%) 177,828k (± 0.03%) +19k (+ 0.01%) 177,726k 178,012k
Parse Time 1.63s (± 1.71%) 1.61s (± 1.65%) -0.02s (- 1.16%) 1.58s 1.71s
Bind Time 0.66s (± 1.05%) 0.65s (± 0.95%) -0.01s (- 0.92%) 0.64s 0.66s
Check Time 4.90s (± 0.50%) 4.89s (± 0.40%) -0.01s (- 0.16%) 4.84s 4.94s
Emit Time 2.83s (± 1.05%) 2.85s (± 1.01%) +0.03s (+ 0.99%) 2.80s 2.93s
Total Time 10.01s (± 0.57%) 10.01s (± 0.26%) -0.01s (- 0.06%) 9.95s 10.06s
material-ui - node (v8.9.0, x86)
Memory used 282,078k (± 0.02%) 283,651k (± 0.01%) +1,573k (+ 0.56%) 283,585k 283,716k
Parse Time 2.56s (± 0.90%) 2.54s (± 0.59%) -0.02s (- 0.66%) 2.53s 2.59s
Bind Time 0.71s (± 4.82%) 0.70s (± 2.21%) -0.01s (- 1.54%) 0.68s 0.76s
Check Time 16.66s (± 0.68%) 16.51s (± 0.48%) -0.15s (- 0.89%) 16.34s 16.72s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 19.93s (± 0.72%) 19.76s (± 0.44%) -0.18s (- 0.89%) 19.57s 20.01s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-197-generic
Architecturex64
Available Memory16 GB
Available Memory9 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v8.9.0, x64)
  • node (v8.9.0, x86)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v8.9.0, x64)
  • Angular - node (v8.9.0, x86)
  • Compiler-Unions - node (v10.16.3, x64)
  • Compiler-Unions - node (v12.1.0, x64)
  • Compiler-Unions - node (v8.9.0, x64)
  • Compiler-Unions - node (v8.9.0, x86)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v8.9.0, x64)
  • Monaco - node (v8.9.0, x86)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v8.9.0, x64)
  • TFS - node (v8.9.0, x86)
  • material-ui - node (v10.16.3, x64)
  • material-ui - node (v12.1.0, x64)
  • material-ui - node (v8.9.0, x64)
  • material-ui - node (v8.9.0, x86)
Benchmark Name Iterations
Current 42211 10
Baseline master 10

@microsoft microsoft deleted a comment from typescript-bot Jan 6, 2021
@weswigham
Copy link
Member Author

@typescript-bot perf test this faster

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jan 6, 2021

Heya @weswigham, I've started to run the abridged perf test suite on this PR at 7879343. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

@weswigham
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..42211

Metric master 42211 Delta Best Worst
Angular - node (v12.1.0, x64)
Memory used 323,253k (± 0.11%) 323,452k (± 0.06%) +200k (+ 0.06%) 322,838k 323,771k
Parse Time 1.98s (± 0.73%) 1.97s (± 0.52%) -0.01s (- 0.55%) 1.95s 1.99s
Bind Time 0.82s (± 0.89%) 0.81s (± 0.71%) -0.01s (- 0.98%) 0.80s 0.82s
Check Time 4.88s (± 0.51%) 4.90s (± 1.26%) +0.02s (+ 0.39%) 4.83s 5.14s
Emit Time 5.49s (± 0.61%) 5.50s (± 0.58%) +0.00s (+ 0.04%) 5.42s 5.57s
Total Time 13.18s (± 0.26%) 13.18s (± 0.67%) +0.00s (+ 0.02%) 13.05s 13.51s
Compiler-Unions - node (v12.1.0, x64)
Memory used 198,607k (± 0.05%) 200,036k (± 0.04%) +1,429k (+ 0.72%) 199,759k 200,190k
Parse Time 0.78s (± 1.13%) 0.77s (± 0.98%) -0.01s (- 0.77%) 0.76s 0.79s
Bind Time 0.50s (± 0.94%) 0.50s (± 0.45%) -0.00s (- 0.20%) 0.49s 0.50s
Check Time 9.56s (± 1.05%) 9.49s (± 0.89%) -0.07s (- 0.70%) 9.25s 9.63s
Emit Time 2.35s (± 1.32%) 2.36s (± 1.07%) +0.01s (+ 0.60%) 2.30s 2.42s
Total Time 13.19s (± 0.92%) 13.13s (± 0.75%) -0.06s (- 0.43%) 12.88s 13.32s
Monaco - node (v12.1.0, x64)
Memory used 337,115k (± 0.02%) 337,207k (± 0.02%) +92k (+ 0.03%) 337,081k 337,375k
Parse Time 1.59s (± 0.36%) 1.58s (± 0.41%) -0.01s (- 0.50%) 1.57s 1.59s
Bind Time 0.71s (± 1.36%) 0.71s (± 0.53%) -0.01s (- 0.84%) 0.70s 0.71s
Check Time 4.92s (± 0.36%) 4.91s (± 0.60%) -0.00s (- 0.10%) 4.84s 4.97s
Emit Time 2.87s (± 0.78%) 2.87s (± 0.57%) 0.00s ( 0.00%) 2.84s 2.92s
Total Time 10.09s (± 0.38%) 10.07s (± 0.42%) -0.02s (- 0.16%) 9.96s 10.17s
TFS - node (v12.1.0, x64)
Memory used 292,215k (± 0.01%) 292,403k (± 0.04%) +188k (+ 0.06%) 292,182k 292,660k
Parse Time 1.25s (± 0.80%) 1.25s (± 0.47%) -0.01s (- 0.56%) 1.24s 1.26s
Bind Time 0.66s (± 1.33%) 0.65s (± 0.51%) -0.01s (- 1.52%) 0.64s 0.66s
Check Time 4.54s (± 0.23%) 4.52s (± 0.52%) -0.02s (- 0.40%) 4.46s 4.59s
Emit Time 2.96s (± 0.84%) 2.98s (± 0.67%) +0.02s (+ 0.78%) 2.93s 3.02s
Total Time 9.41s (± 0.34%) 9.40s (± 0.32%) -0.01s (- 0.15%) 9.34s 9.48s
material-ui - node (v12.1.0, x64)
Memory used 470,657k (± 0.01%) 473,112k (± 0.06%) +2,455k (+ 0.52%) 472,031k 473,368k
Parse Time 2.08s (± 0.86%) 2.07s (± 0.48%) -0.01s (- 0.38%) 2.05s 2.10s
Bind Time 0.64s (± 0.47%) 0.64s (± 0.77%) -0.00s (- 0.31%) 0.63s 0.65s
Check Time 12.15s (± 0.82%) 12.10s (± 0.73%) -0.05s (- 0.40%) 11.99s 12.36s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 14.87s (± 0.79%) 14.81s (± 0.63%) -0.06s (- 0.40%) 14.68s 15.07s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-197-generic
Architecturex64
Available Memory16 GB
Available Memory4 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v12.1.0, x64)
Scenarios
  • Angular - node (v12.1.0, x64)
  • Compiler-Unions - node (v12.1.0, x64)
  • Monaco - node (v12.1.0, x64)
  • TFS - node (v12.1.0, x64)
  • material-ui - node (v12.1.0, x64)
Benchmark Name Iterations
Current 42211 10
Baseline master 10

@weswigham
Copy link
Member Author

Yeah, those perf numbers look fine.

@sandersn
Copy link
Member

Unfortunately, we never finished reviewing this PR. It is pretty old now, so I'm going to close it to reduce the number of open PRs.

@sandersn sandersn closed this May 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Milestone Bug PRs that fix a bug with a specific milestone
Projects
Archived in project
3 participants