Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(44466): Fixes parsing contextual keyword casts as arrow functions #49029

Conversation

DavidSouther
Copy link
Contributor

@DavidSouther DavidSouther commented May 9, 2022

Fixes #44466
Fixes #49021

@ghost
Copy link

ghost commented May 9, 2022

CLA assistant check
All CLA requirements met.

@DanielRosenwasser
Copy link
Member

Can you use the test provided in the #49021?

tests/cases/conformance/expressions/typeAssertions/variablesWithModifierNamesInAsTypeAssertions01.ts

// @declaration: true

let readonly: any = undefined;
let override: any = undefined;
let out: any = undefined;
let declare: any = undefined;

export const a = (readonly as number);
export const b = (override as number);
export const c = (out as number);
export const d = (declare as number);

@DanielRosenwasser
Copy link
Member

@typescript-bot pack this
@typescript-bot test this
@typescript-bot user test this inline
@typescript-bot run dt
@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 9, 2022

Heya @DanielRosenwasser, I've started to run the diff-based community code test suite on this PR at 3dd9ef4. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 9, 2022

Heya @DanielRosenwasser, I've started to run the parallelized Definitely Typed test suite on this PR at 3dd9ef4. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 9, 2022

Heya @DanielRosenwasser, I've started to run the perf test suite on this PR at 3dd9ef4. You can monitor the build here.

Update: The results are in!

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 9, 2022

Heya @DanielRosenwasser, I've started to run the extended test suite on this PR at 3dd9ef4. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 9, 2022

Heya @DanielRosenwasser, I've started to run the tarball bundle task on this PR at 3dd9ef4. You can monitor the build here.

@DavidSouther
Copy link
Contributor Author

@DanielRosenwasser done

src/compiler/parser.ts Outdated Show resolved Hide resolved
@typescript-bot
Copy link
Collaborator

@DanielRosenwasser
Great news! no new errors were found between main..refs/pull/49029/merge

@@ -4538,6 +4538,10 @@ namespace ts {
// isn't actually allowed, but we want to treat it as a lambda so we can provide
// a good error message.
if (isModifierKind(second) && second !== SyntaxKind.AsyncKeyword && lookAhead(nextTokenIsIdentifier)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fact that nextTokenIsIdentifier checks for async and yield itself makes me feel like there should be some sort of context for as, rather than special casing, but like you said on #44466 (comment), I'm not entirely certain that there's another expression that can behave like this.

This whole "does this look like an arrow function" is effectively a heuristic and continues to bite us (see: #16241), so I think it's likely ok to disable this heuristic here like you've done.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the other option is reworking this along the lines Daniel suggested earlier in #44466, but looking for a matched close paren followed by an arrow? But doing that leaves a lot of lookahead for errors to accumulate, and you're not guaranteed to find the matching paren, much less the arrow itself.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think that this fix is good as-is; this block (as commented above) is to do better error recovery if someone writes code like (foo number) => ..., which is guaranteed to fail to parse later. The likelihood of that broken code appearing and someone legitimately naming a type as seems very, very low, so I'm happy with just not having recovery in this case.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose though, that one could write a test that is (foo as) => 1234 or something, and see if you should actually return TriState.Unknown here instead.

Or, that this should be a lookAhead in the if statement before the code you've added (the "isn't actually allowed" one), to better fit with the heuristic.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tested both, and they both pass the tests in the PR, though they all differ on a test like:

let out: any = undefined;
type as = any;

export const e = (declare as) => 1234;

With the two lookahead versions, you get an emitted exports.e = declare;, but with Unknown, it actually tries to parse that broken code as an arrow function again and complains about the modifier, and emits:

var e = function (as) { return 1234; };
exports.e = e;

Maybe that's better in the error case, preserving the recovery?

@DanielRosenwasser curious as to what you think; I'm not convinced it matters either way if both fix the bug, just an esoteric recovery.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is an okay fix; however, if we add satisfies (or any other new infix keyword) then we'll have another similar bug in the future.

@typescript-bot
Copy link
Collaborator

@DanielRosenwasser
The results of the perf run you requested are in!

Here they are:

Comparison Report - main..49029

Metric main 49029 Delta Best Worst
Angular - node (v10.16.3, x64)
Memory used 357,532k (± 0.01%) 357,583k (± 0.02%) +51k (+ 0.01%) 357,408k 357,788k
Parse Time 2.07s (± 0.41%) 2.07s (± 0.34%) +0.00s (+ 0.10%) 2.06s 2.09s
Bind Time 0.87s (± 0.79%) 0.86s (± 0.61%) -0.01s (- 0.81%) 0.85s 0.87s
Check Time 5.84s (± 0.46%) 5.84s (± 0.34%) -0.00s (- 0.03%) 5.81s 5.89s
Emit Time 6.00s (± 0.43%) 6.00s (± 0.35%) -0.00s (- 0.05%) 5.94s 6.04s
Total Time 14.78s (± 0.33%) 14.77s (± 0.21%) -0.02s (- 0.10%) 14.70s 14.85s
Compiler-Unions - node (v10.16.3, x64)
Memory used 205,933k (± 0.03%) 205,953k (± 0.02%) +20k (+ 0.01%) 205,831k 206,016k
Parse Time 0.83s (± 0.72%) 0.84s (± 0.66%) +0.00s (+ 0.36%) 0.83s 0.85s
Bind Time 0.51s (± 1.16%) 0.51s (± 0.93%) -0.00s (- 0.19%) 0.51s 0.53s
Check Time 7.90s (± 0.68%) 7.90s (± 0.73%) -0.00s (- 0.03%) 7.76s 8.09s
Emit Time 2.50s (± 0.76%) 2.52s (± 1.20%) +0.02s (+ 0.72%) 2.46s 2.61s
Total Time 11.75s (± 0.60%) 11.77s (± 0.54%) +0.02s (+ 0.13%) 11.69s 11.99s
Monaco - node (v10.16.3, x64)
Memory used 343,791k (± 0.01%) 343,766k (± 0.02%) -25k (- 0.01%) 343,636k 343,895k
Parse Time 1.59s (± 0.42%) 1.58s (± 0.56%) -0.01s (- 0.44%) 1.57s 1.60s
Bind Time 0.76s (± 0.66%) 0.76s (± 0.49%) 0.00s ( 0.00%) 0.75s 0.76s
Check Time 5.82s (± 0.58%) 5.77s (± 0.46%) -0.04s (- 0.72%) 5.71s 5.83s
Emit Time 3.24s (± 0.73%) 3.23s (± 0.81%) -0.00s (- 0.12%) 3.17s 3.29s
Total Time 11.40s (± 0.40%) 11.35s (± 0.19%) -0.05s (- 0.46%) 11.29s 11.40s
TFS - node (v10.16.3, x64)
Memory used 305,461k (± 0.04%) 305,409k (± 0.02%) -53k (- 0.02%) 305,276k 305,501k
Parse Time 1.28s (± 0.28%) 1.28s (± 0.39%) +0.00s (+ 0.08%) 1.27s 1.29s
Bind Time 0.71s (± 0.73%) 0.71s (± 0.51%) +0.00s (+ 0.42%) 0.71s 0.72s
Check Time 5.23s (± 0.52%) 5.24s (± 0.45%) +0.01s (+ 0.15%) 5.20s 5.29s
Emit Time 3.48s (± 1.19%) 3.39s (± 0.77%) -0.09s (- 2.64%) 3.34s 3.45s
Total Time 10.71s (± 0.49%) 10.63s (± 0.35%) -0.09s (- 0.79%) 10.55s 10.74s
material-ui - node (v10.16.3, x64)
Memory used 470,239k (± 0.01%) 470,250k (± 0.01%) +11k (+ 0.00%) 470,133k 470,364k
Parse Time 1.82s (± 0.42%) 1.83s (± 0.51%) +0.01s (+ 0.27%) 1.81s 1.86s
Bind Time 0.67s (± 0.86%) 0.68s (± 0.82%) +0.00s (+ 0.74%) 0.67s 0.69s
Check Time 14.30s (± 0.55%) 14.25s (± 0.20%) -0.06s (- 0.38%) 14.17s 14.31s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 16.80s (± 0.48%) 16.76s (± 0.17%) -0.05s (- 0.27%) 16.68s 16.83s
xstate - node (v10.16.3, x64)
Memory used 571,728k (± 0.01%) 571,826k (± 0.02%) +99k (+ 0.02%) 571,587k 572,030k
Parse Time 2.58s (± 0.29%) 2.59s (± 0.34%) +0.00s (+ 0.15%) 2.56s 2.60s
Bind Time 1.01s (± 1.05%) 1.00s (± 0.47%) -0.00s (- 0.30%) 0.99s 1.01s
Check Time 1.52s (± 0.43%) 1.52s (± 0.54%) -0.00s (- 0.07%) 1.49s 1.53s
Emit Time 0.07s (± 0.00%) 0.07s (± 3.14%) +0.00s (+ 1.43%) 0.07s 0.08s
Total Time 5.18s (± 0.18%) 5.17s (± 0.28%) -0.01s (- 0.12%) 5.14s 5.21s
Angular - node (v12.1.0, x64)
Memory used 335,312k (± 0.03%) 335,267k (± 0.02%) -45k (- 0.01%) 335,131k 335,369k
Parse Time 2.07s (± 0.65%) 2.06s (± 0.51%) -0.01s (- 0.29%) 2.05s 2.09s
Bind Time 0.83s (± 0.60%) 0.83s (± 1.07%) +0.00s (+ 0.12%) 0.81s 0.85s
Check Time 5.65s (± 0.47%) 5.67s (± 0.97%) +0.02s (+ 0.30%) 5.59s 5.88s
Emit Time 6.25s (± 0.40%) 6.28s (± 1.23%) +0.03s (+ 0.46%) 6.18s 6.52s
Total Time 14.80s (± 0.28%) 14.85s (± 0.67%) +0.04s (+ 0.30%) 14.70s 15.06s
Compiler-Unions - node (v12.1.0, x64)
Memory used 193,535k (± 0.07%) 193,528k (± 0.12%) -8k (- 0.00%) 192,613k 193,817k
Parse Time 0.84s (± 0.87%) 0.84s (± 1.30%) +0.00s (+ 0.48%) 0.82s 0.87s
Bind Time 0.53s (± 0.75%) 0.53s (± 0.63%) -0.00s (- 0.19%) 0.52s 0.54s
Check Time 7.42s (± 0.27%) 7.40s (± 0.48%) -0.02s (- 0.23%) 7.33s 7.51s
Emit Time 2.54s (± 0.88%) 2.55s (± 1.59%) +0.01s (+ 0.28%) 2.48s 2.64s
Total Time 11.33s (± 0.35%) 11.32s (± 0.59%) -0.01s (- 0.11%) 11.21s 11.50s
Monaco - node (v12.1.0, x64)
Memory used 326,837k (± 0.01%) 326,843k (± 0.02%) +6k (+ 0.00%) 326,667k 327,004k
Parse Time 1.55s (± 0.49%) 1.56s (± 0.61%) +0.01s (+ 0.78%) 1.54s 1.58s
Bind Time 0.74s (± 0.67%) 0.74s (± 0.70%) +0.00s (+ 0.41%) 0.73s 0.75s
Check Time 5.65s (± 0.54%) 5.65s (± 0.53%) -0.00s (- 0.05%) 5.58s 5.71s
Emit Time 3.28s (± 0.70%) 3.27s (± 0.49%) -0.01s (- 0.43%) 3.24s 3.31s
Total Time 11.21s (± 0.31%) 11.21s (± 0.40%) -0.01s (- 0.04%) 11.12s 11.31s
TFS - node (v12.1.0, x64)
Memory used 290,116k (± 0.02%) 290,095k (± 0.02%) -21k (- 0.01%) 289,986k 290,210k
Parse Time 1.29s (± 0.73%) 1.29s (± 0.67%) -0.00s (- 0.08%) 1.28s 1.32s
Bind Time 0.73s (± 0.71%) 0.72s (± 0.77%) -0.01s (- 0.96%) 0.71s 0.73s
Check Time 5.20s (± 0.45%) 5.20s (± 0.34%) +0.00s (+ 0.04%) 5.16s 5.25s
Emit Time 3.46s (± 0.93%) 3.48s (± 0.66%) +0.02s (+ 0.58%) 3.43s 3.55s
Total Time 10.69s (± 0.55%) 10.70s (± 0.33%) +0.01s (+ 0.09%) 10.62s 10.78s
material-ui - node (v12.1.0, x64)
Memory used 449,252k (± 0.01%) 449,296k (± 0.02%) +43k (+ 0.01%) 449,141k 449,506k
Parse Time 1.82s (± 0.44%) 1.83s (± 0.38%) +0.01s (+ 0.38%) 1.81s 1.84s
Bind Time 0.65s (± 0.46%) 0.65s (± 0.76%) -0.00s (- 0.31%) 0.64s 0.66s
Check Time 12.83s (± 0.34%) 12.89s (± 0.56%) +0.06s (+ 0.45%) 12.78s 13.05s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.30s (± 0.28%) 15.36s (± 0.44%) +0.06s (+ 0.41%) 15.26s 15.52s
xstate - node (v12.1.0, x64)
Memory used 537,370k (± 0.01%) 537,382k (± 0.01%) +13k (+ 0.00%) 537,232k 537,536k
Parse Time 2.53s (± 0.49%) 2.52s (± 0.42%) -0.01s (- 0.32%) 2.50s 2.56s
Bind Time 1.03s (± 0.66%) 1.02s (± 0.91%) -0.01s (- 0.78%) 1.00s 1.04s
Check Time 1.46s (± 0.55%) 1.46s (± 0.64%) +0.00s (+ 0.07%) 1.44s 1.49s
Emit Time 0.07s (± 0.00%) 0.07s (± 0.00%) 0.00s ( 0.00%) 0.07s 0.07s
Total Time 5.09s (± 0.20%) 5.07s (± 0.38%) -0.02s (- 0.35%) 5.03s 5.12s
Angular - node (v14.15.1, x64)
Memory used 333,536k (± 0.01%) 333,532k (± 0.01%) -4k (- 0.00%) 333,467k 333,571k
Parse Time 2.05s (± 0.49%) 2.04s (± 0.36%) -0.00s (- 0.20%) 2.02s 2.06s
Bind Time 0.87s (± 0.97%) 0.87s (± 0.38%) -0.00s (- 0.11%) 0.86s 0.88s
Check Time 5.64s (± 0.61%) 5.64s (± 0.36%) -0.00s (- 0.07%) 5.59s 5.68s
Emit Time 6.33s (± 0.67%) 6.30s (± 0.52%) -0.03s (- 0.54%) 6.22s 6.37s
Total Time 14.89s (± 0.40%) 14.85s (± 0.24%) -0.04s (- 0.27%) 14.75s 14.90s
Compiler-Unions - node (v14.15.1, x64)
Memory used 192,210k (± 0.01%) 192,202k (± 0.01%) -7k (- 0.00%) 192,156k 192,284k
Parse Time 0.84s (± 0.70%) 0.85s (± 0.76%) +0.00s (+ 0.47%) 0.84s 0.86s
Bind Time 0.56s (± 0.93%) 0.56s (± 1.03%) +0.00s (+ 0.18%) 0.55s 0.57s
Check Time 7.53s (± 0.41%) 7.54s (± 0.73%) +0.01s (+ 0.12%) 7.42s 7.65s
Emit Time 2.51s (± 0.68%) 2.52s (± 0.91%) +0.01s (+ 0.28%) 2.48s 2.59s
Total Time 11.45s (± 0.32%) 11.46s (± 0.50%) +0.02s (+ 0.17%) 11.35s 11.58s
Monaco - node (v14.15.1, x64)
Memory used 325,593k (± 0.01%) 325,594k (± 0.01%) +1k (+ 0.00%) 325,539k 325,660k
Parse Time 1.56s (± 0.51%) 1.58s (± 0.43%) +0.01s (+ 0.90%) 1.56s 1.59s
Bind Time 0.77s (± 0.62%) 0.78s (± 0.94%) +0.00s (+ 0.65%) 0.77s 0.80s
Check Time 5.52s (± 0.49%) 5.51s (± 0.33%) -0.01s (- 0.16%) 5.48s 5.57s
Emit Time 3.33s (± 0.67%) 3.32s (± 0.76%) -0.00s (- 0.09%) 3.28s 3.37s
Total Time 11.18s (± 0.41%) 11.19s (± 0.38%) +0.01s (+ 0.12%) 11.10s 11.31s
TFS - node (v14.15.1, x64)
Memory used 289,104k (± 0.01%) 289,124k (± 0.01%) +20k (+ 0.01%) 289,050k 289,161k
Parse Time 1.36s (± 0.91%) 1.37s (± 2.04%) +0.01s (+ 0.88%) 1.33s 1.47s
Bind Time 0.72s (± 0.46%) 0.72s (± 0.80%) 0.00s ( 0.00%) 0.71s 0.73s
Check Time 5.17s (± 0.28%) 5.19s (± 0.37%) +0.02s (+ 0.35%) 5.15s 5.23s
Emit Time 3.52s (± 1.78%) 3.51s (± 2.01%) -0.01s (- 0.37%) 3.39s 3.65s
Total Time 10.78s (± 0.57%) 10.79s (± 0.91%) +0.02s (+ 0.14%) 10.63s 11.00s
material-ui - node (v14.15.1, x64)
Memory used 447,536k (± 0.00%) 447,543k (± 0.01%) +8k (+ 0.00%) 447,508k 447,621k
Parse Time 1.88s (± 0.48%) 1.87s (± 0.44%) -0.01s (- 0.48%) 1.85s 1.89s
Bind Time 0.70s (± 0.57%) 0.70s (± 0.53%) -0.00s (- 0.57%) 0.69s 0.70s
Check Time 12.96s (± 0.36%) 12.97s (± 0.67%) +0.01s (+ 0.08%) 12.83s 13.26s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.54s (± 0.34%) 15.54s (± 0.57%) -0.00s (- 0.02%) 15.38s 15.82s
xstate - node (v14.15.1, x64)
Memory used 535,151k (± 0.00%) 535,148k (± 0.00%) -4k (- 0.00%) 535,100k 535,194k
Parse Time 2.58s (± 0.40%) 2.58s (± 0.45%) -0.00s (- 0.12%) 2.56s 2.60s
Bind Time 1.15s (± 0.86%) 1.15s (± 0.83%) -0.00s (- 0.26%) 1.13s 1.17s
Check Time 1.52s (± 0.63%) 1.51s (± 0.41%) -0.00s (- 0.26%) 1.50s 1.53s
Emit Time 0.07s (± 0.00%) 0.07s (± 0.00%) 0.00s ( 0.00%) 0.07s 0.07s
Total Time 5.31s (± 0.24%) 5.31s (± 0.39%) -0.01s (- 0.13%) 5.28s 5.37s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-210-generic
Architecturex64
Available Memory16 GB
Available Memory2 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v14.15.1, x64)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v14.15.1, x64)
  • Compiler-Unions - node (v10.16.3, x64)
  • Compiler-Unions - node (v12.1.0, x64)
  • Compiler-Unions - node (v14.15.1, x64)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v14.15.1, x64)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v14.15.1, x64)
  • material-ui - node (v10.16.3, x64)
  • material-ui - node (v12.1.0, x64)
  • material-ui - node (v14.15.1, x64)
  • xstate - node (v10.16.3, x64)
  • xstate - node (v12.1.0, x64)
  • xstate - node (v14.15.1, x64)
Benchmark Name Iterations
Current 49029 10
Baseline main 10

Developer Information:

Download Benchmark

@@ -4538,6 +4538,10 @@ namespace ts {
// isn't actually allowed, but we want to treat it as a lambda so we can provide
// a good error message.
if (isModifierKind(second) && second !== SyntaxKind.AsyncKeyword && lookAhead(nextTokenIsIdentifier)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think that this fix is good as-is; this block (as commented above) is to do better error recovery if someone writes code like (foo number) => ..., which is guaranteed to fail to parse later. The likelihood of that broken code appearing and someone legitimately naming a type as seems very, very low, so I'm happy with just not having recovery in this case.

@DanielRosenwasser
Copy link
Member

@typescript-bot pack this

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 10, 2022

Heya @DanielRosenwasser, I've started to run the tarball bundle task on this PR at e9d39bd. You can monitor the build here.

@typescript-bot typescript-bot added For Backlog Bug PRs that fix a backlog bug For Milestone Bug PRs that fix a bug with a specific milestone labels May 10, 2022
@DanielRosenwasser
Copy link
Member

I'm going to cherry-pick this manually.

@DanielRosenwasser DanielRosenwasser merged commit 3e24f14 into microsoft:main May 10, 2022
@DanielRosenwasser
Copy link
Member

No I'm not, I accidentally merged instead of sqaushed.

@typescript-bot cherry-pick this to release-4.7

@typescript-bot
Copy link
Collaborator

typescript-bot commented May 10, 2022

Heya @DanielRosenwasser, I've started to run the task to cherry-pick this into release-4.7 on this PR at e9d39bd. You can monitor the build here.

@typescript-bot
Copy link
Collaborator

Hey @DanielRosenwasser, I couldn't open a PR with the cherry-pick. (You can check the log here). You may need to squash and pick this PR into release-4.7 manually.

@DanielRosenwasser
Copy link
Member

I guess I am 🙄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
For Backlog Bug PRs that fix a backlog bug For Milestone Bug PRs that fix a bug with a specific milestone
Projects
None yet
4 participants