Improve unexpected close and mismatch delimiter hint in TokenTreesReader

[chenyukang]

Fixes #103882
Fixes #68987
Fixes #69259

The inner indentation mismatching will be covered by outer block, the new added function report_error_prone_delim_block will find out the error prone candidates for reporting.

[rustbot]

r? @wesleywiser

(rustbot has picked a reviewer for you, use r? to override)

[chenyukang]

This is an issue I found from rustc dev experience, for example this file:
c3.txt

We have an extra '{' at line 1420, but the first error report from compiler is:

    --> e3.rs:2605:3
     |
38   | impl<'a, 'tcx> FnCtxt<'a, 'tcx> {
     |                                 - unclosed delimiter
...
110  |                                  sugg_span: Span| {
     |                                                   - this delimiter might not be properly closed...
...
240  |         };
     |         - ...as it matches this but it has different indentation
...
2605 | }
     |   ^

error: expected expression, found `let` statement
    --> e3.rs:1421:12
     |
1421 |         && let Some((fields, substs)) =
     |            ^^^

The first diagnostic's tip this delimiter might not be properly closed is misleading, this fix will remove it, then the second diagnostic help us narrowing the scope of root cause.

[chenyukang]

Oops, find a better solution....

[chenyukang]

@estebank
I find out a better solution for this issue 😁

[chenyukang]

It will report out this, we have narrowed the scope of root-cause:

error: this file contains an unclosed delimiter
  --> src/test/ui/parser/deli-ident-issue-1.rs:27:65
   |
7  | impl dyn Demo {
   |               - unclosed delimiter
...
19 |         && let Some(c) = num {
   |                              - this delimiter might not be properly closed...
...
24 |     }
   |     - ...as it matches this but it has different indentation
...
27 | fn main() { } //~ ERROR this file contains an unclosed delimiter
   |                                                                 ^

[chenyukang]

But we still can not handle well for this kind of case:

fn f(i: u32, j: u32) {
    let res = String::new();
    let mut cnt = i;
    while cnt < j {
        write!&mut res, " ");
    }
}

Current error is:

error: unexpected closing delimiter: `}`
 --> bug3.rs:7:1
  |
1 | fn f(i: u32, j: u32) {
  |                      - this delimiter might not be properly closed...
...
6 |     }
  |     - ...as it matches this but it has different indentation
7 | }
  | ^ unexpected closing delimiter

error: mismatched closing delimiter: `)`

And this code:

async fn obstest() -> Result<> {
    let obs_connect = || -> Result<(), MyError) {
        async {
        }
    }

    if let Ok(version, scene_list) = obs_connect() {

    } else {

    }
}

report:

error: unexpected closing delimiter: `}`
  --> bug2.rs:12:1
   |
12 | }
   | ^ unexpected closing delimiter

error: mismatched closing delimiter: `)`
 --> bug2.rs:1:32
  |
1 | async fn obstest() -> Result<> {
  |                                ^ unclosed delimiter
2 |     let obs_connect = || -> Result<(), MyError) {
  |                                               ^ mismatched closing delimiter

error: aborting due to 2 previous errors

This is because we currently may use ')' to match '{' as a close delimiter.
To fix this need more work, I have a working branch and need more tweaks.

[wesleywiser]

I'm not very familiar with the reporting code here so

r? @estebank

[chenyukang]

@estebank

I made some more changes in another branch, I'm not sure whether it's Ok for merging it.
chenyukang@479fdb5

The main idea is reducing later errors when delimiter mismatch error happens.

It is somewhat common to ignore all but the first error when you have mismatched delimiters.

I read the source code, we exit when unexpected closing delimiter:

rust/compiler/rustc_parse/src/lexer/tokentrees.rs

Line 58 in b7b7f27

Err(self.close_delim_err(delim))

So for code like this:

}

fn main() {
    x = 1;
}

Compiler will exit with the only first error:

error: unexpected closing delimiter: `}`
 --> p/f.rs:2:1
  |
2 | }
  | ^ unexpected closing delimiter

error: aborting due to previous error

But we only report an error when this error: this file contains an unclosed delimiter happens:

rust/compiler/rustc_parse/src/lexer/tokentrees.rs

Line 63 in b7b7f27

self.eof_err().emit();

In real dev practice, if the file is somehow very long, unclosed delimier will make parser totally undertand wrongly on AST, so it will give out many errors, such as the case in my comment: #104012 (comment).

My change also returning Err on this line, and we can avoid later noises.

I'm not sure whether we have any other special considertaion about this, seems there are some recovery code in TokenTreesReader, but actually we can not recovery from delimiter errors, because there are too many possible ways to fix a mismatch or unclosing error, compiler can only give proper diagnostics for user.

[estebank]

Yeah, let's create a PR for chenyukang@479fdb5 and land it. Sadly delimiter recovery is quite hard and the current behavior isn't what's best for our users.

[chenyukang]

Conflict is fixed,
@estebank any more comments for this PR?

[petrochenkov]

I didn't read too much into the details of report_suspicious_mismatch_block, but the test diffs look fine.
@rustbot author

[petrochenkov]

r=me after addressing #104012 (comment).

[chenyukang]

@rustbot ready

[petrochenkov]

@bors r+

[bors]

📌 Commit cd23323 has been approved by petrochenkov

It is now in the queue for this repository.