Improve parse item fallback #125388

dev-ardi · 2024-05-21T22:56:18Z

Closes #92615 and closes #101622
Please review the new diagnostics cafefully.

This PR also adds a few more comments to the parser.

rustbot · 2024-05-21T22:56:26Z

rustbot has assigned @wesleywiser.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

compiler-errors · 2024-05-22T02:47:07Z

tests/ui/parser/shebang/shebang-doc-comment.stderr

   |
-   = note: for a full list of items that can appear in modules, see <https://doc.rust-lang.org/reference/items.html>
+   = help: consider putting it inside a function: fn main() { [allow(unused_variables)] }


Hm, I don't think we want to have this recovery, at least when you could parse it like a macro attr w/o a #.

Hm it thinks that it's an expression so it suggests putting it inside a function.
How can we tell that this is a bad attr and not an array literal with a function call?

SparrowLii

I don't think this PR solves #101622 :(

SparrowLii · 2024-05-24T01:51:10Z

compiler/rustc_parse/src/parser/item.rs

+                    .dcx()
+                    .struct_span_err(self.token.span, format!("expected item, found {token_str}"));
+
+                match self.parse_full_stmt(AttemptLocalParseRecovery::No) {


We'd be better doing more analysis to figure out what went wrong here, rather than treating them all as statements. For example, whether it contains ; in [] can be used to distinguish whether it is an attribute or an array.

SparrowLii · 2024-05-24T02:02:16Z

compiler/rustc_parse/src/parser/item.rs

+                                ));
+                            }
+                            StmtKind::MacCall(_) => {
+                                err.span_label(span, "unexpected macro call at this position");


At least one example should be added for it

you mean consider putting it inside a function?

compiler/rustc_parse/src/parser/item.rs

SparrowLii · 2024-05-24T02:15:52Z

compiler/rustc_parse/src/parser/item.rs

+
+                            StmtKind::Semi(e) | StmtKind::Expr(e) => {
+                                err.span_label(span, "unexpected expression").help(format!(
+                                    "consider putting it inside a function: fn main() {{ {} }}",


This tip also applies to the let statement above

dev-ardi · 2024-05-24T10:48:07Z

I don't think this PR solves #101622 :(

It addresses the main point raised in the issue, right?

dev-ardi · 2024-05-31T15:32:27Z

compiler/rustc_ast_pretty/src/pprust/mod.rs

@@ -76,6 +76,14 @@ pub fn attribute_to_string(attr: &ast::Attribute) -> String {
    State::new().attribute_to_string(attr)
 }

+pub fn local_to_string(local: &ast::Local) -> String {


This ended up being useless but I don't think that it's bad to have, right?

dev-ardi · 2024-05-31T15:35:51Z

@rustbot ready
@SparrowLii I've addressed your concerns and reworked the approach a bit.

bors · 2024-06-18T12:28:21Z

☔ The latest upstream changes (presumably #126049) made this pull request unmergeable. Please resolve the merge conflicts.

dev-ardi · 2024-06-20T06:10:50Z

@rustbot author

rust-log-analyzer · 2024-07-04T22:57:44Z

The job x86_64-gnu-llvm-17 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

 - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 46)
 - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 49)
 - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 61)
##[endgroup]
Setting extra environment values for docker:  --env ENABLE_GCC_CODEGEN=1 --env GCC_EXEC_PREFIX=/usr/lib/gcc/
[CI_JOB_NAME=x86_64-gnu-llvm-17]
---
sccache: Starting the server...
##[group]Configure the build
configure: processing command line
configure: 
configure: build.configure-args := ['--build=x86_64-unknown-linux-gnu', '--llvm-root=/usr/lib/llvm-17', '--enable-llvm-link-shared', '--set', 'rust.thin-lto-import-instr-limit=10', '--set', 'change-id=99999999', '--enable-verbose-configure', '--enable-sccache', '--disable-manage-submodules', '--enable-locked-deps', '--enable-cargo-native-static', '--set', 'rust.codegen-units-std=1', '--set', 'dist.compression-profile=balanced', '--dist-compression-formats=xz', '--set', 'rust.lld=false', '--disable-dist-src', '--release-channel=nightly', '--enable-debug-assertions', '--enable-overflow-checks', '--enable-llvm-assertions', '--set', 'rust.verify-llvm-ir', '--set', 'rust.codegen-backends=llvm,cranelift,gcc', '--set', 'llvm.static-libstdcpp', '--enable-new-symbol-mangling']
configure: target.x86_64-unknown-linux-gnu.llvm-config := /usr/lib/llvm-17/bin/llvm-config
configure: llvm.link-shared     := True
configure: rust.thin-lto-import-instr-limit := 10
configure: change-id            := 99999999
---

---- [ui] tests/ui/proc-macro/issue-78675-captured-inner-attrs.rs stdout ----
diff of stdout:

- PRINT-BANG INPUT (DISPLAY): foo! { #[fake_attr] mod bar { #![doc = r" Foo"] } }
- PRINT-BANG DEEP-RE-COLLECTED (DISPLAY): foo! { #[fake_attr] mod bar { #! [doc = r" Foo"] } }
+ PRINT-BANG INPUT (DISPLAY): foo! { #[fake_attr] mod bar { #! [doc = r" Foo"] } }
3 PRINT-BANG INPUT (DEBUG): TokenStream [
5         ident: "foo",


The actual stdout differed from the expected stdout.
The actual stdout differed from the expected stdout.
Actual stdout saved to /checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/proc-macro/issue-78675-captured-inner-attrs/issue-78675-captured-inner-attrs.stdout
To update references, rerun the tests and pass the `--bless` flag
To only update this specific test, also pass `--test-args proc-macro/issue-78675-captured-inner-attrs.rs`

error: 1 errors occurred comparing output.
status: exit status: 0
command: env -u RUSTC_LOG_COLOR RUSTC_ICE="0" RUST_BACKTRACE="short" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/tests/ui/proc-macro/issue-78675-captured-inner-attrs.rs" "-Zthreads=1" "-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX" "-Ztranslate-remapped-path-to-local-path=no" "-Z" "ignore-directory-in-diagnostics-source-blocks=/cargo" "-Z" "ignore-directory-in-diagnostics-source-blocks=/checkout/vendor" "--sysroot" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2" "--target=x86_64-unknown-linux-gnu" "--check-cfg" "cfg(FALSE)" "--error-format" "json" "--json" "future-incompat" "-Ccodegen-units=1" "-Zui-testing" "-Zdeduplicate-diagnostics=no" "-Zwrite-long-types-to-disk=no" "-Cstrip=debuginfo" "--emit" "metadata" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/proc-macro/issue-78675-captured-inner-attrs" "-A" "unused" "-A" "internal_features" "-Crpath" "-Cdebuginfo=0" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/proc-macro/issue-78675-captured-inner-attrs/auxiliary" "--edition=2018" "-Z" "span-debug"
--- stdout -------------------------------
PRINT-BANG INPUT (DISPLAY): foo! { #[fake_attr] mod bar { #! [doc = r" Foo"] } }
PRINT-BANG INPUT (DEBUG): TokenStream [
        ident: "foo",
        ident: "foo",
        span: /checkout/tests/ui/proc-macro/issue-78675-captured-inner-attrs.rs:20:9: 20:12 (#3),
    Punct {
    Punct {
        ch: '!',
        spacing: Alone,
        span: /checkout/tests/ui/proc-macro/issue-78675-captured-inner-attrs.rs:20:12: 20:13 (#3),
    Group {
        delimiter: Brace,
        stream: TokenStream [
            Punct {
            Punct {
                ch: '#',
                spacing: Alone,
                span: /checkout/tests/ui/proc-macro/issue-78675-captured-inner-attrs.rs:21:13: 21:14 (#3),
            Group {
                delimiter: Bracket,
                stream: TokenStream [
                    Ident {
                    Ident {
                        ident: "fake_attr",
                        span: /checkout/tests/ui/proc-macro/issue-78675-captured-inner-attrs.rs:21:15: 21:24 (#3),
                ],
                ],
                span: /checkout/tests/ui/proc-macro/issue-78675-captured-inner-attrs.rs:21:14: 21:25 (#3),
            Group {
                delimiter: None,
                stream: TokenStream [
                    Ident {
                    Ident {
                        ident: "mod",
                        span: /checkout/tests/ui/proc-macro/issue-78675-captured-inner-attrs.rs:27:5: 27:8 (#0),
                    Ident {
                        ident: "bar",
                        ident: "bar",
                        span: /checkout/tests/ui/proc-macro/issue-78675-captured-inner-attrs.rs:27:9: 27:12 (#0),
                    Group {
                        delimiter: Brace,
                        stream: TokenStream [
                            Punct {
                            Punct {
                                ch: '#',
                                spacing: Joint,
                                span: /checkout/tests/ui/proc-macro/issue-78675-captured-inner-attrs.rs:28:9: 28:16 (#0),
                            Punct {
                            Punct {
                                ch: '!',
                                spacing: Alone,
                                span: /checkout/tests/ui/proc-macro/issue-78675-captured-inner-attrs.rs:28:9: 28:16 (#0),
                            Group {
                                delimiter: Bracket,
                                stream: TokenStream [
                                    Ident {
                                    Ident {
                                        ident: "doc",
                                        span: /checkout/tests/ui/proc-macro/issue-78675-captured-inner-attrs.rs:28:9: 28:16 (#0),
                                    Punct {
                                    Punct {
                                        ch: '=',
                                        spacing: Alone,
                                        span: /checkout/tests/ui/proc-macro/issue-78675-captured-inner-attrs.rs:28:9: 28:16 (#0),
                                    Literal {
                                        kind: StrRaw(0),
                                        symbol: " Foo",
                                        suffix: None,
                                        suffix: None,
                                        span: /checkout/tests/ui/proc-macro/issue-78675-captured-inner-attrs.rs:28:9: 28:16 (#0),
                                ],
                                ],
                                span: /checkout/tests/ui/proc-macro/issue-78675-captured-inner-attrs.rs:28:9: 28:16 (#0),
                        ],
                        ],
                        span: /checkout/tests/ui/proc-macro/issue-78675-captured-inner-attrs.rs:27:13: 29:6 (#0),
                ],
                ],
                span: /checkout/tests/ui/proc-macro/issue-78675-captured-inner-attrs.rs:22:13: 22:18 (#3),
        ],
        ],
        span: /checkout/tests/ui/proc-macro/issue-78675-captured-inner-attrs.rs:20:14: 23:10 (#3),
]
------------------------------------------
stderr: none

bors · 2024-07-18T13:54:09Z

☔ The latest upstream changes (presumably #127906) made this pull request unmergeable. Please resolve the merge conflicts.

JohnCSimon · 2024-09-03T22:47:16Z

@dev-ardi
ping from triage - can you post your status on this PR? This PR has not received an update in a few months.

dev-ardi · 2024-09-03T23:11:31Z

It was left unreviewed for a while so it became full of conflicts. I intend to finish it at some point though...

rustbot assigned wesleywiser May 21, 2024

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels May 21, 2024

compiler-errors reviewed May 22, 2024

View reviewed changes

SparrowLii reviewed May 24, 2024

View reviewed changes

dev-ardi force-pushed the improve-parse-item-fallback branch 3 times, most recently from 9a129e5 to 06a1ccf Compare May 31, 2024 15:29

dev-ardi commented May 31, 2024

View reviewed changes

dev-ardi force-pushed the improve-parse-item-fallback branch from 06a1ccf to 1d326d5 Compare June 19, 2024 20:19

This comment has been minimized.

Sign in to view

dev-ardi force-pushed the improve-parse-item-fallback branch 2 times, most recently from 3f2ccd7 to b9ddf53 Compare June 20, 2024 22:31

improve parse item fallback

c66b446

dev-ardi force-pushed the improve-parse-item-fallback branch from b9ddf53 to c66b446 Compare July 4, 2024 22:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve parse item fallback #125388

Improve parse item fallback #125388

dev-ardi commented May 21, 2024 •

edited

Loading

rustbot commented May 21, 2024

compiler-errors May 22, 2024

dev-ardi May 22, 2024

SparrowLii left a comment

SparrowLii May 24, 2024 •

edited

Loading

SparrowLii May 24, 2024 •

edited

Loading

dev-ardi May 24, 2024

SparrowLii May 24, 2024

dev-ardi commented May 24, 2024

dev-ardi May 31, 2024

dev-ardi commented May 31, 2024

bors commented Jun 18, 2024

This comment has been minimized.

dev-ardi commented Jun 20, 2024 •

edited

Loading

rust-log-analyzer commented Jul 4, 2024

bors commented Jul 18, 2024

JohnCSimon commented Sep 3, 2024

dev-ardi commented Sep 3, 2024

Improve parse item fallback #125388

Are you sure you want to change the base?

Improve parse item fallback #125388

Conversation

dev-ardi commented May 21, 2024 • edited Loading

rustbot commented May 21, 2024

compiler-errors May 22, 2024

Choose a reason for hiding this comment

dev-ardi May 22, 2024

Choose a reason for hiding this comment

SparrowLii left a comment

Choose a reason for hiding this comment

SparrowLii May 24, 2024 • edited Loading

Choose a reason for hiding this comment

SparrowLii May 24, 2024 • edited Loading

Choose a reason for hiding this comment

dev-ardi May 24, 2024

Choose a reason for hiding this comment

SparrowLii May 24, 2024

Choose a reason for hiding this comment

dev-ardi commented May 24, 2024

dev-ardi May 31, 2024

Choose a reason for hiding this comment

dev-ardi commented May 31, 2024

bors commented Jun 18, 2024

This comment has been minimized.

dev-ardi commented Jun 20, 2024 • edited Loading

rust-log-analyzer commented Jul 4, 2024

bors commented Jul 18, 2024

JohnCSimon commented Sep 3, 2024

dev-ardi commented Sep 3, 2024

dev-ardi commented May 21, 2024 •

edited

Loading

SparrowLii May 24, 2024 •

edited

Loading

SparrowLii May 24, 2024 •

edited

Loading

dev-ardi commented Jun 20, 2024 •

edited

Loading