Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

assume all sequences have (unique) spans for now #62831

Closed
SOF3 opened this issue Jul 20, 2019 · 3 comments · Fixed by #62956
Closed

assume all sequences have (unique) spans for now #62831

SOF3 opened this issue Jul 20, 2019 · 3 comments · Fixed by #62956
Labels
A-macros Area: All kinds of macros (custom derive, macro_rules!, proc macros, ..) C-bug Category: This is a bug. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@SOF3
Copy link
Contributor

SOF3 commented Jul 20, 2019 •
edited
Loading

Steps to reproduce:

The following generated macro is a likely source of error:

pub trait Foo {
    fn len(&self) -> usize;
}
macro_rules ! delegate_foo
{
    ($ (< $ ($ types : ty) , * > :) ? $ (< $ ($ trait_types : ty) , * > for) ?
     $ struct : ty : $ inner : ident ; $ ($ extra : tt) *) =>
    {
        impl Foo for $ struct
        { fn len (& self) -> usize { self . $ inner . len () } $ ($ extra) * }
    } ;
}

However, directly putting this macro does not seem to cause a compiler crash: playground link

It appears that the crash only occurs when the macro-by-example generated from a proc-macro.

The full error message:

 # RUST_BACKTRACE=1 cargo test -v
       Fresh unicode-xid v0.1.0
       Fresh strsim v0.7.0
       Fresh fnv v1.0.6
       Fresh ident_case v1.0.1
       Fresh unicode-segmentation v1.3.0
       Fresh heck v0.3.1
       Fresh proc-macro2 v0.4.30
       Fresh quote v0.6.13
       Fresh syn v0.15.39
       Fresh darling_core v0.9.0
       Fresh darling_macro v0.9.0
       Fresh darling v0.9.0
   Compiling delegate-impl v0.1.0 (/home/sofe/tmp/rust-macro-panic/delegate_impl)
     Running `rustc --edition=2018 --crate-name basic tests/basic.rs --color always --emit=dep-info,link -C debuginfo=2 --test -C metadata=4c780bcd93c95670 -C extra-filename=-4c780bcd93c95670 --out-dir /home/sofe/tmp/rust-macro-panic/delegate_impl/target/debug/deps -C incremental=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/debug/incremental -L dependency=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/debug/deps --extern darling=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/debug/deps/libdarling-1cf0f95a99f4d219.rlib --extern delegate_impl=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/debug/deps/libdelegate_impl-9e2dd1dcd849a18f.so --extern heck=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/debug/deps/libheck-dd3bc44162558753.rlib --extern proc_macro2=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/debug/deps/libproc_macro2-1e51e1908d73493d.rlib --extern quote=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/debug/deps/libquote-5c27d0a83206aa3b.rlib --extern syn=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/debug/deps/libsyn-26e5871ced9ec69a.rlib`
pub trait Foo { fn len (& self) -> usize ; } macro_rules ! delegate_foo
{
    ($ (< $ ($ types : ty) , * > :) ? $ (< $ ($ trait_types : ty) , * > for) ?
     $ struct : ty : $ inner : ident ; $ ($ extra : tt) *) =>
    {
        impl Foo for $ struct
        { fn len (& self) -> usize { self . $ inner . len () } $ ($ extra) * }
    } ;
}
thread 'rustc' panicked at 'assume all sequences have (unique) spans for now', src/libsyntax/ext/tt/macro_rules.rs:694:29
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/git.luolix.top-1ecc6299db9ec823/backtrace-0.3.29/src/backtrace/libunwind.rs:88
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/git.luolix.top-1ecc6299db9ec823/backtrace-0.3.29/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:47
   3: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:36
   4: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:200
   5: std::panicking::default_hook
             at src/libstd/panicking.rs:214
   6: rustc::util::common::panic_hook
   7: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:481
   8: std::panicking::begin_panic
   9: syntax::ext::tt::macro_rules::check_matcher_core::{{closure}}
  10: syntax::ext::tt::macro_rules::check_matcher_core
  11: syntax::ext::tt::macro_rules::check_lhs_nt_follows
  12: <core::iter::adapters::Map<I,F> as core::iter::traits::iterator::Iterator>::fold
  13: <alloc::vec::Vec<T> as alloc::vec::SpecExtend<T,I>>::from_iter
  14: syntax::ext::tt::macro_rules::compile
  15: rustc_resolve::macros::<impl rustc_resolve::Resolver>::define_macro
  16: syntax::ext::expand::AstFragment::visit_with
  17: rustc_resolve::macros::<impl syntax::ext::base::Resolver for rustc_resolve::Resolver>::visit_ast_fragment_with_placeholders
  18: syntax::ext::expand::MacroExpander::collect_invocations
  19: syntax::ext::expand::MacroExpander::expand_fragment
  20: syntax::ext::expand::MacroExpander::expand_crate
  21: rustc_interface::passes::configure_and_expand_inner::{{closure}}
  22: rustc::util::common::time
  23: rustc_interface::passes::configure_and_expand_inner
  24: rustc_interface::passes::configure_and_expand::{{closure}}
  25: rustc_data_structures::box_region::PinnedGenerator<I,A,R>::new
  26: rustc_interface::passes::configure_and_expand
  27: rustc_interface::queries::Query<T>::compute
  28: rustc_interface::queries::<impl rustc_interface::interface::Compiler>::expansion
  29: rustc_interface::interface::run_compiler_in_existing_thread_pool
  30: std::thread::local::LocalKey<T>::with
  31: syntax::with_globals
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
query stack during panic:
end of query stack

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.38.0-nightly (e3cebcb3b 2019-07-19) running on x86_64-unknown-linux-gnu

note: compiler flags: -C debuginfo=2 -C incremental

note: some of the compiler flags provided by cargo are hidden

error: Could not compile `delegate-impl`.

Caused by:
  process didn't exit successfully: `rustc --edition=2018 --crate-name basic tests/basic.rs --color always --emit=dep-info,link -C debuginfo=2 --test -C metadata=4c780bcd93c95670 -C extra-filename=-4c780bcd93c95670 --out-dir /home/sofe/tmp/rust-macro-panic/delegate_impl/target/debug/deps -C incremental=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/debug/incremental -L dependency=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/debug/deps --extern darling=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/debug/deps/libdarling-1cf0f95a99f4d219.rlib --extern delegate_impl=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/debug/deps/libdelegate_impl-9e2dd1dcd849a18f.so --extern heck=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/debug/deps/libheck-dd3bc44162558753.rlib --extern proc_macro2=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/debug/deps/libproc_macro2-1e51e1908d73493d.rlib --extern quote=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/debug/deps/libquote-5c27d0a83206aa3b.rlib --extern syn=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/debug/deps/libsyn-26e5871ced9ec69a.rlib` (exit code: 101)

Using --release gives a similar result:

# RUST_BACKTRACE=1 cargo test --release -v
       Fresh unicode-xid v0.1.0
       Fresh fnv v1.0.6
       Fresh strsim v0.7.0
       Fresh ident_case v1.0.1
       Fresh unicode-segmentation v1.3.0
       Fresh proc-macro2 v0.4.30
       Fresh heck v0.3.1
       Fresh quote v0.6.13
       Fresh syn v0.15.39
       Fresh darling_core v0.9.0
       Fresh darling_macro v0.9.0
       Fresh darling v0.9.0
   Compiling delegate-impl v0.1.0 (/home/sofe/tmp/rust-macro-panic/delegate_impl)
     Running `rustc --edition=2018 --crate-name basic tests/basic.rs --color always --emit=dep-info,link -C opt-level=3 --test -C metadata=9e9bf208733a178a -C extra-filename=-9e9bf208733a178a --out-dir /home/sofe/tmp/rust-macro-panic/delegate_impl/target/release/deps -L dependency=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/release/deps --extern darling=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/release/deps/libdarling-0afbc4e9dd403117.rlib --extern delegate_impl=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/release/deps/libdelegate_impl-af9dd6baa1beb54b.so --extern heck=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/release/deps/libheck-d871e96a3d2eb560.rlib --extern proc_macro2=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/release/deps/libproc_macro2-2ff74a27828d9340.rlib --extern quote=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/release/deps/libquote-c099edfd78a33426.rlib --extern syn=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/release/deps/libsyn-42c6f8f520bc6764.rlib`
pub trait Foo { fn len (& self) -> usize ; } macro_rules ! delegate_foo
{
    ($ (< $ ($ types : ty) , * > :) ? $ (< $ ($ trait_types : ty) , * > for) ?
     $ struct : ty : $ inner : ident ; $ ($ extra : tt) *) =>
    {
        impl Foo for $ struct
        { fn len (& self) -> usize { self . $ inner . len () } $ ($ extra) * }
    } ;
}
thread 'rustc' panicked at 'assume all sequences have (unique) spans for now', src/libsyntax/ext/tt/macro_rules.rs:694:29
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/git.luolix.top-1ecc6299db9ec823/backtrace-0.3.29/src/backtrace/libunwind.rs:88
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/git.luolix.top-1ecc6299db9ec823/backtrace-0.3.29/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:47
   3: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:36
   4: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:200
   5: std::panicking::default_hook
             at src/libstd/panicking.rs:214
   6: rustc::util::common::panic_hook
   7: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:481
   8: std::panicking::begin_panic
   9: syntax::ext::tt::macro_rules::check_matcher_core::{{closure}}
  10: syntax::ext::tt::macro_rules::check_matcher_core
  11: syntax::ext::tt::macro_rules::check_lhs_nt_follows
  12: <core::iter::adapters::Map<I,F> as core::iter::traits::iterator::Iterator>::fold
  13: <alloc::vec::Vec<T> as alloc::vec::SpecExtend<T,I>>::from_iter
  14: syntax::ext::tt::macro_rules::compile
  15: rustc_resolve::macros::<impl rustc_resolve::Resolver>::define_macro
  16: syntax::ext::expand::AstFragment::visit_with
  17: rustc_resolve::macros::<impl syntax::ext::base::Resolver for rustc_resolve::Resolver>::visit_ast_fragment_with_placeholders
  18: syntax::ext::expand::MacroExpander::collect_invocations
  19: syntax::ext::expand::MacroExpander::expand_fragment
  20: syntax::ext::expand::MacroExpander::expand_crate
  21: rustc_interface::passes::configure_and_expand_inner::{{closure}}
  22: rustc::util::common::time
  23: rustc_interface::passes::configure_and_expand_inner
  24: rustc_interface::passes::configure_and_expand::{{closure}}
  25: rustc_data_structures::box_region::PinnedGenerator<I,A,R>::new
  26: rustc_interface::passes::configure_and_expand
  27: rustc_interface::queries::Query<T>::compute
  28: rustc_interface::queries::<impl rustc_interface::interface::Compiler>::expansion
  29: rustc_interface::interface::run_compiler_in_existing_thread_pool
  30: std::thread::local::LocalKey<T>::with
  31: syntax::with_globals
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
query stack during panic:
end of query stack

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.38.0-nightly (e3cebcb3b 2019-07-19) running on x86_64-unknown-linux-gnu

note: compiler flags: -C opt-level=3

note: some of the compiler flags provided by cargo are hidden

error: Could not compile `delegate-impl`.

Caused by:
  process didn't exit successfully: `rustc --edition=2018 --crate-name basic tests/basic.rs --color always --emit=dep-info,link -C opt-level=3 --test -C metadata=9e9bf208733a178a -C extra-filename=-9e9bf208733a178a --out-dir /home/sofe/tmp/rust-macro-panic/delegate_impl/target/release/deps -L dependency=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/release/deps --extern darling=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/release/deps/libdarling-0afbc4e9dd403117.rlib --extern delegate_impl=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/release/deps/libdelegate_impl-af9dd6baa1beb54b.so --extern heck=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/release/deps/libheck-d871e96a3d2eb560.rlib --extern proc_macro2=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/release/deps/libproc_macro2-2ff74a27828d9340.rlib --extern quote=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/release/deps/libquote-c099edfd78a33426.rlib --extern syn=/home/sofe/tmp/rust-macro-panic/delegate_impl/target/release/deps/libsyn-42c6f8f520bc6764.rlib` (exit code: 101)

You can try removing the first one or two parts in the macro pattern and the error disappears.
@jonas-schievink jonas-schievink added C-bug Category: This is a bug. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. A-macros Area: All kinds of macros (custom derive, macro_rules!, proc macros, ..) labels Jul 20, 2019
@Centril
Copy link
Contributor

Centril commented Jul 21, 2019

cc @petrochenkov

@petrochenkov
Copy link
Contributor

petrochenkov commented Jul 21, 2019 •
edited
Loading

cc @mark-i-m @ia0

@mark-i-m
Copy link
Member

The offending code is here:

rust/src/libsyntax/ext/tt/macro_rules.rs

Lines 653 to 655 in 299ef86

Some(&None) => {
panic!("assume all sequences have (unique) spans for now");
}

TBH, I'm not really sure what it is checking, and I don't have time to look into it right now (work is busy)... Sorry about that.

It seems that the proc_macro is somehow generating some tokens or TokenTrees (especially in a Sequence [i.e. a token repetition]) that don't have spans somehow. @petrochenkov is that even possible?

It looks like pnkfelix wrote this code originally 4 years ago. It hasn't been touched in years. @petrochenkov do you know if there have been any changes in proc_macros that could have exposed this behavior?

@ia0 ia0 mentioned this issue Jul 24, 2019
Merged
ia0 added a commit to ia0/rust that referenced this issue Jul 25, 2019
@ia0
Implement slow-path for FirstSets::first
df4b23e 
When 2 or more sequences share the same span, we can't use the precomputed map
for their first set. So we compute it recursively.

Fixes rust-lang#62831.
Centril added a commit to Centril/rust that referenced this issue Jul 26, 2019
@Centril
Rollup merge of rust-lang#62956 - ia0:fix_62831, r=petrochenkov
58fa03a 
Implement slow-path for FirstSets::first

When 2 or more sequences share the same span, we can't use the precomputed map
for their first set. So we compute it recursively.

Fixes rust-lang#62831.
Centril added a commit to Centril/rust that referenced this issue Jul 26, 2019
@Centril
Rollup merge of rust-lang#62956 - ia0:fix_62831, r=petrochenkov
5ef908a 
Implement slow-path for FirstSets::first

When 2 or more sequences share the same span, we can't use the precomputed map
for their first set. So we compute it recursively.

Fixes rust-lang#62831.
Centril added a commit to Centril/rust that referenced this issue Jul 26, 2019
@Centril
Rollup merge of rust-lang#62956 - ia0:fix_62831, r=petrochenkov
c6c8693 
Implement slow-path for FirstSets::first

When 2 or more sequences share the same span, we can't use the precomputed map
for their first set. So we compute it recursively.

Fixes rust-lang#62831.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-macros Area: All kinds of macros (custom derive, macro_rules!, proc macros, ..) C-bug Category: This is a bug. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Implement slow-path for FirstSets::first ia0/rust
5 participants
@Centril @jonas-schievink @petrochenkov @mark-i-m @SOF3