Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Span::resolved_at and Span::located_at to combine behavior of two spans #47149

Merged
merged 1 commit into from
Jan 5, 2018

Conversation

dtolnay
Copy link
Member

@dtolnay dtolnay commented Jan 3, 2018

Proc macro spans serve two mostly unrelated purposes: controlling name resolution and controlling error messages. It can be useful to mix the name resolution behavior of one span with the line/column error message locations of a different span.

In particular, consider the case of a trait brought into scope within the def_site of a custom derive. I want to invoke trait methods on the fields of the user's struct. If the field type does not implement the right trait, I want the error message to underline the corresponding struct field.

Generating the method call with the def_site span is not ideal -- it compiles and runs but error messages sadly always point to the derive attribute like we saw with Macros 1.1.

  |
4 | #[derive(HeapSize)]
  |          ^^^^^^^^

Generating the method call with the same span as the struct field's ident or type is not correct -- it shows the right underlines but fails to resolve to the trait in scope at the def_site.

  |
7 |     bad: std::thread::Thread,
  |     ^^^^^^^^^^^^^^^^^^^^^^^^

The correct span for the method call is one that combines the def_site's name resolution with the struct field's line/column.

field.span.resolved_at(Span::def_site())

// equivalently
Span::def_site().located_at(field.span)

Adding both because which one is more natural will depend on context.

Addresses #38356 (comment). r? @jseyfried

Proc macro spans serve two mostly unrelated purposes: controlling name
resolution and controlling error messages. It can be useful to mix the
name resolution behavior of one span with the line/column error message
locations of a different span.

In particular, consider the case of a trait brought into scope within
the def_site of a custom derive. I want to invoke trait methods on the
fields of the user's struct. If the field type does not implement the
right trait, I want the error message to underline the corresponding
struct field.

Generating the method call with the def_site span is not ideal -- it
compiles and runs but error messages sadly always point to the derive
attribute like we saw with Macros 1.1.

```
  |
4 | #[derive(HeapSize)]
  |          ^^^^^^^^
```

Generating the method call with the same span as the struct field's
ident or type is not correct -- it shows the right underlines but fails
to resolve to the trait in scope at the def_site.

```
  |
7 |     bad: std::thread::Thread,
  |     ^^^^^^^^^^^^^^^^^^^^^^^^
```

The correct span for the method call is one that combines the def_site's
name resolution with the struct field's line/column.

```
field.span.resolved_at(Span::def_site())

// equivalently
Span::def_site().located_at(field.span)
```

Adding both because which one is more natural will depend on context.
@SergioBenitez
Copy link
Contributor

SergioBenitez commented Jan 3, 2018

I don't understand how/when you're supposed to use these methods; the comments attached to them aren't very helpful. In particular, what do you mean by (emphasis mine):

Generating the method call with the same span as the struct field's ident or type is not correct -- it shows the right underlines but fails to resolve to the trait in scope at the def_site.

What is failing to resolve which trait where, why should it, and why does it matter?

Perhaps you could add some ui-tests that clearly show what these methods let you do that you couldn't do before?

@dtolnay
Copy link
Member Author

dtolnay commented Jan 3, 2018

@SergioBenitez check out #45934 for some background. The span of the tokens in a method call or type name or basically any other Term determine whether it is supposed to refer to something in scope where the macro is called or to things defined by the proc macro. By default, if my custom derive parses an input struct then the tokens in that struct resolve to types that are in scope where the macro is called -- which makes sense because in something like #[derive(HeapSize)] struct Sergio { s: String } my proc macro is not responsible for defining what String means. In contrast, tokens that originate from quote!(...) inside of my custom derive are by default only able to refer to things defined by my proc macro -- with some nuance discussed in #45934.

Generating the method call with the same span as the struct field's ident or type is not correct -- it shows the right underlines but fails to resolve to the trait in scope at the def_site.

What is failing to resolve which trait where, why should it, and why does it matter?

  • What is failing to resolve: the method call emitted by the expansion of my custom derive, for example the heap_size() in self.s.heap_size().
  • Which trait is it resolving: the trait containing the method with that signature, brought into scope by the expansion of my custom derive such as by doing extern crate heapsize; use self::heapsize::HeapSize; in the generated code.
  • Where is resolution happening: my custom derive spits out some tokens and each token carries a span, after that it is the compiler's job to figure out what those tokens mean as Rust source code.
  • Why should it resolve: because I want to hygienically invoke a particular known trait method without caring about what crazy things are in scope at the call site.
  • Why does it matter that this fails to resolve: if my custom derive uses the struct field's ident's span for the heap_size() in self.s.heap_size() and that results in the generated code not compiling, then that means my custom derive cannot use the struct field's ident's span for the heap_size() in self.s.heap_size(); this is because my custom derive wants to emit code that compiles.

@dtolnay
Copy link
Member Author

dtolnay commented Jan 3, 2018

Regarding documentation of call_site vs def_site and other things one would need to know about hygiene in proc macros, I believe that is part of what is being tracked in #38356 and will be a blocker for stabilization.

@SergioBenitez
Copy link
Contributor

SergioBenitez commented Jan 3, 2018

While I'm aware of how Spans are used for hygiene/resolution purposes, I wasn't sure what names were failing to resolve in your first comment. I came across dtolnay/syn#290, however, and started piecing things together. Coupled with:

What is failing to resolve: the method call emitted by the expansion of my custom derive, for example the heap_size() in self.s.heap_size().

I now understand the issue. In effect, you're trying to emit names that you want Rust to resolve as if they were from the definition site. In particular, a method call that requires a trait to be in scope.

Why can't you generate a use MyTrait; from the proc-macro using the Span from the trait in the input? Wouldn't that resolve this issue and provide the correct spans? It looks like you did something similar in dtolnay/syn#290.

Also, I'm curious about this:

Generating the method call with the same span as the struct field's ident or type is not correct -- it shows the right underlines but fails to resolve to the trait in scope at the def_site.

Shouldn't the fields have the same name resolution as def_site() since they are, in fact, at the definition site?

@dtolnay
Copy link
Member Author

dtolnay commented Jan 3, 2018

Why can't you generate a use MyTrait; from the proc-macro using the Span from the trait in the input? Wouldn't that resolve this issue and provide the correct spans?

There is already a use MyTrait being generated, and its span needs to be def_site because we want this to work regardless of what the MyTrait in scope at the call site happens to be. I do not believe that altering the spans of use MyTrait can address the issue.

Shouldn't the fields have the same name resolution as def_site() since they are, in fact, at the definition site?

The fields in the input to a custom derive are contained within the input to the custom derive and so they come from the call site. This is why in #[derive(...)] struct Sergio { s: String } the String refers hygienically to whatever String is in scope at the call site.

@kennytm kennytm added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jan 3, 2018
@dtolnay
Copy link
Member Author

dtolnay commented Jan 3, 2018

Discussed in IRC -- I understand Sergio's comments and concerns better after confirming that he had his def_site and call_site mixed up.

@SergioBenitez
Copy link
Contributor

SergioBenitez commented Jan 3, 2018

Yes, my apologies. I was mixing up def_site and call_site.

I was also under the impression that quote! replaced the resolution context of all tokens with def_site() for hygiene purposes. As a result, you'd be able to do the following and get the behavior you're looking for:

let trait_call = quote_span!(field.span, #field.method_call());
return quote!(#trait_call);

Thus, trait_call would have the positional information from field.span but resolution context of def_site().

Instead, however, quote! keeps all of the existing span information for any tokens and sets the span of any span-less tokens to def_site. Thus we see the two unwanted outcomes that @dtolnay showcased:

// `method_call()` incorrectly resolves at call site, correctly has field span position
let trait_call = quote_span!(field.span, #field.method_call());
return quote!(#trait_call);

// `method_call()` correctly resolves at def site, incorrectly has def site span position
return quote!(#field.method_call());

Is there a reason that quote! doesn't set the resolution context of all spans to def_site()? (assuming resolution is limited to items) Except for purposefully circumventing hygiene, I can't think of a good reason why this wouldn't be desired. Furthermore, we could also look into having a quote_contextual!(span, ...) that sets the resolution context of all tokens being quoted to that of span.

@dtolnay dtolnay mentioned this pull request Jan 3, 2018
@jseyfried
Copy link
Contributor

@bors r+

@bors
Copy link
Contributor

bors commented Jan 3, 2018

📌 Commit 000e907 has been approved by jseyfried

@jseyfried
Copy link
Contributor

jseyfried commented Jan 3, 2018

@SergioBenitez

Is there a reason that quote! doesn't set the resolution context of all spans to def_site()?

Yeah, if the interpolated tokens come from the macro invocation, you typically don't want to change how they resolve. E.g. say I have a macro that passes its input to a function:

#[proc_macro]
fn m(input: TokenStream) -> TokenStream {
    quote! {
        extern crate my_crate;
        my_crate::my_function($input);
    }
}

Here, I want my_crate to have def site span so it doesn't conflict with anything, but I want the interpolated tokens from $input to resolve at the call site so the macro invocation resolves like the corresponding direct function call.

kennytm added a commit to kennytm/rust that referenced this pull request Jan 4, 2018
Span::resolved_at and Span::located_at to combine behavior of two spans

Proc macro spans serve two mostly unrelated purposes: controlling name resolution and controlling error messages. It can be useful to mix the name resolution behavior of one span with the line/column error message locations of a different span.

In particular, consider the case of a trait brought into scope within the def_site of a custom derive. I want to invoke trait methods on the fields of the user's struct. If the field type does not implement the right trait, I want the error message to underline the corresponding struct field.

Generating the method call with the def_site span is not ideal -- it compiles and runs but error messages sadly always point to the derive attribute like we saw with Macros 1.1.

```
  |
4 | #[derive(HeapSize)]
  |          ^^^^^^^^
```

Generating the method call with the same span as the struct field's ident or type is not correct -- it shows the right underlines but fails to resolve to the trait in scope at the def_site.

```
  |
7 |     bad: std::thread::Thread,
  |     ^^^^^^^^^^^^^^^^^^^^^^^^
```

The correct span for the method call is one that combines the def_site's name resolution with the struct field's line/column.

```rust
field.span.resolved_at(Span::def_site())

// equivalently
Span::def_site().located_at(field.span)
```

Adding both because which one is more natural will depend on context.

Addresses rust-lang#38356 (comment). r? @jseyfried
kennytm added a commit to kennytm/rust that referenced this pull request Jan 5, 2018
Span::resolved_at and Span::located_at to combine behavior of two spans

Proc macro spans serve two mostly unrelated purposes: controlling name resolution and controlling error messages. It can be useful to mix the name resolution behavior of one span with the line/column error message locations of a different span.

In particular, consider the case of a trait brought into scope within the def_site of a custom derive. I want to invoke trait methods on the fields of the user's struct. If the field type does not implement the right trait, I want the error message to underline the corresponding struct field.

Generating the method call with the def_site span is not ideal -- it compiles and runs but error messages sadly always point to the derive attribute like we saw with Macros 1.1.

```
  |
4 | #[derive(HeapSize)]
  |          ^^^^^^^^
```

Generating the method call with the same span as the struct field's ident or type is not correct -- it shows the right underlines but fails to resolve to the trait in scope at the def_site.

```
  |
7 |     bad: std::thread::Thread,
  |     ^^^^^^^^^^^^^^^^^^^^^^^^
```

The correct span for the method call is one that combines the def_site's name resolution with the struct field's line/column.

```rust
field.span.resolved_at(Span::def_site())

// equivalently
Span::def_site().located_at(field.span)
```

Adding both because which one is more natural will depend on context.

Addresses rust-lang#38356 (comment). r? @jseyfried
bors added a commit that referenced this pull request Jan 5, 2018
Rollup of 10 pull requests

- Successful merges: #47030, #47033, #47110, #47149, #47150, #47160, #47162, #47182, #47198, #47199
- Failed merges:
@bors bors merged commit 000e907 into rust-lang:master Jan 5, 2018
@dtolnay dtolnay deleted the spans branch May 26, 2018 02:46
Dylan-DPC-zz pushed a commit to Dylan-DPC-zz/rust that referenced this pull request Apr 25, 2020
proc_macro: Stabilize `Span::resolved_at` and `Span::located_at`

Introduced in rust-lang#47149.
Part of rust-lang#54725.

Motivation: rust-lang#68716 (comment).
Identifiers in proc macros may want to inherit span locations for diagnostics from one tokens (e.g. some tokens from the macro input), but resolve those identifiers from some different location (e.g. from the macro's definition site).
This becomes especially important when multiple resolution locations become available with stabilization of [`Span::mixed_site`](rust-lang#68716).

Why I think this is the right API for setting span's location and hygiene - rust-lang#69041 (comment).

r? @dtolnay
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Awaiting review from the assignee but also interested parties.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants