Feature request: get access to the exact text source (&str) behind a Span

[ogoffart]

In order to be able to upgrade the syn dependency of the rust-cpp crate ( mystor/rust-cpp#41 ), one would need to be able to extract the raw source code for a given TokenTree (or a Span)

One could do that, I guess, by using TokenTree::span, and then getting the start/end and source_file of the span. But these function are "semver exempt". Even if they were not, that would mean we would need to recompute the offset in the file from line/column, which would be inefficient.

So what would be nice would be an start_offset and end_offset within Span.
Or better, a function that would return a String or a &str with the exact spelling.

TokenStream::to_string() does not work, as it adds spaces between token (that makes the resulting string invalid C++ as the two languages don't have the same grammar).
This is useful for crates, like rust-cpp, that allow to embed arbitrary languages within a rust macro (as long as the [], () and {} are balanced))

[alexcrichton]

Seems like a plausible feature to me! To implement in this library though it needs to be implemented in upstream rustc first

[roblabla]

I am also interested in this. Is there an upstream ticket in rustc?

[alexcrichton]

I think rust-lang/rust#54725 is the closest

[idubrov]

Access to offset would also help my case where I use proc_macro2+syn completely outside of the compiler and I want to be able to get. Currently it seems I'll have to recover them from line+number (which is a shame since internally these are offsets in "fallback" implementation).

I'm probably using the tool the wrong way, though, not sure how useful my use-case is 😅

[ogoffart]

Yes, my use case is also unrelated to proc_macro: In the cpp_build crate. The use case is to extract the cpp! macro from the build.rs script.

Anyway, even upstream now has a way to get the source text: https://doc.rust-lang.org/proc_macro/struct.Span.html#method.source_text

[rust-shell-script]

It would be really useful for creating DSL inside rust language itself. I saw people use all kinds of tricks to get the source code (proc-macro-faithful-display, manipulating stringify!, heuristic string with line/column clue ...), I also created a hack macro to achieve the same goal.

I hope it can come along with a simple API, so I can make the below code work without much hack any more:

if run_cmd! {
    ls / | wc -w
    echo "bad cmd"
    ls -l /nofile
    date
}.is_err() {
    warn!("Run group command failed");
}

[dtolnay]

Closing in favor of consolidating the discussion to rust-lang/rust#54725.