-
Notifications
You must be signed in to change notification settings - Fork 12.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Embed MSVC .natvis files into .pdbs and mangle debuginfo for &str, *T, and [T]. #43221
Conversation
Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @eddyb (or someone else) soon. If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes. Please see the contribution instructions for more information. |
Thanks so much for the PR @MaulingMonkey! I can't speak to the content, but I must say that that is a top-notch commit message! We'll get someone knowledgable to review this shortly! |
r? @brson I think |
Thanks a lot for the PR, @MaulingMonkey! This looks like a great contribution!
At a first glance, this looks like the right place.
Staying MSVC specific for this is the way to go, I'd say.
You can worry about that in another PR
Yeah, they are in conflict with rustc's current code style guide.
As long as this is limited to MSVC and that still works as good or better than before, this looks good to me.
There are no tests for MSVC debugging yet. There might be a handful of other tests that depend debuginfo type names, but CI will tell us about those soon enough. Now for a question of mine: Is there a way to transform the mangled name back to something more Rust-like when they are displayed? |
Thanks for the kind words all!
Okay, I've got a followup changelist fixing this, that I've got the build and test scripts chewing through. Any other style issues while I'm in there? I'm also inclined to rename, comment on, and move this to the top of // When targeting MSVC, emit C++ style type names for compatability with
// .natvis visualizers (and perhaps other existing native debuggers?)
let cpp_like_names = cx.sess().target.target.options.is_like_msvc;
The only regression I can think of is if someone built with MSVC but then wanted to debug with GDB instead for some reason (debugging binaries that someone else built?)
Not that I'm aware of, short of writing a full blown debug engine (e.g. maybe on VisualRust's far future roadmap?) Such a .natvis feature would be useful to unexpand C++ templates too, if it were an option (e.g. |
This is already incredibly unsupported and doesn't work due to vastly different debuginfo formats. I wouldn't be worried about this. |
src/librustc_trans/back/linker.rs
Outdated
if let Some(natvis_path_str) = path.to_str() { | ||
self.cmd.arg(&format!("/NATVIS:{}",natvis_path_str)); | ||
} else { | ||
self.sess.warn(&format!("natvis path not unicode: {:?}", path)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Command::arg()
should be able to take a OsStr
. You probably can't use format!
with that but you could build up the OsString
with its push
method.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch! Testing a patch now.
Didn't see any. If you support non-unicode paths too (see comment) then I'd be happy to r+ this. |
Good catch - using traits for "overloading" was new to me, so I'd mistakenly assumed |
@MaulingMonkey: Have you tested the string visualizers with non-ASCII strings? Since Rust strings are defined to be UTF-8, we should use the <Type Name="str">
<DisplayString>{data_ptr,[length]s8}</DisplayString>
<StringView>data_ptr,[length]s8</StringView>
... I found out about this some time ago, but the official documentation is sparse unfortunately. It's listed on https://msdn.microsoft.com/en-us/library/75w45ekt.aspx, but it doesn't say that it means UTF-8. It does say, however, that |
I have now. With your suggested changes, I grabbed https://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-demo.txt . Breakpointing slice2 of: use std::io::BufReader;
use std::io::BufRead;
use std::fs::File;
use std::path::Path;
fn main() {
let f = File::open("I:\\home\\projects\\test\\rust\\UTF-8-demo.txt").unwrap();
let mut file = BufReader::new(&f);
for line in file.lines() {
let line = line.unwrap();
let slice : &str = &line;
let slice2 = slice;
println!("{}", slice);
}
} And stepping to this line:
Shows me the following with the The combining mark for the second T in " STARGΛ̊TE" is also misplaced to the left of the T in the Value column, but not in the full blown Text Visualizer. Then again, Chrome also shows it misplaced to the left of the T in this Chrome tab, but not in the original text file location, so this might just be more font issues? I can roll those changes into this pull request. |
One caveat is that the expanded array representation is still individual ASCII-ish characters - ArrayItems/ValuePointer will just plain ignore the postfix if added - but that might be a feature anyways. |
Just as an addendum, I found the link where |
ping @michaelwoerister for a follow-up review, this looks ready to go! |
@bors r+ |
📌 Commit 90a7cac has been approved by |
…erister Embed MSVC .natvis files into .pdbs and mangle debuginfo for &str, *T, and [T]. No idea if these changes are reasonable - please feel free to suggest changes/rewrites. And these are some of my first real commits to any rust codebase - *don't* be gentle, and nitpick away, I need to learn! ;) ### Overview Embedding `.natvis` files into `.pdb`s allows MSVC (and potentially other debuggers) to automatically pick up the visualizers without having to do any additional configuration (other than to perhaps add the relevant .pdb paths to symbol search paths.) The native debug engine for MSVC parses the type names, making various C++ish assumptions about what they mean and adding various limitations to valid type names. `&str` cannot be matched against a visualizer, but if we emit `str&` instead, it'll be recognized as a reference to a `str`, solving the problem. `[T]` is similarly problematic, but emitting `slice<T>` instead works fine as it looks like a template. I've been unable to get e.g. `slice<u32>&` to match visualizers in VS2015u3, so I've gone with `str*` and `slice<u32>*` instead. ### Possible Issues * I'm not sure if `slice<T>` is a great mangling for `[T]` or if I should worry about name collisions. * I'm not sure if `linker.rs` is the right place to be enumerating natvis files. * I'm not sure if these type name mangling changes should actually be MSVC specific. I recall seeing gdb visualizer tests that might be broken if made more general? I'm hesitant to mess with them without a gdb install. But perhaps I'm just wracking up technical debt. Should I try `pacman -S mingw-w64-x86_64-gdb` and to make things consistent? * I haven't touched `const` / `mut` yet, and I'm worried MSVC might trip up on `mut` or their placement. * I may like terse oneliners too much. * I don't know if there's broader implications for messing with debug type names here. * I may have been mistaken about bellow test failures being ignorable / unrelated to this changelist. ### Test Failures on `x86_64-pc-windows-gnu` ``` ---- [debuginfo-gdb] debuginfo-gdb\associated-types.rs stdout ---- thread '[debuginfo-gdb] debuginfo-gdb\associated-types.rs' panicked at 'gdb not available but debuginfo gdb debuginfo test requested', src\tools\compiletest\src\runtest.rs:48:16 note: Run with `RUST_BACKTRACE=1` for a backtrace. [...identical panic causes omitted...] ---- [debuginfo-gdb] debuginfo-gdb\vec.rs stdout ---- thread '[debuginfo-gdb] debuginfo-gdb\vec.rs' panicked at 'gdb not available but debuginfo gdb debuginfo test requested', src\tools\compiletest\src\runtest.rs:48:16 ``` ### Relevant Issues * #40460 Metaissue for Visual Studio debugging Rust * #36503 Investigate natvis for improved msvc debugging * PistonDevelopers/VisualRust#160 Debug visualization of Rust data structures ### Pretty Pictures ![Collapsed Watch Window](https://user-images.githubusercontent.com/75894/28180998-e44c7516-67bb-11e7-8b48-d4f9605973ae.png) ![Expanded Watch Window](https://user-images.githubusercontent.com/75894/28181000-e8da252e-67bb-11e7-96b8-d613310c04dc.png)
☀️ Test successful - status-appveyor, status-travis |
Thanks again, @MaulingMonkey, for kicking this off! |
My pleasure. Thanks for the review, @michaelwoerister ! And thanks @Boddlnagg (and more!) for cluing me into the |
For the record, this breaks people trying to use LLD with msvc because LLD doesn't yet understand |
Does |
Currently I'm just passing |
Possible temporary workarounds to unblock your testing:
I imagine Fixing this on LLD's end might be as simple as adding If I do take a stab at a workaround on rustc's end, would http://releases.llvm.org/5.0.0/LLVM-5.0.0-win64.exe work for testing, or have you installed LLVM via another method that I should prefer? |
A similar patch for Tempted to find someone who's already set up for committing to/testing LLVM - perhaps on one of their mailing lists if nobody here already is - a job has eaten my free time. Maybe the pcc fellow who implemented https://reviews.llvm.org/D37607 ? |
I went with the former workaround. I don't know why anyone would use the latter workaround considering how horrifying LLD does support emitting debuginfo now (http://blog.llvm.org/2017/08/llvm-on-windows-now-supports-pdb-debug.html). It's definitely a lot closer to the point of being a legitimate alternative. I would prefer LLD supporting the flag or at least ignoring it and not erroring. But until then it would probably be better to not pass That is a valid way to acquire LLVM, although I personally use http://llvm.org/builds/ to get more bleeding edge builds. |
I've created a potential workaround patch for rustc, although I can't yet test it as I've also created a patch for lld-link which works, now I just need to hand it off for submission somehow. EDIT: I've sent this off to llvm-commits@lists.llvm.org as
although it's been held in moderation thanks to screwing up my email aliases. |
LLVM patch landed: Still stuck in trying to resolve build hell for the rustc patch. |
Awesome, thanks @MaulingMonkey! |
I've updated https://github.com/MaulingMonkey/rust/tree/lld-link-natvis-regression-fix which at least builds on I'll submit a pull request once any of the following happen:
|
The massive amounts of logs are due to a |
@retep998 thanks for the link - I was able to merge that for testing before it landed in trunk. Work has eaten a lot of my free time, but I've managed to cobble together a pull request that actually works when tested, not just works in my head 😄 |
…ssion-fix, r=michaelwoerister Skip passing /natvis to lld-link until supported. ### Overview Teaching rustc about MSVC's undocumented linker flag, /NATVIS, broke rustc's compatability with LLVM's `lld-link` frontend, as it does not recognize the flag. This pull request works around the problem by excluding `lld-link` by name. @retep998 discovered this regression. ### Possible Issues - Other linkers that try to be compatible with the MSVC linker flavor may also be broken and in need of workarounds. - Warning about the workaround may be overkill for a minor reduction in debug functionality. - Depending on how long this workaround sticks around, it may eventually be preferred to version check `lld-link` instead of assuming all versions are incompatible. ### Relevant issues * Broke in rust-lang#43221 Embed MSVC .natvis files into .pdbs and mangle debuginfo for &str, *T, and [T]. * LLVM patched in llvm-mirror/lld@27b9c42 to ignore the flag instead of erroring. r? @michaelwoerister
No idea if these changes are reasonable - please feel free to suggest changes/rewrites. And these are some of my first real commits to any rust codebase - don't be gentle, and nitpick away, I need to learn! ;)
Overview
Embedding
.natvis
files into.pdb
s allows MSVC (and potentially other debuggers) to automatically pick up the visualizers without having to do any additional configuration (other than to perhaps add the relevant .pdb paths to symbol search paths.)The native debug engine for MSVC parses the type names, making various C++ish assumptions about what they mean and adding various limitations to valid type names.
&str
cannot be matched against a visualizer, but if we emitstr&
instead, it'll be recognized as a reference to astr
, solving the problem.[T]
is similarly problematic, but emittingslice<T>
instead works fine as it looks like a template. I've been unable to get e.g.slice<u32>&
to match visualizers in VS2015u3, so I've gone withstr*
andslice<u32>*
instead.Possible Issues
slice<T>
is a great mangling for[T]
or if I should worry about name collisions.linker.rs
is the right place to be enumerating natvis files.Should I try
pacman -S mingw-w64-x86_64-gdb
and to make things consistent?const
/mut
yet, and I'm worried MSVC might trip up onmut
or their placement.Test Failures on
x86_64-pc-windows-gnu
Relevant Issues
Pretty Pictures