-Zmiri-track-pointer-tag=untagged fails (ICE)

[ssbr]

I'd like to be able to track untagged when debugging, but currently I think the only way to do this is -Zmiri-track-raw-pointers.

Code

fn main() {}

Meta

rustc --version --verbose:

rustc 1.56.0-nightly (a49e38e67 2021-08-23)
binary: rustc
commit-hash: a49e38e672c60da788360e088f00ad12353e3913
commit-date: 2021-08-23
host: x86_64-unknown-linux-gnu
release: 1.56.0-nightly
LLVM version: 13.0.0

Error output

$ MIRIFLAGS='-Zmiri-track-pointer-tag=untagged' cargo miri run
    Finished dev [unoptimized + debuginfo] target(s) in 0.00s
     Running `/home/devinj/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/cargo-miri target/miri/x86_64-unknown-linux-gnu/debug/test_miri`
thread 'main' panicked at '-Zmiri-track-pointer-tag requires a valid `u64` argument: invalid digit found in string', src/tools/miri/src/bin/miri.rs:393:33
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: rustc 1.56.0-nightly (a49e38e67 2021-08-23) running on x86_64-unknown-linux-gnu

note: compiler flags: -Z miri-track-pointer-tag=untagged -C embed-bitcode=no -C debuginfo=2 -C incremental --crate-type bin

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
end of query stack

Backtrace

thread 'main' panicked at '-Zmiri-track-pointer-tag requires a valid `u64` argument: invalid digit found in string', src/tools/miri/src/bin/miri.rs:393:33
stack backtrace:
   0: rust_begin_unwind
             at /rustc/a49e38e672c60da788360e088f00ad12353e3913/library/std/src/panicking.rs:517:5
   1: std::panicking::begin_panic_fmt
             at /rustc/a49e38e672c60da788360e088f00ad12353e3913/library/std/src/panicking.rs:460:5
   2: miri::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

[RalfJung]

Thanks for the report! I moved it to the Miri repo.

Indeed -Zmiri-track-pointer-tag=untagged does not work. And I also don't see how it can work: there are a lot of untagged pointers, and what this option would do is print a diagnostic whenever any one of them is being created or invalidated. That would be entirely useless.

Note that the verb "track" in -Zmiri-track-raw-pointers and -Zmiri-track-pointer-tag has a very different meaning... which is unfortunate but we didn't quite plan this well enough ahead. :/ Quoting from the docs:

-Zmiri-track-pointer-tag= shows a backtrace when the given pointer tag is popped from a borrow stack (which is where the tag becomes invalid and any future use of it will error). This helps you in finding out why UB is happening and where in your code would be a good place to look for it.

-Zmiri-track-raw-pointers makes Stacked Borrows track a pointer tag even for raw pointers. This can make valid code fail to pass the checks, but also can help identify latent aliasing issues in code that Miri accepts by default. You can recognize false positives by occurring in the message -- this indicates a pointer that was cast from an integer, so Miri was unable to track this pointer. Note that it is not currently guaranteed that code that works with -Zmiri-track-raw-pointers also works without -Zmiri-track-raw-pointers, but for the vast majority of code, this will be the case.

[RalfJung]

Maybe we should rename -Zmiri-track-raw-pointers to -Zmiri-tag-raw-pointers and deprecate the old name? We can't really remove it, but at least we can update the docs and hopefully over time the new name will become more used...

Cc @oli-obk

[camelid]

Maybe we should rename -Zmiri-track-raw-pointers to -Zmiri-tag-raw-pointers and deprecate the old name? We can't really remove it, but at least we can update the docs and hopefully over time the new name will become more used...

IIUC, Miri is only available on nightly, and these flags begin with -Z, so technically, I think it'd be okay to remove the old name (though Miri's stability guarantees are a bit unclear to me). But at least going through a deprecation period would probably be a good idea.

[RalfJung]

Technically we can do whatever we want, true -- but I see no reason to cause pain to all users that use the old name.

Miri has no stability guarantees, but we try to not annoy our users more than we think is necessary. ;)

[RalfJung]

The proposed rename is happening in #1911.

@ssbr could you say a bit more about what you are trying to debug? As I said above, many pointers are untagged, so there's not really any point in having Miri print something whenever an untagged pointer is used. Can you say a bit more about what you are trying to debug?

[ssbr]

Sorry for not chiming in earlier, lost track of this.

Can you say a bit more about what you are trying to debug?

The only way I'm aware of to make miri output how it is mutating the borrow stack is to tell it to track a tag. But it can only do so for non-raw pointers, so I was (and am) a bit stuck: AFAIK there's no way to determine how the borrow stack is mutated for raw pointers except to read the code closely and follow along with the paper, pretending to be miri. (But, in fact, I wanted miri to tell me what it was doing so that I could be sure I was correctly understanding the paper to begin with!)

You can see examples of the kinds of snippets I was working with e.g. here: #1878 . I don't know exactly what snippet prompted this particular attempt, but it would've been something along those lines -- short 5-10 line snippets where all pointers are interesting and do something important, and where I want to fully understand the borrow stack for every touched region of memory.

[ssbr]

A more recent example where something like this might be useful -- I was recently trying to make sure I understood why Vec<T> was allowed to touch e.g. its second T, despite holding a *mut T and not a *mut [T]. Seeing the borrow stack would've shown that it's using a raw pointer borrow that spans the whole array and predates the held *mut T. Similarly, I'm hoping to implement virtual method dispatch into C++ objects -- even if you only hold a pointer to a base class (a subobject, only part of the span). if I get it wrong, I will probably want to see when a raw pointer borrow was pushed/popped from the object.

As you say, in a larger program, there's too many pointers. But for a minimal reproducible example, where every pointer is relevant, I think that isn't a problem.

[RalfJung]

As you say, in a larger program, there's too many pointers. But for a minimal reproducible example, where every pointer is relevant, I think that isn't a problem.

Note that even if you write an empty main function, there are a lot of pointers -- Miri interprets programs the way they are actually executed, which means starting at lang_start.

I do agree that it would be great to have better visualizations and debugging features for Stacked Borrows. But asking specifically for -Zmiri-track-pointer-tag=untagged is an instance of the X/Y problem. We probably want something more like "show me in more detail what's happening in a particular region of memory", or so. So I will close this issue, since most of the discussion here is besides the point you were after.

There is #974 for debugability of Stacked Borrows failures; if that issue does not reflect what you are looking for then please open a new one and describe the problem you want to solve. I think this would also be a great use-case for https://github.com/oli-obk/priroda, if it could display Stacked Borrows data.

[RalfJung]

Ah I should also say, there are some tracing features you could play around with, but they will drown you in information:

MIRI_LOG=info,miri::stacked_borrows=trace ./miri run tests/run-pass/hello.rs

This is how I debugged Stacked Borrows while developing it.