If you want to hack on Miri yourself, great! Here are some resources you might find useful.
Check out the issues on this GitHub repository for some ideas. In particular,
look for the green E-* labels which mark issues that should be rather
well-suited for onboarding. For more ideas or help with hacking on Miri, you can
contact us on the Rust Zulip. See the Rust website
for a list of Miri maintainers.
When you get a review, please take care of the requested changes in new commits. Do not amend
existing commits. Generally avoid force-pushing. The only time you should force push is when there
is a conflict with the master branch (in that case you should rebase across master, not merge), and
all the way at the end of the review process when the reviewer tells you that the PR is done and you
should squash the commits. For the latter case, use git rebase --keep-base ... to squash without
changing the base commit your PR branches off of. Use your own judgment and the reviewer's guidance
to decide whether the PR should be squashed into a single commit or multiple logically separate
commits. (All this is to work around the fact that Github is quite bad at dealing with force pushes
and does not support git range-diff. Maybe one day Github will be good at git and then life can
become easier.)
Most PRs bounce back and forth between the reviewer and the author several times, so it is good to
keep track of who is expected to take the next step. We are using the S-waiting-for-review and
S-waiting-for-author labels for that. If a reviewer asked you to do some changes and you think
they are all taken care of, post a comment saying @rustbot ready to mark a PR as ready for the
next round of review.
If you are thinking about making a larger-scale contribution -- in particular anything that needs more than can reasonably fit in a single PR to be feature-complete -- then please talk to us before writing significant amounts of code. Generally, we will ask that you follow a three-step "project" process for such contributions:
-
Clearly define the goal of the project. This defines the scope of the project, i.e. which part of which APIs should be supported. If this involves functions that expose a big API surface with lots of flags, the project may want to support only a tiny subset of flags; that should be documented. A good way to express the goal is with one or more test cases that Miri should be able to successfully execute when the project is completed. It is a good idea to get feedback from team members already at this stage to ensure that the project is reasonably scoped and aligns with our interests.
-
Make a design for how to realize the goal. A larger project will likely have to do global changes to Miri, like adding new global state to the
Machinetype or new methods to theFileDescriptiontrait. Often we have to iterate on those changes, which can quite substantially change how the final implementation looks like.The design should be reasonably concrete, i.e. for new global state or methods the corresponding Rust types and method signatures should be spelled out. We realize that it can be hard to make a design without doing implementation work, in particular if you are not yet familiar with the codebase. Doing draft implementations in phase 2 of this process is perfectly fine, just please be aware that we might request fundamental changes that can require significantly reworking what you already did. If you open a PR in this stage, please clearly indicate that this project is still in the design stage.
-
Finish the implementation and have it reviewed.
This process is largely informal, and its primary goal is to more clearly communicate expectations. Please get in touch with us if you have any questions!
Miri heavily relies on internal and unstable rustc interfaces to execute MIR, which means it is important that you install a version of rustc that Miri actually works with.
The rust-version file contains the commit hash of rustc that Miri is currently
tested against. Other versions will likely not work. After installing
rustup-toolchain-install-master, you can run the following command to
install that exact version of rustc as a toolchain:
./miri toolchain
This will set up a rustup toolchain called miri and set it as an override for
the current directory.
You can also create a .auto-everything file (contents don't matter, can be empty), which
will cause any ./miri command to automatically call ./miri toolchain, clippy and rustfmt
for you. If you don't want all of these to happen, you can add individual .auto-toolchain,
.auto-clippy and .auto-fmt files respectively.
Invoking Miri requires getting a bunch of flags right and setting up a custom
sysroot. The miri script takes care of that for you. With the
build environment prepared, compiling Miri is just one command away:
./miri build
Run ./miri without arguments to see the other commands our build tool
supports.
The Miri driver compiled from src/bin/miri.rs is the "heart" of Miri: it is
basically a version of rustc that, instead of compiling your code, runs it.
It accepts all the same flags as rustc (though the ones only affecting code
generation and linking obviously will have no effect) and more.
For example, you can (cross-)run the driver on a particular file by doing
./miri run tests/pass/format.rs
./miri run tests/pass/hello.rs --target i686-unknown-linux-gnuTests in pass-dep need to be run using ./miri run --dep <filename>.
For example:
./miri run --dep tests/pass-dep/shims/libc-fs.rsYou can (cross-)run the entire test suite using:
./miri test
./miri test --target i686-unknown-linux-gnu./miri test FILTER only runs those tests that contain FILTER in their filename (including the
base directory, e.g. ./miri test fail will run all compile-fail tests). Multiple filters are
supported: ./miri test FILTER1 FILTER2 runs all tests that contain either string.
You can get a trace of which MIR statements are being executed by setting the
MIRI_LOG environment variable. For example:
MIRI_LOG=info ./miri run tests/pass/vec.rsSetting MIRI_LOG like this will configure logging for Miri itself as well as
the rustc_middle::mir::interpret and rustc_mir::interpret modules in rustc. You
can also do more targeted configuration, e.g. the following helps debug the
stacked borrows implementation:
MIRI_LOG=rustc_mir::interpret=info,miri::stacked_borrows ./miri run tests/pass/vec.rsNote that you will only get info, warn or error messages if you use a prebuilt compiler.
In order to get debug and trace level messages, you need to build miri with a locally built
compiler that has debug=true set in config.toml.
You can set MIRI_BACKTRACE=1 to get a backtrace of where an
evaluation error was originally raised.
We use ui-testing in Miri, meaning we generate .stderr and .stdout files for the output
produced by Miri. You can use ./miri test --bless to automatically (re)generate these files when
you add new tests or change how Miri presents certain output.
Note that when you also use MIRIFLAGS to change optimizations and similar, the ui output
will change in unexpected ways. In order to still be able
to run the other checks while ignoring the ui output, use MIRI_SKIP_UI_CHECKS=1 ./miri test.
For more info on how to configure ui tests see the documentation on the ui test crate
Working with the driver directly gives you full control, but you also lose all the convenience provided by cargo. Once your test case depends on a crate, it is probably easier to test it with the cargo wrapper. You can install your development version of Miri using
./miri install
and then you can use it as if it was installed by rustup as a component of the
miri toolchain. Note that the miri and cargo-miri executables are placed
in the miri toolchain's sysroot to prevent conflicts with other toolchains.
The Miri binaries in the cargo bin directory (usually ~/.cargo/bin) are managed by rustup.
There's a test for the cargo wrapper in the test-cargo-miri directory; run ./run-test.py in
there to execute it. You can pass --target to execute the test for another target.
Miri re-builds the standard library into a custom sysroot, so it is fairly easy
to test Miri against a modified standard library -- you do not even have to
build Miri yourself, the Miri shipped by rustup will work. All you have to do
is set the MIRI_LIB_SRC environment variable to the library folder of a
rust-lang/rust repository checkout. Note that changing files in that directory
does not automatically trigger a re-build of the standard library; you have to
clear the Miri build cache manually (on Linux, rm -rf ~/.cache/miri;
on Windows, rmdir /S "%LOCALAPPDATA%\rust-lang\miri\cache";
and on macOS, rm -rf ~/Library/Caches/org.rust-lang.miri).
Miri comes with a few benchmarks; you can run ./miri bench to run them with the locally built
Miri. Note: this will run ./miri install as a side-effect. Also requires hyperfine to be
installed (cargo install hyperfine).
To configure rust-analyzer and the IDE for working on Miri, copy one of the provided
configuration files according to the instructions below. You can also set up a symbolic
link to keep the configuration in sync with our recommendations.
Copy etc/rust_analyzer_vscode.json to .vscode/settings.json in the project root directory.
Copy etc/rust_analyzer_helix.toml to .helix/languages.toml in the project root directory.
Since working on Miri requires a custom toolchain, and Helix requires the language server
to be installed with the toolchain, you have to run ./miri toolchain -c rust-analyzer
when installing the Miri toolchain. Alternatively, set the RUSTUP_TOOLCHAIN environment variable according to
the documentation.
If you are building Miri with a locally built rustc, set
rust-analyzer.rustcSource to the relative path from your Miri clone to the
root Cargo.toml of the locally built rustc. For example, the path might look
like ../rust/Cargo.toml. In addition to that, replace clippy by check
in the rust-analyzer.check.overrideCommand setting.
See the rustc-dev-guide's docs on "Configuring rust-analyzer for rustc"
for more information about configuring the IDE and rust-analyzer.
We described above the simplest way to get a working build environment for Miri,
which is to use the version of rustc indicated by rustc-version. But
sometimes, that is not enough.
A big part of the Miri driver is shared with rustc, so working on Miri will
sometimes require also working on rustc itself. In this case, you should not
work in a clone of the Miri repository, but in a clone of the
main Rust repository. There is a copy of
Miri located at src/tools/miri that you can work on directly. A maintainer
will eventually sync those changes back into this repository.
When working on Miri in the rustc tree, here's how you can run tests:
./x.py test miri
--bless will work, too.
You can also directly run Miri on a Rust source file:
./x.py run miri --stage 1 --args src/tools/miri/tests/pass/hello.rs
We use the josh proxy to transmit changes between the
rustc and Miri repositories. You can install it as follows:
RUSTFLAGS="--cap-lints=warn" cargo +stable install josh-proxy --git https://github.com/josh-project/josh --tag r23.12.04Josh will automatically be started and stopped by ./miri.
Note: this usually happens automatically, so these steps rarely have to be done by hand.
We assume we start on an up-to-date master branch in the Miri repo.
# Fetch and merge rustc side of the history. Takes ca 5 min the first time.
# This will also update the `rustc-version` file.
./miri rustc-pull
# Update local toolchain and apply formatting.
./miri toolchain && ./miri fmt
git commit -am "rustup"Now push this to a new branch in your Miri fork, and create a PR. It is worth
running ./miri test locally in parallel, since the test suite in the Miri repo
is stricter than the one on the rustc side, so some small tweaks might be
needed.
We will use the josh proxy to push to your fork of rustc. Run the following in the Miri repo, assuming we are on an up-to-date master branch:
# Push the Miri changes to your rustc fork (substitute your github handle for YOUR_NAME).
./miri rustc-push YOUR_NAME miriThis will create a new branch called miri in your fork, and the output should include a link that
creates a rustc PR to integrate those changes into the main repository. If that PR has conflicts,
you need to pull rustc changes into Miri first, and then re-do the rustc push.
If this fails due to authentication problems, it can help to make josh push via ssh instead of
https. Add the following to your .gitconfig:
[url "git@github.com:"]
pushInsteadOf = https://github.com/The following environment variables are relevant to ./miri:
MIRI_AUTO_OPSindicates whether the automatic execution of rustfmt, clippy and toolchain setup (as controlled by the./auto-*files) should be skipped. If it is set tono, they are skipped. This is used to allow automated IDE actions to avoid the auto ops.MIRI_LOG,MIRI_BACKTRACEcontrol logging and backtrace printing during Miri executions.MIRI_TEST_THREADS(recognized by./miri test) sets the number of threads to use for running tests. By default, the number of cores is used.MIRI_SKIP_UI_CHECKS(recognized by./miri test) disables checking that thestderrorstdoutfiles match the actual output.
Furthermore, the usual environment variables recognized by cargo miri also work for ./miri, e.g.
MIRI_LIB_SRC. Note that MIRIFLAGS is ignored by ./miri test as each test controls the flags it
is run with.
The following environment variables are internal and must not be used by anyone but Miri itself. They are used to communicate between different Miri binaries, and as such worth documenting:
CARGO_EXTRA_FLAGSis understood by./miriand passed to all host cargo invocations. It is reserved for CI usage; setting the wrong flags this way can easily confuse the script.MIRI_BE_RUSTCcan be set tohostortarget. It tells the Miri driver to actually not interpret the code but compile it like rustc would. Withtarget, Miri sets some compiler flags to prepare the code for interpretation; withhost, this is not done. This environment variable is useful to be sure that the compiledrlibs are compatible with Miri.MIRI_CALLED_FROM_SETUPis set during the Miri sysroot build, which will re-invokecargo-mirias therustcto use for this build.MIRI_CALLED_FROM_RUSTDOCwhen set to any value tellscargo-mirithat it is running as a child process ofrustdoc, which invokes it twice for each doc-test and requires special treatment, most notably a check-only build before interpretation. This is set bycargo-miriitself when running as arustdoc-wrapper.MIRI_CWDwhen set to any value tells the Miri driver to change to the given directory after loading all the source files, but before commencing interpretation. This is useful if the interpreted program wants a different working directory at run-time than at build-time.MIRI_LOCAL_CRATESis set bycargo-mirito tell the Miri driver which crates should be given special treatment in diagnostics, in addition to the crate currently being compiled.MIRI_ORIG_RUSTDOCis set and read by different phases ofcargo-mirito remember the value ofRUSTDOCfrom before it was overwritten.MIRI_REPLACE_LIBRS_IF_NOT_TESTwhen set to any value enables a hack that helps bootstrap run the standard library tests in Miri.MIRI_TEST_TARGETis set by./miri test(and./x.py test miri) to tell the test harness about the chosen target.MIRI_VERBOSEwhen set to any value tells the variouscargo-miriphases to perform verbose logging.MIRI_HOST_SYSROOTis set by bootstrap to tellcargo-miriwhich sysroot to use for host operations.RUSTC_BLESSis set by./miri test(and./x.py test miri) to indicate bless-mode to the test harness.