Auto merge of rust-lang#112418 - ferrocene:pa-mir-opt-panic, r=ozkano…

…nur,saethlin

Add support for targets without unwinding in `mir-opt`, and improve `--bless` for it

The main goal of this PR is to add support for targets without unwinding support in the `mir-opt` test suite, by adding the `EMIT_MIR_FOR_EACH_PANIC_STRATEGY` comment. Similarly to 32bit vs 64bit, when that comment is present, blessed output files will have the `.panic-unwind` or `.panic-abort` suffix, and the right one will be chosen depending on the target's panic strategy.

The `EMIT_MIR_FOR_EACH_PANIC_STRATEGY` comment replaced all the `ignore-wasm32` comments in the `mir-opt` test suite, as those comments were added due to `wasm32` being a target without unwinding support. The comment was also added on other tests that were only executed on x86 but were still panic strategy dependent.

The `mir-opt` suite was then blessed, which caused a ton of churn as most of the existing output files had to be renamed and (mostly) duplicated with the abort strategy.

---

After [asking on Zulip](https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/mir-opt.20tests.20and.20panic.3Dabort), the main concern about this change is it'd make blessing the `mir-opt` suite even harder, as you'd need to both bless it with an unwinding target and an aborting target. This exacerbated the current situation, where you'd need to bless it with a 32bit and a 64bit target already.

Because of that, this PR also makes significant enhancements to `--bless` for the `mir-opt` suite, where it will automatically bless the suite four times with different targets, while requiring minimal cross-compilation.

To handle the 32bit vs 64bit blessing, there is now an hardcoded list of target mapping between 32bit and 64bit. The goal of the list is to find a related target that will *probably* work without requiring additional cross-compilation toolchains on the system. If a mapping is found, bootstrap will bless the suite with both targets, otherwise just with the current target.

To handle the panic strategy blessing (abort vs unwind), I had to resort to what I call "synthetic targets". For each of the target we're blessing (so either the current one, or a 32bit and a 64bit depending on the previous paragraph), bootstrap will extract the JSON spec of the target and change it to include `"panic-strategy": "abort"`. It will then build the standard library with this synthetic target, and bless the `mir-opt` suite with it.

As a result of these changes, blessing the `mir-opt` suite will actually bless it two or four times with different targets, ensuring all possible variants are actually blessed.

---

This PR is best reviewed commit-by-commit.

r? `@jyn514`
cc `@saethlin` `@oli-obk`

library/std/build.rs

@@ -36,6 +36,8 @@ fn main() {
         || target.contains("nintendo-3ds")
         || target.contains("vita")
         || target.contains("nto")
+        // See src/bootstrap/synthetic_targets.rs
+        || env::var("RUSTC_BOOTSTRAP_SYNTHETIC_TARGET").is_ok()
     {
         // These platforms don't have any special requirements.
     } else {

src/bootstrap/builder.rs

@@ -1650,7 +1650,8 @@ impl<'a> Builder<'a> {
             }
         };
         cargo.env(profile_var("DEBUG"), debuginfo_level.to_string());
-        if self.cc[&target].args().iter().any(|arg| arg == "-gz") {
+        if !self.config.dry_run() && self.cc.borrow()[&target].args().iter().any(|arg| arg == "-gz")
+        {
             rustflags.arg("-Clink-arg=-gz");
         }
         cargo.env(

src/bootstrap/cc_detect.rs

@@ -89,7 +89,7 @@ fn new_cc_build(build: &Build, target: TargetSelection) -> cc::Build {
     cfg
 }
 
-pub fn find(build: &mut Build) {
+pub fn find(build: &Build) {
     // For all targets we're going to need a C compiler for building some shims
     // and such as well as for being a linker for Rust code.
     let targets = build
@@ -100,60 +100,64 @@ pub fn find(build: &mut Build) {
         .chain(iter::once(build.build))
         .collect::<HashSet<_>>();
     for target in targets.into_iter() {
-        let mut cfg = new_cc_build(build, target);
-        let config = build.config.target_config.get(&target);
-        if let Some(cc) = config.and_then(|c| c.cc.as_ref()) {
-            cfg.compiler(cc);
-        } else {
-            set_compiler(&mut cfg, Language::C, target, config, build);
-        }
+        find_target(build, target);
+    }
+}
 
-        let compiler = cfg.get_compiler();
-        let ar = if let ar @ Some(..) = config.and_then(|c| c.ar.clone()) {
-            ar
-        } else {
-            cc2ar(compiler.path(), target)
-        };
+pub fn find_target(build: &Build, target: TargetSelection) {
+    let mut cfg = new_cc_build(build, target);
+    let config = build.config.target_config.get(&target);
+    if let Some(cc) = config.and_then(|c| c.cc.as_ref()) {
+        cfg.compiler(cc);
+    } else {
+        set_compiler(&mut cfg, Language::C, target, config, build);
+    }
 
-        build.cc.insert(target, compiler.clone());
-        let cflags = build.cflags(target, GitRepo::Rustc, CLang::C);
+    let compiler = cfg.get_compiler();
+    let ar = if let ar @ Some(..) = config.and_then(|c| c.ar.clone()) {
+        ar
+    } else {
+        cc2ar(compiler.path(), target)
+    };
 
-        // If we use llvm-libunwind, we will need a C++ compiler as well for all targets
-        // We'll need one anyways if the target triple is also a host triple
-        let mut cfg = new_cc_build(build, target);
-        cfg.cpp(true);
-        let cxx_configured = if let Some(cxx) = config.and_then(|c| c.cxx.as_ref()) {
-            cfg.compiler(cxx);
-            true
-        } else if build.hosts.contains(&target) || build.build == target {
-            set_compiler(&mut cfg, Language::CPlusPlus, target, config, build);
-            true
-        } else {
-            // Use an auto-detected compiler (or one configured via `CXX_target_triple` env vars).
-            cfg.try_get_compiler().is_ok()
-        };
+    build.cc.borrow_mut().insert(target, compiler.clone());
+    let cflags = build.cflags(target, GitRepo::Rustc, CLang::C);
 
-        // for VxWorks, record CXX compiler which will be used in lib.rs:linker()
-        if cxx_configured || target.contains("vxworks") {
-            let compiler = cfg.get_compiler();
-            build.cxx.insert(target, compiler);
-        }
+    // If we use llvm-libunwind, we will need a C++ compiler as well for all targets
+    // We'll need one anyways if the target triple is also a host triple
+    let mut cfg = new_cc_build(build, target);
+    cfg.cpp(true);
+    let cxx_configured = if let Some(cxx) = config.and_then(|c| c.cxx.as_ref()) {
+        cfg.compiler(cxx);
+        true
+    } else if build.hosts.contains(&target) || build.build == target {
+        set_compiler(&mut cfg, Language::CPlusPlus, target, config, build);
+        true
+    } else {
+        // Use an auto-detected compiler (or one configured via `CXX_target_triple` env vars).
+        cfg.try_get_compiler().is_ok()
+    };
 
-        build.verbose(&format!("CC_{} = {:?}", &target.triple, build.cc(target)));
-        build.verbose(&format!("CFLAGS_{} = {:?}", &target.triple, cflags));
-        if let Ok(cxx) = build.cxx(target) {
-            let cxxflags = build.cflags(target, GitRepo::Rustc, CLang::Cxx);
-            build.verbose(&format!("CXX_{} = {:?}", &target.triple, cxx));
-            build.verbose(&format!("CXXFLAGS_{} = {:?}", &target.triple, cxxflags));
-        }
-        if let Some(ar) = ar {
-            build.verbose(&format!("AR_{} = {:?}", &target.triple, ar));
-            build.ar.insert(target, ar);
-        }
+    // for VxWorks, record CXX compiler which will be used in lib.rs:linker()
+    if cxx_configured || target.contains("vxworks") {
+        let compiler = cfg.get_compiler();
+        build.cxx.borrow_mut().insert(target, compiler);
+    }
 
-        if let Some(ranlib) = config.and_then(|c| c.ranlib.clone()) {
-            build.ranlib.insert(target, ranlib);
-        }
+    build.verbose(&format!("CC_{} = {:?}", &target.triple, build.cc(target)));
+    build.verbose(&format!("CFLAGS_{} = {:?}", &target.triple, cflags));
+    if let Ok(cxx) = build.cxx(target) {
+        let cxxflags = build.cflags(target, GitRepo::Rustc, CLang::Cxx);
+        build.verbose(&format!("CXX_{} = {:?}", &target.triple, cxx));
+        build.verbose(&format!("CXXFLAGS_{} = {:?}", &target.triple, cxxflags));
+    }
+    if let Some(ar) = ar {
+        build.verbose(&format!("AR_{} = {:?}", &target.triple, ar));
+        build.ar.borrow_mut().insert(target, ar);
+    }
+
+    if let Some(ranlib) = config.and_then(|c| c.ranlib.clone()) {
+        build.ranlib.borrow_mut().insert(target, ranlib);
     }
 }
 

src/bootstrap/compile.rs

@@ -169,6 +169,11 @@ impl Step for Std {
             cargo.arg("-p").arg(krate);
         }
 
+        // See src/bootstrap/synthetic_targets.rs
+        if target.is_synthetic() {
+            cargo.env("RUSTC_BOOTSTRAP_SYNTHETIC_TARGET", "1");
+        }
+
         let _guard = builder.msg(
             Kind::Build,
             compiler.stage,
@@ -314,7 +319,7 @@ fn copy_self_contained_objects(
         }
     } else if target.ends_with("windows-gnu") {
         for obj in ["crt2.o", "dllcrt2.o"].iter() {
-            let src = compiler_file(builder, builder.cc(target), target, CLang::C, obj);
+            let src = compiler_file(builder, &builder.cc(target), target, CLang::C, obj);
             let target = libdir_self_contained.join(obj);
             builder.copy(&src, &target);
             target_deps.push((target, DependencyType::TargetSelfContained));
@@ -995,8 +1000,13 @@ fn rustc_llvm_env(builder: &Builder<'_>, cargo: &mut Cargo, target: TargetSelect
         && !target.contains("apple")
         && !target.contains("solaris")
     {
-        let file =
-            compiler_file(builder, builder.cxx(target).unwrap(), target, CLang::Cxx, "libstdc++.a");
+        let file = compiler_file(
+            builder,
+            &builder.cxx(target).unwrap(),
+            target,
+            CLang::Cxx,
+            "libstdc++.a",
+        );
         cargo.env("LLVM_STATIC_STDCPP", file);
     }
     if builder.llvm_link_shared() {
@@ -1267,6 +1277,9 @@ pub fn compiler_file(
     c: CLang,
     file: &str,
 ) -> PathBuf {
+    if builder.config.dry_run() {
+        return PathBuf::new();
+    }
     let mut cmd = Command::new(compiler);
     cmd.args(builder.cflags(target, GitRepo::Rustc, c));
     cmd.arg(format!("-print-file-name={}", file));

src/bootstrap/config.rs

@@ -429,6 +429,7 @@ impl std::str::FromStr for RustcLto {
 pub struct TargetSelection {
     pub triple: Interned<String>,
     file: Option<Interned<String>>,
+    synthetic: bool,
 }
 
 /// Newtype over `Vec<TargetSelection>` so we can implement custom parsing logic
@@ -460,7 +461,15 @@ impl TargetSelection {
         let triple = INTERNER.intern_str(triple);
         let file = file.map(|f| INTERNER.intern_str(f));
 
-        Self { triple, file }
+        Self { triple, file, synthetic: false }
+    }
+
+    pub fn create_synthetic(triple: &str, file: &str) -> Self {
+        Self {
+            triple: INTERNER.intern_str(triple),
+            file: Some(INTERNER.intern_str(file)),
+            synthetic: true,
+        }
     }
 
     pub fn rustc_target_arg(&self) -> &str {
@@ -478,6 +487,11 @@ impl TargetSelection {
     pub fn ends_with(&self, needle: &str) -> bool {
         self.triple.ends_with(needle)
     }
+
+    // See src/bootstrap/synthetic_targets.rs
+    pub fn is_synthetic(&self) -> bool {
+        self.synthetic
+    }
 }
 
 impl fmt::Display for TargetSelection {

src/bootstrap/dist.rs

@@ -170,6 +170,10 @@ fn make_win_dist(
     target: TargetSelection,
     builder: &Builder<'_>,
 ) {
+    if builder.config.dry_run() {
+        return;
+    }
+
     //Ask gcc where it keeps its stuff
     let mut cmd = Command::new(builder.cc(target));
     cmd.arg("-print-search-dirs");

src/bootstrap/lib.rs

@@ -61,6 +61,7 @@ mod run;
 mod sanity;
 mod setup;
 mod suggest;
+mod synthetic_targets;
 mod tarball;
 mod test;
 mod tool;
@@ -226,10 +227,10 @@ pub struct Build {
 
     // Runtime state filled in later on
     // C/C++ compilers and archiver for all targets
-    cc: HashMap<TargetSelection, cc::Tool>,
-    cxx: HashMap<TargetSelection, cc::Tool>,
-    ar: HashMap<TargetSelection, PathBuf>,
-    ranlib: HashMap<TargetSelection, PathBuf>,
+    cc: RefCell<HashMap<TargetSelection, cc::Tool>>,
+    cxx: RefCell<HashMap<TargetSelection, cc::Tool>>,
+    ar: RefCell<HashMap<TargetSelection, PathBuf>>,
+    ranlib: RefCell<HashMap<TargetSelection, PathBuf>>,
     // Miscellaneous
     // allow bidirectional lookups: both name -> path and path -> name
     crates: HashMap<Interned<String>, Crate>,
@@ -451,10 +452,10 @@ impl Build {
             miri_info,
             rustfmt_info,
             in_tree_llvm_info,
-            cc: HashMap::new(),
-            cxx: HashMap::new(),
-            ar: HashMap::new(),
-            ranlib: HashMap::new(),
+            cc: RefCell::new(HashMap::new()),
+            cxx: RefCell::new(HashMap::new()),
+            ar: RefCell::new(HashMap::new()),
+            ranlib: RefCell::new(HashMap::new()),
             crates: HashMap::new(),
             crate_paths: HashMap::new(),
             is_sudo,
@@ -482,7 +483,7 @@ impl Build {
         }
 
         build.verbose("finding compilers");
-        cc_detect::find(&mut build);
+        cc_detect::find(&build);
         // When running `setup`, the profile is about to change, so any requirements we have now may
         // be different on the next invocation. Don't check for them until the next time x.py is
         // run. This is ok because `setup` never runs any build commands, so it won't fail if commands are missing.
@@ -1103,16 +1104,22 @@ impl Build {
     }
 
     /// Returns the path to the C compiler for the target specified.
-    fn cc(&self, target: TargetSelection) -> &Path {
-        self.cc[&target].path()
+    fn cc(&self, target: TargetSelection) -> PathBuf {
+        if self.config.dry_run() {
+            return PathBuf::new();
+        }
+        self.cc.borrow()[&target].path().into()
     }
 
     /// Returns a list of flags to pass to the C compiler for the target
     /// specified.
     fn cflags(&self, target: TargetSelection, which: GitRepo, c: CLang) -> Vec<String> {
+        if self.config.dry_run() {
+            return Vec::new();
+        }
         let base = match c {
-            CLang::C => &self.cc[&target],
-            CLang::Cxx => &self.cxx[&target],
+            CLang::C => self.cc.borrow()[&target].clone(),
+            CLang::Cxx => self.cxx.borrow()[&target].clone(),
         };
 
         // Filter out -O and /O (the optimization flags) that we picked up from
@@ -1153,41 +1160,53 @@ impl Build {
     }
 
     /// Returns the path to the `ar` archive utility for the target specified.
-    fn ar(&self, target: TargetSelection) -> Option<&Path> {
-        self.ar.get(&target).map(|p| &**p)
+    fn ar(&self, target: TargetSelection) -> Option<PathBuf> {
+        if self.config.dry_run() {
+            return None;
+        }
+        self.ar.borrow().get(&target).cloned()
     }
 
     /// Returns the path to the `ranlib` utility for the target specified.
-    fn ranlib(&self, target: TargetSelection) -> Option<&Path> {
-        self.ranlib.get(&target).map(|p| &**p)
+    fn ranlib(&self, target: TargetSelection) -> Option<PathBuf> {
+        if self.config.dry_run() {
+            return None;
+        }
+        self.ranlib.borrow().get(&target).cloned()
     }
 
     /// Returns the path to the C++ compiler for the target specified.
-    fn cxx(&self, target: TargetSelection) -> Result<&Path, String> {
-        match self.cxx.get(&target) {
-            Some(p) => Ok(p.path()),
+    fn cxx(&self, target: TargetSelection) -> Result<PathBuf, String> {
+        if self.config.dry_run() {
+            return Ok(PathBuf::new());
+        }
+        match self.cxx.borrow().get(&target) {
+            Some(p) => Ok(p.path().into()),
             None => {
                 Err(format!("target `{}` is not configured as a host, only as a target", target))
             }
         }
     }
 
     /// Returns the path to the linker for the given target if it needs to be overridden.
-    fn linker(&self, target: TargetSelection) -> Option<&Path> {
-        if let Some(linker) = self.config.target_config.get(&target).and_then(|c| c.linker.as_ref())
+    fn linker(&self, target: TargetSelection) -> Option<PathBuf> {
+        if self.config.dry_run() {
+            return Some(PathBuf::new());
+        }
+        if let Some(linker) = self.config.target_config.get(&target).and_then(|c| c.linker.clone())
         {
             Some(linker)
         } else if target.contains("vxworks") {
             // need to use CXX compiler as linker to resolve the exception functions
             // that are only existed in CXX libraries
-            Some(self.cxx[&target].path())
+            Some(self.cxx.borrow()[&target].path().into())
         } else if target != self.config.build
             && util::use_host_linker(target)
             && !target.contains("msvc")
         {
             Some(self.cc(target))
         } else if self.config.use_lld && !self.is_fuse_ld_lld(target) && self.build == target {
-            Some(&self.initial_lld)
+            Some(self.initial_lld.clone())
         } else {
             None
         }