compilertest (UI test): Support custom normalization.

src/test/COMPILER_TESTS.md

@@ -37,7 +37,7 @@ The error levels that you can have are:
 Header commands specify something about the entire test file as a
 whole, instead of just a few lines inside the test.
 
-* `ignore-X` where `X` is an architecture, OS or stage will ignore the test accordingly
+* `ignore-X` where `X` is a target detail or stage will ignore the test accordingly (see below)
 * `ignore-pretty` will not compile the pretty-printed test (this is done to test the pretty-printer, but might not always work)
 * `ignore-test` always ignores the test
 * `ignore-lldb` and `ignore-gdb` will skip the debuginfo tests
@@ -50,6 +50,14 @@ whole, instead of just a few lines inside the test.
   feature is attempted without the proper `#![feature(X)]` tag.
   Each unstable lang feature is required to have a gate test.
 
+Some examples of `X` in `ignore-X`:
+
+* Architecture: `aarch64`, `arm`, `asmjs`, `mips`, `wasm32`, `x86_64`, `x86`, ...
+* OS: `android`, `emscripten`, `freebsd`, `ios`, `linux`, `macos`, `windows`, ...
+* Environment (fourth word of the target triple): `gnu`, `msvc`, `musl`.
+* Pointer width: `32bit`, `64bit`.
+* Stage: `stage0`, `stage1`, `stage2`.
+
 ## Revisions
 
 Certain classes of tests support "revisions" (as of the time of this
@@ -86,3 +94,66 @@ For example, the `ignore-test` header (and all "ignore" headers)
 currently only apply to the test as a whole, not to particular
 revisions. The only headers that are intended to really work when
 customized to a revision are error patterns and compiler flags.
+
+## Guide to the UI Tests
+
+The UI tests are intended to capture the compiler's complete output,
+so that we can test all aspects of the presentation. They work by
+compiling a file (e.g., `ui/hello_world/main.rs`), capturing the output,
+and then applying some normalization (see below). This normalized
+result is then compared against reference files named
+`ui/hello_world/main.stderr` and `ui/hello_world/main.stdout`. If either of
+those files doesn't exist, the output must be empty. If the test run
+fails, we will print out the current output, but it is also saved in
+`build/<target-triple>/test/ui/hello_world/main.stdout` (this path is
+printed as part of the test failure mesage), so you can run `diff` and
+so forth.
+
+### Editing and updating the reference files
+
+If you have changed the compiler's output intentionally, or you are
+making a new test, you can use the script `ui/update-references.sh` to
+update the references. When you run the test framework, it will report
+various errors: in those errors is a command you can use to run the
+`ui/update-references.sh` script, which will then copy over the files
+from the build directory and use them as the new reference. You can
+also just run `ui/update-all-references.sh`. In both cases, you can run
+the script with `--help` to get a help message.
+
+### Normalization
+
+The normalization applied is aimed at eliminating output difference
+between platforms, mainly about filenames:
+
+- the test directory is replaced with `$DIR`
+- all backslashes (`\`) are converted to forward slashes (`/`) (for Windows)
+- all CR LF newlines are converted to LF
+
+Sometimes these built-in normalizations are not enough. In such cases, you
+may provide custom normalization rules using the header commands, e.g.
+
+```
+// normalize-stderr-32bit: "fn() (32 bits)" -> "fn() ($PTR bits)"
+// normalize-stderr-64bit: "fn() (64 bits)" -> "fn() ($PTR bits)"
+```
+
+This tells the test, on 32-bit platforms, whenever the compiler writes
+`fn() (32 bits)` to stderr, it should be normalized to read `fn() ($PTR bits)`
+instead. Similar for 64-bit.
+
+The corresponding reference file will use the normalized output to test both
+32-bit and 64-bit platforms:
+
+```
+...
+   |
+   = note: source type: fn() ($PTR bits)
+   = note: target type: u16 (16 bits)
+...
+```
+
+Please see `ui/transmute/main.rs` and `.stderr` for a concrete usage example.
+
+Besides `normalize-stderr-32bit` and `-64bit`, one may use any target
+information or stage supported by `ignore-X` here as well (e.g.
+`normalize-stderr-windows`).

src/test/run-pass/i128-ffi.rs

@@ -13,12 +13,7 @@
 // should look like.
 
 // ignore-windows
-
-// Ignore 32 bit targets:
-// ignore-x86
-// ignore-arm
-
-// ignore-emscripten
+// ignore-32bit
 
 #![feature(i128_type)]
 

src/test/ui/enum-size-variance.rs

@@ -9,10 +9,6 @@
 // except according to those terms.
 
 // run-pass
-// ignore-x86
-// ignore-arm
-// ignore-emscripten
-// ^ ignore 32-bit targets, as the error message is target-dependent. see PR #41968.
 
 #![warn(variant_size_differences)]
 #![allow(dead_code)]
@@ -24,26 +20,26 @@ enum Enum1 { }
 
 enum Enum2 { A, B, C }
 
-enum Enum3 { D(isize), E, F }
+enum Enum3 { D(i64), E, F }
 
-enum Enum4 { H(isize), I(isize), J }
+enum Enum4 { H(i64), I(i64), J }
 
 enum Enum5 {
-    L(isize, isize, isize, isize), //~ WARNING three times larger
-    M(isize),
+    L(i64, i64, i64, i64), //~ WARNING three times larger
+    M(i64),
     N
 }
 
 enum Enum6<T, U> {
     O(T),
     P(U),
-    Q(isize)
+    Q(i64)
 }
 
 #[allow(variant_size_differences)]
 enum Enum7 {
-    R(isize, isize, isize, isize),
-    S(isize),
+    R(i64, i64, i64, i64),
+    S(i64),
     T
 }
 pub fn main() { }

src/test/ui/enum-size-variance.stderr

@@ -1,12 +1,12 @@
 warning: enum variant is more than three times larger (32 bytes) than the next largest
-  --> $DIR/enum-size-variance.rs:32:5
+  --> $DIR/enum-size-variance.rs:28:5
    |
-32 |     L(isize, isize, isize, isize), //~ WARNING three times larger
-   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+28 |     L(i64, i64, i64, i64), //~ WARNING three times larger
+   |     ^^^^^^^^^^^^^^^^^^^^^
    |
 note: lint level defined here
-  --> $DIR/enum-size-variance.rs:17:9
+  --> $DIR/enum-size-variance.rs:13:9
    |
-17 | #![warn(variant_size_differences)]
+13 | #![warn(variant_size_differences)]
    |         ^^^^^^^^^^^^^^^^^^^^^^^^
 

src/test/ui/transmute/main.rs

@@ -8,10 +8,10 @@
 // option. This file may not be copied, modified, or distributed
 // except according to those terms.
 
-// ignore-x86
-// ignore-arm
-// ignore-emscripten
-// ignore 32-bit platforms (test output is different)
+// normalize-stderr-32bit: "&str (64 bits)" -> "&str ($STR bits)"
+// normalize-stderr-64bit: "&str (128 bits)" -> "&str ($STR bits)"
+
+
 
 #![feature(untagged_unions)]
 use std::mem::transmute;

src/test/ui/transmute/main.stderr

@@ -22,7 +22,7 @@ error[E0512]: transmute called with types of different sizes
 34 |     let x: u8 = transmute("test"); //~ ERROR transmute called with types of different sizes
    |                 ^^^^^^^^^
    |
-   = note: source type: &str (128 bits)
+   = note: source type: &str ($STR bits)
    = note: target type: u8 (8 bits)
 
 error[E0512]: transmute called with types of different sizes

src/test/ui/transmute/transmute-from-fn-item-types-error.rs

@@ -8,14 +8,9 @@
 // option. This file may not be copied, modified, or distributed
 // except according to those terms.
 
-// ignore-x86
-// ignore-arm
-// ignore-emscripten
-// ignore 32-bit platforms (test output is different)
-
 use std::mem;
 
-unsafe fn foo() -> (i32, *const (), Option<fn()>) {
+unsafe fn foo() -> (i8, *const (), Option<fn()>) {
     let i = mem::transmute(bar);
     //~^ ERROR is zero-sized and can't be transmuted
     //~^^ NOTE cast with `as` to a pointer instead
@@ -46,7 +41,7 @@ unsafe fn bar() {
     //~^^ NOTE cast with `as` to a pointer instead
 
     // No error if a coercion would otherwise occur.
-    mem::transmute::<fn(), u32>(main);
+    mem::transmute::<fn(), usize>(main);
 }
 
 unsafe fn baz() {
@@ -63,7 +58,7 @@ unsafe fn baz() {
     //~^^ NOTE cast with `as` to a pointer instead
 
     // No error if a coercion would otherwise occur.
-    mem::transmute::<Option<fn()>, u32>(Some(main));
+    mem::transmute::<Option<fn()>, usize>(Some(main));
 }
 
 fn main() {

src/test/ui/transmute/transmute-from-fn-item-types-error.stderr

@@ -1,108 +1,90 @@
 error[E0512]: transmute called with types of different sizes
-  --> $DIR/transmute-from-fn-item-types-error.rs:19:13
+  --> $DIR/transmute-from-fn-item-types-error.rs:14:13
    |
-19 |     let i = mem::transmute(bar);
+14 |     let i = mem::transmute(bar);
    |             ^^^^^^^^^^^^^^
    |
    = note: source type: unsafe fn() {bar} (0 bits)
-   = note: target type: i32 (32 bits)
+   = note: target type: i8 (8 bits)
 
 error[E0591]: can't transmute zero-sized type
-  --> $DIR/transmute-from-fn-item-types-error.rs:23:13
+  --> $DIR/transmute-from-fn-item-types-error.rs:18:13
    |
-23 |     let p = mem::transmute(foo);
+18 |     let p = mem::transmute(foo);
    |             ^^^^^^^^^^^^^^
    |
-   = note: source type: unsafe fn() -> (i32, *const (), std::option::Option<fn()>) {foo}
+   = note: source type: unsafe fn() -> (i8, *const (), std::option::Option<fn()>) {foo}
    = note: target type: *const ()
    = help: cast with `as` to a pointer instead
 
 error[E0591]: can't transmute zero-sized type
-  --> $DIR/transmute-from-fn-item-types-error.rs:27:14
+  --> $DIR/transmute-from-fn-item-types-error.rs:22:14
    |
-27 |     let of = mem::transmute(main);
+22 |     let of = mem::transmute(main);
    |              ^^^^^^^^^^^^^^
    |
    = note: source type: fn() {main}
    = note: target type: std::option::Option<fn()>
    = help: cast with `as` to a pointer instead
 
 error[E0512]: transmute called with types of different sizes
-  --> $DIR/transmute-from-fn-item-types-error.rs:36:5
+  --> $DIR/transmute-from-fn-item-types-error.rs:31:5
    |
-36 |     mem::transmute::<_, u8>(main);
+31 |     mem::transmute::<_, u8>(main);
    |     ^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: source type: fn() {main} (0 bits)
    = note: target type: u8 (8 bits)
 
 error[E0591]: can't transmute zero-sized type
-  --> $DIR/transmute-from-fn-item-types-error.rs:40:5
+  --> $DIR/transmute-from-fn-item-types-error.rs:35:5
    |
-40 |     mem::transmute::<_, *mut ()>(foo);
+35 |     mem::transmute::<_, *mut ()>(foo);
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
-   = note: source type: unsafe fn() -> (i32, *const (), std::option::Option<fn()>) {foo}
+   = note: source type: unsafe fn() -> (i8, *const (), std::option::Option<fn()>) {foo}
    = note: target type: *mut ()
    = help: cast with `as` to a pointer instead
 
 error[E0591]: can't transmute zero-sized type
-  --> $DIR/transmute-from-fn-item-types-error.rs:44:5
+  --> $DIR/transmute-from-fn-item-types-error.rs:39:5
    |
-44 |     mem::transmute::<_, fn()>(bar);
+39 |     mem::transmute::<_, fn()>(bar);
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: source type: unsafe fn() {bar}
    = note: target type: fn()
    = help: cast with `as` to a pointer instead
 
-error[E0512]: transmute called with types of different sizes
-  --> $DIR/transmute-from-fn-item-types-error.rs:49:5
-   |
-49 |     mem::transmute::<fn(), u32>(main);
-   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^
-   |
-   = note: source type: fn() (64 bits)
-   = note: target type: u32 (32 bits)
-
 error[E0591]: can't transmute zero-sized type
-  --> $DIR/transmute-from-fn-item-types-error.rs:53:5
+  --> $DIR/transmute-from-fn-item-types-error.rs:48:5
    |
-53 |     mem::transmute::<_, *mut ()>(Some(foo));
+48 |     mem::transmute::<_, *mut ()>(Some(foo));
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
-   = note: source type: unsafe fn() -> (i32, *const (), std::option::Option<fn()>) {foo}
+   = note: source type: unsafe fn() -> (i8, *const (), std::option::Option<fn()>) {foo}
    = note: target type: *mut ()
    = help: cast with `as` to a pointer instead
 
 error[E0591]: can't transmute zero-sized type
-  --> $DIR/transmute-from-fn-item-types-error.rs:57:5
+  --> $DIR/transmute-from-fn-item-types-error.rs:52:5
    |
-57 |     mem::transmute::<_, fn()>(Some(bar));
+52 |     mem::transmute::<_, fn()>(Some(bar));
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: source type: unsafe fn() {bar}
    = note: target type: fn()
    = help: cast with `as` to a pointer instead
 
 error[E0591]: can't transmute zero-sized type
-  --> $DIR/transmute-from-fn-item-types-error.rs:61:5
+  --> $DIR/transmute-from-fn-item-types-error.rs:56:5
    |
-61 |     mem::transmute::<_, Option<fn()>>(Some(baz));
+56 |     mem::transmute::<_, Option<fn()>>(Some(baz));
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: source type: unsafe fn() {baz}
    = note: target type: std::option::Option<fn()>
    = help: cast with `as` to a pointer instead
 
-error[E0512]: transmute called with types of different sizes
-  --> $DIR/transmute-from-fn-item-types-error.rs:66:5
-   |
-66 |     mem::transmute::<Option<fn()>, u32>(Some(main));
-   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-   |
-   = note: source type: std::option::Option<fn()> (64 bits)
-   = note: target type: u32 (32 bits)
-
-error: aborting due to 11 previous errors
+error: aborting due to 9 previous errors