Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tracking Issue for porting run-make tests to use Rust #121876

Open
jieyouxu opened this issue Mar 1, 2024 · 36 comments
Open

Tracking Issue for porting run-make tests to use Rust #121876

jieyouxu opened this issue Mar 1, 2024 · 36 comments
Labels
A-run-make Area: port run-make Makefiles to rmake.rs A-testsuite Area: The testsuite used to check the correctness of rustc C-tracking-issue Category: An issue tracking the progress of sth. like the implementation of an RFC T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@jieyouxu
Copy link
Member

jieyouxu commented Mar 1, 2024 •
edited
Loading

We want to stop run-make tests from relying on make, and improve the run-make tests so that
they are more accessible to rustc contributors by allowing the tests to be written in Rust (see
#40713 for context). PR #113026 was merged to address this, and now we are able to write run-make
tests in Rust recipes. We would like your help to port over existing run-make tests still using
Makefiles to use Rust recipes instead.

If you would like to work on porting one of the run-make tests, please link to this issue in your
PR and leave a comment to claim the test (or multiple tests). If you are stuck, please don't hesitate to open a thread
on Rust's Zulip.

run-make is the most flexible fallback test kind, but some run-make tests could be migrated to become e.g. ui tests instead. Please check if you can convert the run-make test into other test kinds before porting!

When you try to port a test, also consider:

  • Add some comments on what the test is trying to test.
  • Add some comments on how the test tries to accomplish its goals if the test is non-trivial. This is especially true if you discovered that a test has platform/compiler/architecture/tooling-specific behavior that is a pain to debug.
  • Any related issues? Any relevant links?
  • Is the test still applicable? Has it become outdated? Is it duplicated?
  • Can it be written in other test suites?

Context: Rust recipes?

PR #113026 adds basic infrastructure support to write run-make tests using small Rust programs,
called recipes.

We aim to eliminate the dependency on make and Makefiles for building run-make-style tests.
Makefiles are replaced by recipes (rmake.rs). The PR implements running run-make recipes in
3 steps:

  1. We build the support library run_make_support which the rmake.rs recipes depend on as a tool
    lib.
  2. We build the recipe rmake.rs and link in the support library.
  3. We run the recipe to build and run the tests.

rmake.rs is basically a replacement for Makefile, and allows running arbitrary Rust code. The
support library is built using cargo, and so can depend on external crates if desired.

The infrastructure implemented by the PR is very barebones, and is the minimally required
infrastructure needed to build, run and pass the two example run-make tests ported over to the new
infrastructure:

You likely will find that you would need to improve the API of the support library, and extend
the functionality of the support library.

Common traps and pitfalls, and tips and tricks

  • $PATH uses ; on Windows and : for *nixes. Use std::env::{join,split}_paths to properly
    handle $PATH.

  • Be careful of path separator platform differences. Always prefer PathBuf operations not string
    paths if possible.

  • tests/ are not (currently) formatted by rustfmt.

  • Consult https://github.com/rust-lang/rust/blob/master/tests/run-make/tools.mk for which flags
    and envs are passed to various executables or libraries. May have to triple check on those.

  • You can always request your reviewer to run try jobs to test out your PR on environments you don't locally have access to. Good candidate CI jobs include:

    • aarch64-apple (apple, 64 bits, aarch64/arm64)
    • armhf-gnu (cross-compile)
    • test-various (cross-compile, wasm)
    • x86_64-mingw (x86_64, 64 bits, windows, mingw)
    • x86_64-msvc (x86_64, 64 bits, windows, msvc)
    • x86_64-gnu-llvm-18 (x86_64, 64 bits, windows, gnu, llvm 18)
    • i686-msvc (x86, 32 bits, windows, msvc)

Tests that need porting
@jieyouxu jieyouxu added A-testsuite Area: The testsuite used to check the correctness of rustc T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. E-help-wanted Call for participation: Help is requested to fix this issue. C-tracking-issue Category: An issue tracking the progress of sth. like the implementation of an RFC labels Mar 1, 2024
@jieyouxu jieyouxu mentioned this issue Mar 1, 2024
Open
4 tasks
@5225225

This comment was marked as resolved.

Sign in to view
@5225225 5225225 mentioned this issue Mar 2, 2024
Merged
@abhay-51 abhay-51 mentioned this issue Mar 2, 2024
Closed
@abhay-51

This comment was marked as resolved.

Sign in to view
@high-cloud

This comment was marked as resolved.

Sign in to view
@high-cloud high-cloud mentioned this issue Mar 13, 2024
Merged
@0xhiro

This comment was marked as resolved.

Sign in to view
@Noratrieb Noratrieb mentioned this issue Mar 14, 2024
Closed
@JoverZhang

This comment was marked as resolved.

Sign in to view
@workingjubilee workingjubilee added the E-mentor Call for participation: This issue has a mentor. Use #t-compiler/help on Zulip for discussion. label Mar 22, 2024
@workingjubilee workingjubilee mentioned this issue Mar 22, 2024
Open
5 tasks
@jieyouxu jieyouxu mentioned this issue Mar 28, 2024
Merged
@Oneirical Oneirical mentioned this issue Mar 29, 2024
Merged
@Oneirical

This comment was marked as resolved.

Sign in to view
@Oneirical Oneirical mentioned this issue Mar 31, 2024
Closed
@Oneirical

This comment was marked as resolved.

Sign in to view
workingjubilee added a commit to workingjubilee/rustc that referenced this issue Mar 31, 2024
@workingjubilee
Rollup merge of rust-lang#123180 - Oneirical:master, r=Mark-Simulacrum
17737bf 
Rewrite `core-no-fp-fmt-parse` test in Rust

Claiming the simple "core-no-fp-fmt-parse" test from rust-lang#121876. `run_make_support` was altered with `arg_path` written in rust-lang#121918 by `@abhay-51,` with additional doc comment.

Preliminary GSoC contribution for the project proposal mentored by `@jieyouxu.`
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Mar 31, 2024
@rust-timer
Unrolled build for rust-lang#123180
004af8a 
Rollup merge of rust-lang#123180 - Oneirical:master, r=Mark-Simulacrum

Rewrite `core-no-fp-fmt-parse` test in Rust

Claiming the simple "core-no-fp-fmt-parse" test from rust-lang#121876. `run_make_support` was altered with `arg_path` written in rust-lang#121918 by `@abhay-51,` with additional doc comment.

Preliminary GSoC contribution for the project proposal mentored by `@jieyouxu.`
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this issue Apr 4, 2024
@matthiaskrgr
Rollup merge of rust-lang#122448 - high-cloud:move-hir-tree, r=oli-obk
0b54db7 
Port hir-tree run-make test to ui test

As part of rust-lang#121876

cc `@jieyouxu`
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Apr 4, 2024
@rust-timer
Unrolled build for rust-lang#122448
56662d6 
Rollup merge of rust-lang#122448 - high-cloud:move-hir-tree, r=oli-obk

Port hir-tree run-make test to ui test

As part of rust-lang#121876

cc `@jieyouxu`
@jieyouxu jieyouxu mentioned this issue Apr 4, 2024
Merged
GuillaumeGomez added a commit to GuillaumeGomez/rust that referenced this issue Apr 5, 2024
@GuillaumeGomez
Rollup merge of rust-lang#123149 - jieyouxu:rmake-arguments-non-c-lik…
c36c009 
…e-enum, r=Mark-Simulacrum

Port argument-non-c-like-enum to Rust

Part of rust-lang#121876.
GuillaumeGomez added a commit to GuillaumeGomez/rust that referenced this issue Apr 5, 2024
@GuillaumeGomez
Rollup merge of rust-lang#123474 - jieyouxu:issue-7349-port, r=Mark-S…
0d5ee65 
…imulacrum

Port `run-make/issue-7349` to a codegen test

The test does not need to be a run-make test, it can use the codegen test infrastructure.

Also took the opportunity to rename the test to `no-redundant-item-monomorphization` so it's not just some opaque issue number.

Part of rust-lang#121876.
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Apr 5, 2024
@rust-timer
Unrolled build for rust-lang#123474
c8eeddc 
Rollup merge of rust-lang#123474 - jieyouxu:issue-7349-port, r=Mark-Simulacrum

Port `run-make/issue-7349` to a codegen test

The test does not need to be a run-make test, it can use the codegen test infrastructure.

Also took the opportunity to rename the test to `no-redundant-item-monomorphization` so it's not just some opaque issue number.

Part of rust-lang#121876.
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Apr 5, 2024
@rust-timer
Unrolled build for rust-lang#123149
4e6c7f7 
Rollup merge of rust-lang#123149 - jieyouxu:rmake-arguments-non-c-like-enum, r=Mark-Simulacrum

Port argument-non-c-like-enum to Rust

Part of rust-lang#121876.
This was referenced Nov 14, 2024
Merged
Merged
Merged
Open
This was referenced Nov 22, 2024
Merged
Merged
Open
Merged
Merged
Merged
Merged
Merged
This was referenced Nov 30, 2024
Merged
Merged
Merged
Closed
Merged
This was referenced Dec 7, 2024
Open
Merged
Merged
Open
This was referenced Dec 14, 2024
Open
Open
@rustbot rustbot mentioned this issue Dec 23, 2024
Merged
@jieyouxu jieyouxu mentioned this issue Dec 23, 2024
Merged
This was referenced Dec 23, 2024
Merged
Open
@jieyouxu
Copy link
Member Author

jieyouxu commented Dec 25, 2024 •
edited
Loading

Re-triage (2024-12-25):

Test Latest PR Status
branch-protection-check-IBT #134760 Has both Makefile and rmake.rs but never actually ran due to bug in Makefile version; needs investigation. Test fixed in #134760.
cat-and-grep-sanity-check #134762 Helper test. Should only delete when there are no longer any Makefiles.
extern-fn-reachable #128314 PR author busy, I'll carry it over the finish line when I get to it.
jobserver-error #128789 jobserver stuff... tricky, waiting on review.
libs-through-symlinks #129011 Previously blocked on various symlink bugs that are now fixed in #134659, waiting on review.
split-debuginfo #128754 Just long, I haven't gotten to reviewing it.
symbol-mangling-hashed #128567 Waiting on helpers from #128314, I'll carry it over the finish line when I get to it.
translation #129011 Previously blocked on various symlink bugs that are now fixed in #134659, waiting on review.

GuillaumeGomez added a commit to GuillaumeGomez/rust that referenced this issue Dec 26, 2024
@GuillaumeGomez
Rollup merge of rust-lang#134656 - jieyouxu:migrate-incr-add-rust-src…
c016cd8 
…-component, r=wesleywiser

Migrate `incr-add-rust-src-component` to rmake

This PR partially supersedes rust-lang#128562, and ports the Makefile-based `tests/run-make/incr-add-rust-src-component` to use rmake.rs infra.

Part of rust-lang#121876.

This run-make test is a regression test for rust-lang#70924. It (tries to) checks that if we add the `rust-src` component in between two incremental compiles, that the compiler doesn't ICE on the second invocation.

- Original issue:rust-lang#70924
- Fix PR: rust-lang#72767
- PR adding this regression test: rust-lang#72952

However, the Makefile version of this used `$SYSROOT/lib/rustlib/src/rust/src/libstd/lib.rs`, but that actually got moved around and reorganized over the years. As of Dec 2024, the `rust-src` component is more like (specific for our purposes):

```
$SYSROOT/lib/rustlib/src/rust/
    library/std/src/lib.rs
    src/
```

However, this run-make test is ancient and it exercises incr-comp system logic. I'm not sure if this test would actually catch the original regression.

This PR was co-authored with `@Oneirical.`

r? incremental

try-job: i686-msvc
try-job: x86_64-mingw-1
try-job: x86_64-msvc
try-job: aarch64-apple
rust-timer added a commit to rust-lang-ci/rust that referenced this issue Dec 26, 2024
@rust-timer
Unrolled build for rust-lang#134656
0a5a094 
Rollup merge of rust-lang#134656 - jieyouxu:migrate-incr-add-rust-src-component, r=wesleywiser

Migrate `incr-add-rust-src-component` to rmake

This PR partially supersedes rust-lang#128562, and ports the Makefile-based `tests/run-make/incr-add-rust-src-component` to use rmake.rs infra.

Part of rust-lang#121876.

This run-make test is a regression test for rust-lang#70924. It (tries to) checks that if we add the `rust-src` component in between two incremental compiles, that the compiler doesn't ICE on the second invocation.

- Original issue:rust-lang#70924
- Fix PR: rust-lang#72767
- PR adding this regression test: rust-lang#72952

However, the Makefile version of this used `$SYSROOT/lib/rustlib/src/rust/src/libstd/lib.rs`, but that actually got moved around and reorganized over the years. As of Dec 2024, the `rust-src` component is more like (specific for our purposes):

```
$SYSROOT/lib/rustlib/src/rust/
    library/std/src/lib.rs
    src/
```

However, this run-make test is ancient and it exercises incr-comp system logic. I'm not sure if this test would actually catch the original regression.

This PR was co-authored with `@Oneirical.`

r? incremental

try-job: i686-msvc
try-job: x86_64-mingw-1
try-job: x86_64-msvc
try-job: aarch64-apple
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-run-make Area: port run-make Makefiles to rmake.rs A-testsuite Area: The testsuite used to check the correctness of rustc C-tracking-issue Category: An issue tracking the progress of sth. like the implementation of an RFC T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
Oxidizing run-make tests
Status: In progress
Milestone
No milestone
Development

No branches or pull requests
15 participants
@ChrisDenton @Kobzol @petrochenkov @5225225 @lolbinarycat @Zalathar @high-cloud @JoverZhang @jieyouxu @workingjubilee @abhay-51 @Rejyr @Oneirical @0xhiro and others