Add unstable support for outputting file checksums for use in cargo

[Xaeroxe]

Adds an unstable option that appends file checksums and expected lengths to the end of the dep-info file such that cargo can read and use these values as an alternative to file mtimes.

This PR powers the changes made in this cargo PR rust-lang/cargo#14137

Here's the tracking issue for the cargo feature rust-lang/cargo#14136.

[rustbot]

r? @fmease

rustbot has assigned @fmease.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[Urgau]

Given that this PR adds a new concept and a new flag it requires a minimum of discussion within the team.
Could you start a T-compiler Major Change Proposol following the guide.

[rustbot]

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

[Xaeroxe]

This now has an MCP rust-lang/compiler-team#765

[fmease]

Waiting for rust-lang/compiler-team#765 FCP to complete. That is 2024-07-11 07:23h UTC.
@rustbot blocked

[Xaeroxe]

The MCP has been accepted

@rustbot ready

[rustbot]

The list of allowed third-party dependencies may have been modified! You must ensure that any new dependencies have compatible licenses before merging.

cc @davidtwco, @wesleywiser

[Xaeroxe]

Correct, blake3 has been added as it was mentioned during the MCP review as being desirable.

[Urgau]

We had some LLVM BOLT issues with the blake3 crate in #127754, a couple of days ago, on a simple try build.
Let's try this PR, to see if is also has those compilation issues.

@bors try (@Xaeroxe could you fix the conflicts please)

[Xaeroxe]

@Urgau conflicts resolved.

[Urgau]

@bors try @rust-timer queue

[bors]

⌛ Trying commit d80d73e with merge c5ac80a...

[bors]

☀️ Try build successful - checks-actions
Build commit: c5ac80a (c5ac80a53be9c08bed91b91be19981d3bf07821c)

[rust-timer]

Finished benchmarking commit (c5ac80a): comparison URL.

Overall result: ❌✅ regressions and improvements - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	1.3%	[0.3%, 2.3%]	2
Improvements ✅ (primary)	-0.5%	[-0.5%, -0.5%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.5%	[-0.5%, -0.5%]	1

Max RSS (memory usage)

This benchmark run did not return any relevant results for this metric.

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 768.642s -> 770.102s (0.19%)
Artifact size: 328.71 MiB -> 328.75 MiB (0.01%)

[Xaeroxe]

Merge conflicts resolved, this is ready for review.

[bors]

☔ The latest upstream changes (presumably #125443) made this pull request unmergeable. Please resolve the merge conflicts.

[Xaeroxe]

Merge conflicts resolved.

[bors]

☔ The latest upstream changes (presumably #128534) made this pull request unmergeable. Please resolve the merge conflicts.

[workingjubilee]

hm.

r? compiler

[lcnr]

r? compiler

[weihanglo]

This feels like inverting the dependency relationship. The Cargo team and the compiler team can agree on a set of algorithms, though in compiler code we don't need to mention what is supported in Cargo. Cargo can choose whatever algorithm it uses.