Make compiletest output truncation less disruptive

[RalfJung]

When the test output becomes too large, compiletest stops recording all of it. However:

• this can lead to invalid JSON, which then causes compiletest itself to throw further errors
• the note that output was truncated is in the middle of the output, with >100kb of text on each side; that makes it almost impossible to actually see that note in the terminal

So assuming that we do need to keep the output truncation, I propose that we only ever do a cut at the end, so that it is very clear by looking at the end of the log that truncation happened. I added a message at the beginning of the output as well. Also I added some logic to make it less likely that we'll cut things off in the middle of a JSON record. (I tested that successfully by reducing the output limit to something very low and running a few ui tests.) Furthermore I increased the max buffer size to 512KB; that's really not a lot of memory compared to how much RAM it takes to build rustc (it's ~25% more than the previous maximum HEAD+TAIL length). And finally, the information that things got truncated is now propagated to the higher levels, so that we can fail the test instead of comparing the truncated output with the reference.

Fixes #115675
Fixes #96229
Fixes #94322
Fixes #92211

[rustbot]

r? @wesleywiser

(rustbot has picked a reviewer for you, use r? to override)

[wesleywiser]

@bors r+ rollup

[bors]

📌 Commit 9cf2798 has been approved by wesleywiser

It is now in the queue for this repository.

[bors]

⌛ Testing commit 9cf2798 with merge 8e455db...

[bors]

☀️ Test successful - checks-actions
Approved by: wesleywiser
Pushing 8e455db to master...

[bors]

☀️ Test successful - checks-actions
Approved by: wesleywiser
Pushing 8e455db to master...

[rust-timer]

Finished benchmarking commit (8e455db): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.3%	[2.3%, 2.3%]	1
Improvements ✅ (primary)	-3.0%	[-3.0%, -3.0%]	1
Improvements ✅ (secondary)	-1.7%	[-1.7%, -1.7%]	1
All ❌✅ (primary)	-3.0%	[-3.0%, -3.0%]	1

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 631.641s -> 631.557s (-0.01%)
Artifact size: 317.90 MiB -> 317.99 MiB (0.03%)