CI: test with --with-snapshot on the slower platforms

[refack]

• Version: 8.1.4
• Platform: *
• Subsystem: test,CI

Ref: #14214 (comment)
Should we run the CI tests configured --with-snapshot for the slower (Δt > 50%) platforms? Just so we get back to reasonable time frames? (node-test-commit-arm up to ~1h38m from ~29m)

/cc @nodejs/testing @nodejs/release @nodejs/build

[evanlucas]

I don't think that it is a good idea to run the tests with snapshots, but release builds without snapshots.

[cjihrig]

I agree with @evanlucas. And, as painful as the CI may currently be, I think we should try to work through the snapshot issue before taking any drastic (maybe too strong of a word) measures.

[refack]

Random sample Δt numbers:
#14214 (comment)
without - https://ci.nodejs.org/job/node-test-commit/11091/
with - https://ci.nodejs.org/job/node-test-commit/11045/

platform	with(minutes)	without(minutes)	Δt	good enough
node-test-commit-aix	38	42	10%	✔️
node-test-commit-arm	29	98	237%	🔴
node-test-commit-arm-fanned	35	70	100%	🔴
node-test-commit-freebsd	19	53	178%	🔴
node-test-commit-linux	27	32	18%	✔️
node-test-commit-linux-fips	30	67	123%	🔴
node-test-commit-linuxone	3	6.5	116%	✔️
node-test-commit-osx	19	25	30%	✔️
node-test-commit-plinux	8.5	24	182%	🔴
node-test-commit-smartos	16	21	131%	🔴
node-test-commit-windows-fanned	39	40	2%	✔️

[refack]

Compromise: a check box in the submit job form?

[mhdawson]

I'm in agreement that we need to test in the same way that we ship.

[refack]

I want to make an argument for leniency.

1. We already make some trade offs in our test suite (skipping /pummel/ and /internet/).
2. At some point we'll want to restore the snapshot and we don't want that to rot.
3. Building with --with-snapshot is still a valid use case for those who build from source.

tl;dr adding a checkbox (enable / disable snapshot) in the job submission form might be a good thing, by it increasing coverage.

[gibfahn]

@refack snapshots have been re-enabled so this can be closed right?

Reopen if I'm wrong.