Add job to test flakiness of added/changed tests

[ijjk]

To help detect when newly added/changed assertions are flakey this adds a job to our build and test workflow to re-run them 3 times. If the changed test is an E2E test it will re-run in both development and production mode to ensure it's not flakey specifically in one of those modes.

Test run with changed test can be seen here https://github.com/vercel/next.js/actions/runs/8511797725/job/23312158523?pr=63943

Closes NEXT-2973

[ijjk]

Stats from current PR

Default Build (Increase detected ⚠️)

General

	vercel/next.js canary	ijjk/next.js ci/flakey-tests	Change
buildDuration	13.7s	13.8s	⚠️ +110ms
buildDurationCached	7.4s	6.1s	N/A
nodeModulesSize	199 MB	199 MB	✓
nextStartRea..uration (ms)	399ms	414ms	N/A

Client Bundles (main, webpack)

	vercel/next.js canary	ijjk/next.js ci/flakey-tests	Change
2453-HASH.js gzip	31.4 kB	31.4 kB	N/A
3304.HASH.js gzip	181 B	181 B	✓
3f784ff6-HASH.js gzip	53.7 kB	53.7 kB	✓
8299-HASH.js gzip	5.04 kB	5.04 kB	N/A
framework-HASH.js gzip	45.2 kB	45.2 kB	✓
main-app-HASH.js gzip	241 B	242 B	N/A
main-HASH.js gzip	32.2 kB	32.2 kB	N/A
webpack-HASH.js gzip	1.68 kB	1.68 kB	N/A
Overall change	99 kB	99 kB	✓

Legacy Client Bundles (polyfills)

	vercel/next.js canary	ijjk/next.js ci/flakey-tests	Change
polyfills-HASH.js gzip	31 kB	31 kB	✓
Overall change	31 kB	31 kB	✓

Client Pages

	vercel/next.js canary	ijjk/next.js ci/flakey-tests	Change
_app-HASH.js gzip	196 B	197 B	N/A
_error-HASH.js gzip	184 B	184 B	✓
amp-HASH.js gzip	505 B	505 B	✓
css-HASH.js gzip	324 B	325 B	N/A
dynamic-HASH.js gzip	2.5 kB	2.5 kB	N/A
edge-ssr-HASH.js gzip	258 B	258 B	✓
head-HASH.js gzip	352 B	352 B	✓
hooks-HASH.js gzip	370 B	371 B	N/A
image-HASH.js gzip	4.21 kB	4.21 kB	✓
index-HASH.js gzip	259 B	259 B	✓
link-HASH.js gzip	2.67 kB	2.67 kB	N/A
routerDirect..HASH.js gzip	314 B	312 B	N/A
script-HASH.js gzip	386 B	386 B	✓
withRouter-HASH.js gzip	309 B	309 B	✓
1afbb74e6ecf..834.css gzip	106 B	106 B	✓
Overall change	6.57 kB	6.57 kB	✓

Client Build Manifests

	vercel/next.js canary	ijjk/next.js ci/flakey-tests	Change
_buildManifest.js gzip	481 B	484 B	N/A
Overall change	0 B	0 B	✓

Rendered Page Sizes

	vercel/next.js canary	ijjk/next.js ci/flakey-tests	Change
index.html gzip	529 B	527 B	N/A
link.html gzip	541 B	540 B	N/A
withRouter.html gzip	524 B	522 B	N/A
Overall change	0 B	0 B	✓

Edge SSR bundle Size

	vercel/next.js canary	ijjk/next.js ci/flakey-tests	Change
edge-ssr.js gzip	95.3 kB	95.3 kB	N/A
page.js gzip	3.04 kB	3.04 kB	N/A
Overall change	0 B	0 B	✓

Middleware size

	vercel/next.js canary	ijjk/next.js ci/flakey-tests	Change
middleware-b..fest.js gzip	624 B	622 B	N/A
middleware-r..fest.js gzip	151 B	151 B	✓
middleware.js gzip	25.5 kB	25.5 kB	N/A
edge-runtime..pack.js gzip	839 B	839 B	✓
Overall change	990 B	990 B	✓

Next Runtimes

	vercel/next.js canary	ijjk/next.js ci/flakey-tests	Change
app-page-exp...dev.js gzip	170 kB	170 kB	✓
app-page-exp..prod.js gzip	97 kB	97 kB	✓
app-page-tur..prod.js gzip	98.8 kB	98.8 kB	✓
app-page-tur..prod.js gzip	93 kB	93 kB	✓
app-page.run...dev.js gzip	144 kB	144 kB	✓
app-page.run..prod.js gzip	91.5 kB	91.5 kB	✓
app-route-ex...dev.js gzip	21.4 kB	21.4 kB	✓
app-route-ex..prod.js gzip	15.2 kB	15.2 kB	✓
app-route-tu..prod.js gzip	15.2 kB	15.2 kB	✓
app-route-tu..prod.js gzip	14.9 kB	14.9 kB	✓
app-route.ru...dev.js gzip	21.1 kB	21.1 kB	✓
app-route.ru..prod.js gzip	14.9 kB	14.9 kB	✓
pages-api-tu..prod.js gzip	9.55 kB	9.55 kB	✓
pages-api.ru...dev.js gzip	9.82 kB	9.82 kB	✓
pages-api.ru..prod.js gzip	9.55 kB	9.55 kB	✓
pages-turbo...prod.js gzip	22.5 kB	22.5 kB	✓
pages.runtim...dev.js gzip	23.1 kB	23.1 kB	✓
pages.runtim..prod.js gzip	22.4 kB	22.4 kB	✓
server.runti..prod.js gzip	51 kB	51 kB	✓
Overall change	945 kB	945 kB	✓

build cache Overall increase ⚠️

	vercel/next.js canary	ijjk/next.js ci/flakey-tests	Change
0.pack gzip	1.58 MB	1.58 MB	N/A
index.pack gzip	105 kB	106 kB	⚠️ +586 B
Overall change	105 kB	106 kB	⚠️ +586 B

Diff details

Diff for middleware.js

Diff too large to display

Commit: 656d019

[Ethan-Arrowood]

Reviewed with @WITS during donut call.

changes