Always throw error when build fails during test

[timneutkens]

What?

When creating the Next.js instance fails for whatever reason (most commonly next build failing) it currently exits the process, this causes Jest to not report these tests as failed.

The change in this PR ensures that anytime creating the instance fails it throws an error that is caught by Jest and then causes Jest to report the tests as failed.

The reporting of tests is relevant as our monitoring tools (i.e. DataDog / Turbopack test manifest) rely on correct reporting of all failing tests.

Closes NEXT-3016

[timneutkens]

• Always throw error when build fails during test #64116 👈
• canary

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @timneutkens and the rest of your teammates on Graphite

[ijjk]

Stats from current PR

Default Build (Increase detected ⚠️)

General

	vercel/next.js canary	vercel/next.js 04-05-Always_throw_error_when_build_fails_during_test	Change
buildDuration	13.9s	13.9s	N/A
buildDurationCached	7.5s	6.2s	N/A
nodeModulesSize	199 MB	199 MB	✓
nextStartRea..uration (ms)	401ms	400ms	N/A

Client Bundles (main, webpack)

	vercel/next.js canary	vercel/next.js 04-05-Always_throw_error_when_build_fails_during_test	Change
2453-HASH.js gzip	31.4 kB	31.4 kB	N/A
3304.HASH.js gzip	181 B	181 B	✓
3f784ff6-HASH.js gzip	53.7 kB	53.7 kB	✓
8299-HASH.js gzip	5.04 kB	5.04 kB	N/A
framework-HASH.js gzip	45.2 kB	45.2 kB	✓
main-app-HASH.js gzip	242 B	241 B	N/A
main-HASH.js gzip	32.2 kB	32.2 kB	N/A
webpack-HASH.js gzip	1.68 kB	1.68 kB	N/A
Overall change	99 kB	99 kB	✓

Legacy Client Bundles (polyfills)

	vercel/next.js canary	vercel/next.js 04-05-Always_throw_error_when_build_fails_during_test	Change
polyfills-HASH.js gzip	31 kB	31 kB	✓
Overall change	31 kB	31 kB	✓

Client Pages

	vercel/next.js canary	vercel/next.js 04-05-Always_throw_error_when_build_fails_during_test	Change
_app-HASH.js gzip	196 B	197 B	N/A
_error-HASH.js gzip	184 B	184 B	✓
amp-HASH.js gzip	505 B	505 B	✓
css-HASH.js gzip	324 B	325 B	N/A
dynamic-HASH.js gzip	2.5 kB	2.5 kB	N/A
edge-ssr-HASH.js gzip	258 B	258 B	✓
head-HASH.js gzip	352 B	352 B	✓
hooks-HASH.js gzip	370 B	371 B	N/A
image-HASH.js gzip	4.21 kB	4.21 kB	✓
index-HASH.js gzip	259 B	259 B	✓
link-HASH.js gzip	2.67 kB	2.67 kB	N/A
routerDirect..HASH.js gzip	314 B	312 B	N/A
script-HASH.js gzip	386 B	386 B	✓
withRouter-HASH.js gzip	309 B	309 B	✓
1afbb74e6ecf..834.css gzip	106 B	106 B	✓
Overall change	6.57 kB	6.57 kB	✓

Client Build Manifests

	vercel/next.js canary	vercel/next.js 04-05-Always_throw_error_when_build_fails_during_test	Change
_buildManifest.js gzip	481 B	484 B	N/A
Overall change	0 B	0 B	✓

Rendered Page Sizes

	vercel/next.js canary	vercel/next.js 04-05-Always_throw_error_when_build_fails_during_test	Change
index.html gzip	529 B	529 B	✓
link.html gzip	541 B	541 B	✓
withRouter.html gzip	524 B	524 B	✓
Overall change	1.59 kB	1.59 kB	✓

Edge SSR bundle Size

	vercel/next.js canary	vercel/next.js 04-05-Always_throw_error_when_build_fails_during_test	Change
edge-ssr.js gzip	95.4 kB	95.4 kB	N/A
page.js gzip	3.06 kB	3.06 kB	N/A
Overall change	0 B	0 B	✓

Middleware size

	vercel/next.js canary	vercel/next.js 04-05-Always_throw_error_when_build_fails_during_test	Change
middleware-b..fest.js gzip	626 B	625 B	N/A
middleware-r..fest.js gzip	151 B	151 B	✓
middleware.js gzip	25.5 kB	25.5 kB	N/A
edge-runtime..pack.js gzip	839 B	839 B	✓
Overall change	990 B	990 B	✓

Next Runtimes

	vercel/next.js canary	vercel/next.js 04-05-Always_throw_error_when_build_fails_during_test	Change
app-page-exp...dev.js gzip	170 kB	170 kB	✓
app-page-exp..prod.js gzip	97 kB	97 kB	✓
app-page-tur..prod.js gzip	98.8 kB	98.8 kB	✓
app-page-tur..prod.js gzip	93 kB	93 kB	✓
app-page.run...dev.js gzip	144 kB	144 kB	✓
app-page.run..prod.js gzip	91.5 kB	91.5 kB	✓
app-route-ex...dev.js gzip	21.4 kB	21.4 kB	✓
app-route-ex..prod.js gzip	15.1 kB	15.1 kB	✓
app-route-tu..prod.js gzip	15.1 kB	15.1 kB	✓
app-route-tu..prod.js gzip	14.9 kB	14.9 kB	✓
app-route.ru...dev.js gzip	21.1 kB	21.1 kB	✓
app-route.ru..prod.js gzip	14.9 kB	14.9 kB	✓
pages-api-tu..prod.js gzip	9.55 kB	9.55 kB	✓
pages-api.ru...dev.js gzip	9.82 kB	9.82 kB	✓
pages-api.ru..prod.js gzip	9.55 kB	9.55 kB	✓
pages-turbo...prod.js gzip	22.5 kB	22.5 kB	✓
pages.runtim...dev.js gzip	23.1 kB	23.1 kB	✓
pages.runtim..prod.js gzip	22.4 kB	22.4 kB	✓
server.runti..prod.js gzip	51.1 kB	51.1 kB	✓
Overall change	945 kB	945 kB	✓

build cache Overall increase ⚠️

	vercel/next.js canary	vercel/next.js 04-05-Always_throw_error_when_build_fails_during_test	Change
0.pack gzip	1.58 MB	1.58 MB	N/A
index.pack gzip	105 kB	107 kB	⚠️ +1.05 kB
Overall change	105 kB	107 kB	⚠️ +1.05 kB

Diff details

Diff for middleware.js

Diff too large to display

Commit: dd7a93a