feat(tests): Run Playwright tests using bundles. #4551

onurtemizkan · 2022-02-11T12:08:10Z

Partly Resolves: #4546 (Resolves for Playwright, not for Karma integration tests)

This implementation heavily depends on hard-coded configuration, to filter modules or search paths, as the bundle names and paths don't have a pattern.

Built JavaScript modules (esm and dist) are used as webpack aliases, and imported by webpack, which I understand is how they're generally consumed.

Bundles are injected as <script>s into the generated HTML <head>.

github-actions · 2022-02-11T12:15:52Z

size-limit report

Path	Base Size (ab73aa8)	Current Size	Change
@sentry/browser - ES5 CDN Bundle (gzipped + minified)	19.77 KB	19.77 KB	0%
@sentry/browser - ES5 CDN Bundle (minified)	63.47 KB	63.47 KB	0%
@sentry/browser - ES6 CDN Bundle (gzipped + minified)	18.44 KB	18.44 KB	+0.02% 🔺
@sentry/browser - ES6 CDN Bundle (minified)	56.6 KB	56.6 KB	0%
@sentry/browser - Webpack (gzipped + minified)	22.2 KB	22.2 KB	0%
@sentry/browser - Webpack (minified)	76.36 KB	76.36 KB	0%
@sentry/react - Webpack (gzipped + minified)	22.23 KB	22.23 KB	0%
@sentry/nextjs Client - Webpack (gzipped + minified)	46.4 KB	46.4 KB	0%
@sentry/browser + @sentry/tracing - ES5 CDN Bundle (gzipped + minified)	27.23 KB	27.23 KB	+0.01% 🔺

… into onur/test-bundles

AbhiPrasad · 2022-02-11T14:02:30Z

Resolves for Playwright, not for Karma integration tests

I think this is fine, as we want playwright to be where we focus on from now on.

onurtemizkan · 2022-02-11T14:37:00Z

Found double-importing issue when testing bundles. Will update in a bit.

onurtemizkan · 2022-02-11T19:30:11Z

Could not avoid updating 30 templates after all. :/
I'll look for ways to lighten this configuration a bit, but this version works.

lobsterkatie

This is a great start, thank you!

I left a few comments below, but I think the biggest thing right now is to address the ES6/tracing bundle question. See #4551 (comment).

packages/integration-tests/README.md

packages/integration-tests/utils/generatePage.ts

.github/workflows/build.yml

packages/integration-tests/suites/public-api/addBreadcrumb/template.hbs

Co-authored-by: Katie Byers <lobsterkatie@gmail.com>

… into onur/test-bundles

onurtemizkan · 2022-02-22T19:30:07Z

@lobsterkatie, @AbhiPrasad

I ended up implementing a tiny webpack plugin to detect which SDKs are imported by init.js and inject a bundle accordingly.

Also switched from handlebars templates to plain HTML templates and make HTMLWebpackPlugin to inject:

@sentry/browser or @sentry/tracing bundle [optional]
Sentry initialization script
Test subject script

to the end of <body> with the given order.

Previously, we were adding init script to <head> and subject to <body>, but that doesn't seem needed with the current set of tests (unless there is a case where we need that, you might think of). So I could also delete those duplicate standard templates, switching to a global default.

Also added another env variable PW_TRACING_ONLY to only use @sentry/tracing bundles for all tests, including ones that don't need tracing. Added tracing-only jobs for all bundles on CI.

AbhiPrasad · 2022-02-23T17:02:45Z

Hey, you'll have to update from master since we fixed some test failures!

We'll be taking a look at this soon to review, thanks for all the help @onurtemizkan!

AbhiPrasad

@lobsterkatie you think we can merge this in and add the es6 tracing bundle afterwards?

packages/integration-tests/README.md

… into onur/test-bundles

lobsterkatie

I like the plugin!

I left a few more comments, mostly just trying to make it clearer what various parts of the code mean, but overall this seems like it's in good shape! My only other question is what the empty files do. Will they ever be used, or are they just placeholders?

As for ES6 tracing, @AbhiPrasad, sure, I'm fine with merging this and adding the ES6 tracing bundle afterwards. The way this is set up now doesn't necessitate it the way the old setup did.

packages/integration-tests/README.md

packages/integration-tests/utils/generatePlugin.ts

packages/integration-tests/utils/fixtures.ts

onurtemizkan · 2022-02-24T23:06:45Z

Thanks for the review @lobsterkatie!

Updated 👍

My only other question is what the empty files do. Will they ever be used, or are they just placeholders?

They were just placeholders, but not required with the current set of tests, so removed them.

lobsterkatie

One small comment fix, but otherwise this looks good to go!

Thanks, @onurtemizkan!

packages/integration-tests/utils/generatePlugin.ts

Co-authored-by: Katie Byers <lobsterkatie@gmail.com>

AbhiPrasad · 2022-02-28T16:58:17Z

We might have to remove Browser Playwright Tests as a required check.

feat(test): Run Playwright tests using bundles.

34de27d

onurtemizkan requested review from AbhiPrasad and lobsterkatie February 11, 2022 12:09

onurtemizkan changed the title ~~feat(test): Run Playwright tests using bundles.~~ feat(tests): Run Playwright tests using bundles. Feb 11, 2022

onurtemizkan added 5 commits February 11, 2022 12:22

Merge branch 'master' of https://github.com/getsentry/sentry-javascript…

f64c953

… into onur/test-bundles

Simplify generation logic.

6389f7d

Fix unsafe lookup.

8aa54db

Document code, update README.

3586b49

Remove unnecessary Bundle prefix.

ea4044f

onurtemizkan marked this pull request as ready for review February 11, 2022 14:01

AbhiPrasad added this to the Release Stability milestone Feb 11, 2022

onurtemizkan marked this pull request as draft February 11, 2022 14:35

Move bundle injections to their own HTML scripts.

05692e2

onurtemizkan marked this pull request as ready for review February 11, 2022 19:31

onurtemizkan added 3 commits February 11, 2022 19:33

Remove unused filter.

de4e070

Remove empty meta and title elements from templates.

b90bd61

Move misplaced script.

ad1918d

lobsterkatie reviewed Feb 11, 2022

View reviewed changes

onurtemizkan and others added 2 commits February 14, 2022 10:31

Update comments.

2897b84

Co-authored-by: Katie Byers <lobsterkatie@gmail.com>

Remove default external types and Integration external.

627de51

onurtemizkan force-pushed the onur/test-bundles branch from 5602c3d to 627de51 Compare February 14, 2022 10:45

onurtemizkan added 5 commits February 14, 2022 11:05

Rename dists to cjs.

a7bc9de

Remove unnecessary check.

14cbf6a

Remove typo.

efe3ced

Merge branch 'master' of https://github.com/getsentry/sentry-javascript…

0659941

… into onur/test-bundles

Use a custom webpack plugin to generate scenarios.

3687d0c

onurtemizkan force-pushed the onur/test-bundles branch from 4e10222 to 3687d0c Compare February 19, 2022 16:43

Run tracing-only bundle tests.

f69b8a9

onurtemizkan force-pushed the onur/test-bundles branch from bed88fd to f69b8a9 Compare February 21, 2022 10:01

Merge branch 'master' of https://github.com/getsentry/sentry-javascript…

2d00b51

… into onur/test-bundles

AbhiPrasad reviewed Feb 23, 2022

View reviewed changes

packages/integration-tests/README.md Outdated Show resolved Hide resolved

Merge branch 'master' of https://github.com/getsentry/sentry-javascript…

f4c0ebd

… into onur/test-bundles

lobsterkatie reviewed Feb 24, 2022

View reviewed changes

Address review comments.

69ae75a

onurtemizkan force-pushed the onur/test-bundles branch from 20aebe7 to 69ae75a Compare February 24, 2022 22:06

Remove unused default assets.

2c0f8aa

lobsterkatie approved these changes Feb 28, 2022

View reviewed changes

packages/integration-tests/utils/generatePlugin.ts Outdated Show resolved Hide resolved

AbhiPrasad approved these changes Feb 28, 2022

View reviewed changes

Update packages/integration-tests/utils/generatePlugin.ts

11caa05

Co-authored-by: Katie Byers <lobsterkatie@gmail.com>

AbhiPrasad enabled auto-merge (squash) February 28, 2022 16:57

AbhiPrasad disabled auto-merge February 28, 2022 18:24

AbhiPrasad merged commit 0a4f278 into master Feb 28, 2022

AbhiPrasad deleted the onur/test-bundles branch February 28, 2022 18:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(tests): Run Playwright tests using bundles. #4551

feat(tests): Run Playwright tests using bundles. #4551

onurtemizkan commented Feb 11, 2022 •

edited

Loading

github-actions bot commented Feb 11, 2022 •

edited

Loading

AbhiPrasad commented Feb 11, 2022

onurtemizkan commented Feb 11, 2022

onurtemizkan commented Feb 11, 2022 •

edited

Loading

lobsterkatie left a comment •

edited

Loading

onurtemizkan commented Feb 22, 2022

AbhiPrasad commented Feb 23, 2022

AbhiPrasad left a comment

lobsterkatie left a comment

onurtemizkan commented Feb 24, 2022

lobsterkatie left a comment

AbhiPrasad commented Feb 28, 2022

feat(tests): Run Playwright tests using bundles. #4551

feat(tests): Run Playwright tests using bundles. #4551

Conversation

onurtemizkan commented Feb 11, 2022 • edited Loading

github-actions bot commented Feb 11, 2022 • edited Loading

size-limit report

AbhiPrasad commented Feb 11, 2022

onurtemizkan commented Feb 11, 2022

onurtemizkan commented Feb 11, 2022 • edited Loading

lobsterkatie left a comment • edited Loading

Choose a reason for hiding this comment

onurtemizkan commented Feb 22, 2022

AbhiPrasad commented Feb 23, 2022

AbhiPrasad left a comment

Choose a reason for hiding this comment

lobsterkatie left a comment

Choose a reason for hiding this comment

onurtemizkan commented Feb 24, 2022

lobsterkatie left a comment

Choose a reason for hiding this comment

AbhiPrasad commented Feb 28, 2022

onurtemizkan commented Feb 11, 2022 •

edited

Loading

github-actions bot commented Feb 11, 2022 •

edited

Loading

onurtemizkan commented Feb 11, 2022 •

edited

Loading

lobsterkatie left a comment •

edited

Loading