Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ref(nextjs): Wrap server-side data-fetching methods during build #5503

Merged
merged 10 commits into from
Aug 8, 2022

Conversation

lobsterkatie
Copy link
Member

@lobsterkatie lobsterkatie commented Aug 1, 2022

Ref: Ref: #5505

This adds wrapping of the nextjs server-side data-fetching methods (getStaticPaths, getStaticProps, and getServerSideProps) to the webpack config changes we make during app build. Eventually, this will let us create spans for them, use them to improve route parameterization, and trace requests which aren't currently getting traced on Vercel-deployed nextjs apps. For the moment, though, all the wrappers do is call the functions they wrap, in order to separate out the wrapping into its own PR.

Notes:

  • For the moment, in order to allow us to iterate on it without fear of breaking people's apps, this feature is gated behind the sentry.autoWrapDataFetchers flag in next.config.js.

  • To do the wrapping, we scan each page's AST for references to each of the three functions. If a given function is found, it is renamed (and all references to it similarly adjusted) so that its name starts with one or more underscores (so getServerSideProps becomes _getServerSideProps, for example). This then allows us to create a new, wrapped version of the (now-renamed) function, and export that wrapped version under the original name. (For example, we can then add export const getServerSideProps = withSentryGSSP(_getServerSideProps) to the page code using the loader, and nextjs will never know the difference.)

  • Parsing of user code into an AST (and subsequent traversal of that AST) is done using @babel/parser and jscodeshift. (@babel/parser is used directly because jscodeshift neither exposes its parsers nor provides an option for specifying a parser by name (at least when used programmatically). Therefore, in order to use a jsx parser for JS pages and a tsx parser for TS pages, we need to supply them ourselves.)

Follow-up work:

  • Add tests
  • Make these wrappers do more than call the original functions

@smeubank smeubank mentioned this pull request Aug 1, 2022
43 tasks
@smeubank smeubank linked an issue Aug 1, 2022 that may be closed by this pull request
43 tasks
@lobsterkatie lobsterkatie force-pushed the kmclb-nextjs-wrap-data-fetching-methods branch from f588a2b to f368c40 Compare August 2, 2022 04:21
@github-actions
Copy link
Contributor

github-actions bot commented Aug 2, 2022

size-limit report 📦

Path Size
@sentry/browser - ES5 CDN Bundle (gzipped + minified) 19.37 KB (+0.02% 🔺)
@sentry/browser - ES5 CDN Bundle (minified) 59.96 KB (0%)
@sentry/browser - ES6 CDN Bundle (gzipped + minified) 17.94 KB (0%)
@sentry/browser - ES6 CDN Bundle (minified) 52.83 KB (0%)
@sentry/browser - Webpack (gzipped + minified) 19.7 KB (0%)
@sentry/browser - Webpack (minified) 64.17 KB (0%)
@sentry/react - Webpack (gzipped + minified) 19.72 KB (0%)
@sentry/nextjs Client - Webpack (gzipped + minified) 44.14 KB (0%)
@sentry/browser + @sentry/tracing - ES5 CDN Bundle (gzipped + minified) 25.77 KB (0%)
@sentry/browser + @sentry/tracing - ES6 CDN Bundle (gzipped + minified) 24.02 KB (+0.01% 🔺)

@smeubank smeubank removed a link to an issue Aug 2, 2022
43 tasks
@Lms24 Lms24 mentioned this pull request Aug 2, 2022
14 tasks
lobsterkatie added a commit that referenced this pull request Aug 2, 2022
This is a number of boring structural changes pulled out of #5503 in order to make it easier to review.

Included changes:
- Create folders for loaders and templates, and move the existing loader and template into their respective new folders.
- Convert the `LoaderThis` type into a generic, so that each loader can specify its own options but share everything specified by webpack.
- Since we are only going to support wrapping for pages written in ESM, make sure that templates are ESM, even if the CJS version of the SDK is being used. This required creating a build script, which in turn required updating our ESLint config.
- Add/clarify comments in existing loader files.
@lobsterkatie lobsterkatie force-pushed the kmclb-nextjs-wrap-data-fetching-methods branch 3 times, most recently from 05cb4b5 to 130169d Compare August 3, 2022 04:44
@lobsterkatie lobsterkatie marked this pull request as ready for review August 3, 2022 04:59
@lobsterkatie lobsterkatie requested review from lforst and Lms24 August 3, 2022 04:59
Copy link
Member

@Lms24 Lms24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow that's a lot of groundwork for getting this to work. Thanks for figuring this all out!
I had a couple of questions but overall I think this looks good.

packages/nextjs/src/config/loaders/dataFetchersLoader.ts Outdated Show resolved Hide resolved
packages/nextjs/src/config/loaders/ast.ts Outdated Show resolved Hide resolved
packages/nextjs/src/utils/isESM.ts Show resolved Hide resolved
Copy link
Member

@lforst lforst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wanna put emphasis on how much I love what we're doing here. This is sooo so good DX wise. Thanks for tackling this admittedly very hard problem! 🌮

@lobsterkatie lobsterkatie force-pushed the kmclb-nextjs-wrap-data-fetching-methods branch 2 times, most recently from cd3e26b to 711fa2f Compare August 4, 2022 07:10
@lforst
Copy link
Member

lforst commented Aug 4, 2022

ABSOLUTE mind boggle: I tried to create a simple page in CJS and it just worked. I think you can just copy the following code into any next page and it should compile and run totally fine. The SDK doesn't seem to be working however. 🤔

const { default: Head } = require("next/head");
const { useState } = require("react");

const Sentry = require("@sentry/nextjs");

const Home = () => {
  const [shouldThrow, setShouldThrow] = useState(false);

  if (shouldThrow) {
    throw new Error("This is some render error!");
  }

  return (
    <div>
      <Head>
        <title>Create Next App</title>
        <meta name="description" content="Generated by create next app" />
        <link rel="icon" href="/favicon.ico" />
      </Head>
      <h1>Next.js Test Page</h1>
      <form>
        <input
          type="button"
          value="trigger onClick error"
          onClick={() => {
            try {
              throw new Error("This is some test onClick error!");
            } catch (err) {
              Sentry.captureException(err);
            }
          }}
        />
        <br />
        <input
          type="button"
          value="trigger render error"
          onClick={() => {
            setShouldThrow(true);
          }}
        />
      </form>
    </div>
  );
};

module.exports = {
  default: Home,
};

Copy link
Member

@lforst lforst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another pass. For the sake of timelyness, feel free to ignore.

packages/nextjs/src/config/loaders/parsers.ts Show resolved Hide resolved
packages/nextjs/src/config/loaders/ast.ts Outdated Show resolved Hide resolved
Copy link
Member

@AbhiPrasad AbhiPrasad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok - so this does have the chance to break folks pretty badly as we are mucking with ASTs.

What do you think about this?
a) We gate this functionality behind an SDK option
b) We update the Sentry onboarding wizard/docs to enable this option by default - so only new users of the SDK will encounter possible issues with this for the most part
c) After a week or 2 passes, we enable the option by default

Do we care about this risk that much? Maybe it's fine to just :shipit:

@lobsterkatie
Copy link
Member Author

lobsterkatie commented Aug 5, 2022

Ok - so this does have the chance to break folks pretty badly as we are mucking with ASTs.

What do you think about this? a) We gate this functionality behind an SDK option b) We update the Sentry onboarding wizard/docs to enable this option by default - so only new users of the SDK will encounter possible issues with this for the most part c) After a week or 2 passes, we enable the option by default

Do we care about this risk that much? Maybe it's fine to just :shipit:

Honestly, my instinct was the same (to make it opt-in at first), but Vladan expressed that he felt that was being too cautious and we didn't need to bother. Do you still feel that way @vladanpaunovic? The more I dig into the different cases I'm needing to cover, the more I'm leaning back towards thinking it should start as experimental.

Either way, I think we should make a flag, so that users can either opt in if they choose or opt out if we choose and end up breaking things. (On that topic @lforst made a good point, though: We should figure out if doing this could break people only at build time - not great, but at least they know right away - or if it could also break in such a way that they don't find out until we crash their app (or at least render certain pages non-functional) at runtime - way worse, and something we obviously want to avoid in any way we can.)

UPDATE: We discussed this offline and will put this behind a flag for now, so that it's opt-in.

@lobsterkatie lobsterkatie force-pushed the kmclb-nextjs-wrap-data-fetching-methods branch 2 times, most recently from dbd8903 to 57f6065 Compare August 8, 2022 05:12
@lobsterkatie lobsterkatie force-pushed the kmclb-nextjs-wrap-data-fetching-methods branch from 57f6065 to d9e4118 Compare August 8, 2022 05:34
@vladanpaunovic
Copy link
Contributor

Ok - so this does have the chance to break folks pretty badly as we are mucking with ASTs.
What do you think about this? a) We gate this functionality behind an SDK option b) We update the Sentry onboarding wizard/docs to enable this option by default - so only new users of the SDK will encounter possible issues with this for the most part c) After a week or 2 passes, we enable the option by default
Do we care about this risk that much? Maybe it's fine to just :shipit:

Honestly, my instinct was the same (to make it opt-in at first), but Vladan expressed that he felt that was being too cautious and we didn't need to bother. Do you still feel that way @vladanpaunovic? The more I dig into the different cases I'm needing to cover, the more I'm leaning back towards thinking it should start as experimental.

Either way, I think we should make a flag, so that users can either opt in if they choose or opt out if we choose and end up breaking things. (On that topic @lforst made a good point, though: We should figure out if doing this could break people only at build time - not great, but at least they know right away - or if it could also break in such a way that they don't find out until we crash their app (or at least render certain pages non-functional) at runtime - way worse, and something we obviously want to avoid in any way we can.)

UPDATE: We discussed this offline and will put this behind a flag for now, so that it's opt-in.

Agreed.

@lobsterkatie please, create an issue somewhere in our backlog to promote the experimental feature into a stable one once we've gained more confidence.

@lobsterkatie lobsterkatie merged commit c7550dd into master Aug 8, 2022
@lobsterkatie lobsterkatie deleted the kmclb-nextjs-wrap-data-fetching-methods branch August 8, 2022 13:24
@kachkaev
Copy link

kachkaev commented Aug 11, 2022

After upgrading @sentry/nextjs to 7.10.0, I noticed a significant diff in the yarn.lock file:

Screenshot 2022-08-11 at 12 33 32

It is mainly to do with the introduction of jscodeshift, which has bloated yarn.lock by ≈70-75 entries.

Some of jscodeshift's dependencies are quite dated. E.g. there’s micromatch@3.1.10, released in 2018. It is quite possible that some of the dated transient dependencies will introduce new vulnerabilities and although they are dev-only, this can be a concern for teams with strict security policies.

I am not suggesting to give up jscodeshift because I quite like the feature it enables. Just posting this comment to raise some attention to the problem it has introduced. It’d be great if Sentry team could help Meta release a new version of jscodeshift!

@lforst
Copy link
Member

lforst commented Aug 18, 2022

After upgrading @sentry/nextjs to 7.10.0, I noticed a significant diff in the yarn.lock file

@kachkaev Thanks for the heads up! It's actually something I didn't really consider when reviewing.

We're currently rethinking what we did in this PR and finding a different approach using rollup and @rollup/plugin-sucrase which should have way fewer dependencies and are better maintained. If you're interested you can follow the progress in #5602 and in #5505.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants