-
Notifications
You must be signed in to change notification settings - Fork 10.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(conditional-page-builds): track potentially unsafe Node.js builtin modules usage #29560
Conversation
@@ -0,0 +1,4 @@ | |||
/* eslint-disable filenames/match-regex */ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is because filename (child_process.js
doesn't match our eslint rule :( )
preBodyComponents = preBodyComponents.concat(sanitizeComponents(components)) | ||
} | ||
// for this to work we need this function to be sync or at least ensure there is single execution of it at a time | ||
global.unsafeBuiltinUsage = [] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This hurts my feelings, but I essentially need "context" that "unconnected" arbitrary code can access and push to.
5e71bcc
to
dedecfd
Compare
dedecfd
to
e70af97
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I need to try it, some small nits and questions.
return { html, unsafeBuiltinsUsage: global.unsafeBuiltinUsage } | ||
} catch (e) { | ||
e.unsafeBuiltinsUsage = global.unsafeBuiltinUsage | ||
throw e |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
did anything change except this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suggest checking with "hide whitespace changes" ( https://github.com/gatsbyjs/gatsby/pull/29560/files?w=1#diff-4cd5b620539129dfa79254b9cdae3bd9b619426b83b412d2647cfb416730d362 )
This whole file added
// for this to work we need this function to be sync or at least ensure there is single execution of it at a time
global.unsafeBuiltinUsage = []
at the start of the function, changed return to return object instead of string and wrapped everything in try/catch to decorate caught error with unsafeBuiltinUsage
to not "lose" it's usage (even tho probably this is might be not needed, it feels safer to do)
store.dispatch({ | ||
type: `HTML_GENERATED`, | ||
payload: pageSegment, | ||
type: `SSR_USED_UNSAFE_BUILTIN`, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How does someone clear this flag? Running gatsby clean
and/or gatsby cleans cache?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this flag gets cleared when SSR compilation hash changes (as long as we use same SSR compilation hash is used - we know that if previous build used unsafe builtin methods - following builds will continue to do so, because it's the same code running).
Changing SSR compilation hash also cause all pages to be rebuild so we are "covered" for that case
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
L
Co-authored-by: Ward Peeters <ward@coding-tech.com>
Co-authored-by: Ward Peeters <ward@coding-tech.com>
Co-authored-by: Ward Peeters <ward@coding-tech.com>
235419d
to
197cb62
Compare
Note: best to check changes with whitespace changes hidden as I did some try/catch wrapping which just changed indentation a lot
Description
Usage of certain Node.js builtin modules in SSR (primarly from gatsby-ssr, but not limited to it) risks that our tracking wether html file can be reused from previous build is not catching all the cases that ideally should case file to rebuild.
Current list of builtin modules that are "tracked":
In those cases those modules are used we set a flag so that following builds would rebuild all the pages as we can't be certain of the exact "contract" usage of those APIs have. We also do display warnings about those (including code location) and that future builds will be full ones and not selective:
Related Issues
[ch24532]