-
Notifications
You must be signed in to change notification settings - Fork 27.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Populate sourcemap ignoreList
when Webpack is used
#71821
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
93 changes: 93 additions & 0 deletions
93
packages/next/src/build/webpack/plugins/devtools-ignore-list-plugin.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,93 @@ | ||
// Source: https://github.com/mondaychen/devtools-ignore-webpack-plugin/blob/e35ce41d9606a92a455ef247f509a1c2ccab5778/src/index.ts | ||
|
||
import { webpack } from 'next/dist/compiled/webpack/webpack' | ||
|
||
// Following the naming conventions from | ||
// https://tc39.es/source-map/#source-map-format | ||
const IGNORE_LIST = 'ignoreList' | ||
|
||
const PLUGIN_NAME = 'devtools-ignore-plugin' | ||
|
||
interface SourceMap { | ||
sources: string[] | ||
[IGNORE_LIST]: number[] | ||
} | ||
|
||
interface PluginOptions { | ||
shouldIgnorePath?: (path: string) => boolean | ||
isSourceMapAsset?: (name: string) => boolean | ||
} | ||
|
||
interface ValidatedOptions extends PluginOptions { | ||
shouldIgnorePath: Required<PluginOptions>['shouldIgnorePath'] | ||
isSourceMapAsset: Required<PluginOptions>['isSourceMapAsset'] | ||
} | ||
|
||
function defaultShouldIgnorePath(path: string): boolean { | ||
return path.includes('/node_modules/') || path.includes('/webpack/') | ||
} | ||
|
||
function defaultIsSourceMapAsset(name: string): boolean { | ||
return name.endsWith('.map') | ||
} | ||
|
||
/** | ||
* This plugin adds a field to source maps that identifies which sources are | ||
* vendored or runtime-injected (aka third-party) sources. These are consumed by | ||
* Chrome DevTools to automatically ignore-list sources. | ||
*/ | ||
export default class DevToolsIgnorePlugin { | ||
options: ValidatedOptions | ||
|
||
constructor(options: PluginOptions = {}) { | ||
this.options = { | ||
shouldIgnorePath: options.shouldIgnorePath ?? defaultShouldIgnorePath, | ||
isSourceMapAsset: options.isSourceMapAsset ?? defaultIsSourceMapAsset, | ||
} | ||
} | ||
|
||
apply(compiler: webpack.Compiler) { | ||
const { RawSource } = compiler.webpack.sources | ||
|
||
compiler.hooks.compilation.tap(PLUGIN_NAME, (compilation) => { | ||
compilation.hooks.processAssets.tap( | ||
{ | ||
name: PLUGIN_NAME, | ||
stage: webpack.Compilation.PROCESS_ASSETS_STAGE_DEV_TOOLING, | ||
additionalAssets: true, | ||
}, | ||
(assets) => { | ||
for (const [name, asset] of Object.entries(assets)) { | ||
// Instead of using `asset.map()` to fetch the source maps from | ||
// SourceMapSource assets, process them directly as a RawSource. | ||
// This is because `.map()` is slow and can take several seconds. | ||
if (!this.options.isSourceMapAsset(name)) { | ||
// Ignore non source map files. | ||
continue | ||
} | ||
|
||
const mapContent = asset.source().toString() | ||
if (!mapContent) { | ||
continue | ||
} | ||
|
||
const sourcemap = JSON.parse(mapContent) as SourceMap | ||
|
||
const ignoreList = [] | ||
for (const [index, path] of sourcemap.sources.entries()) { | ||
if (this.options.shouldIgnorePath(path)) { | ||
ignoreList.push(index) | ||
} | ||
} | ||
|
||
sourcemap[IGNORE_LIST] = ignoreList | ||
compilation.updateAsset( | ||
name, | ||
new RawSource(JSON.stringify(sourcemap)) | ||
) | ||
} | ||
} | ||
) | ||
}) | ||
} | ||
} |
249 changes: 249 additions & 0 deletions
249
packages/next/src/build/webpack/plugins/eval-source-map-dev-tool-plugin.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,249 @@ | ||
/* | ||
huozhi marked this conversation as resolved.
Show resolved
Hide resolved
|
||
MIT License http://www.opensource.org/licenses/mit-license.php | ||
Author Tobias Koppers @sokra | ||
*/ | ||
import { | ||
type webpack, | ||
type SourceMapDevToolPluginOptions, | ||
ConcatenatedModule, | ||
makePathsAbsolute, | ||
ModuleFilenameHelpers, | ||
NormalModule, | ||
RuntimeGlobals, | ||
SourceMapDevToolModuleOptionsPlugin, | ||
} from 'next/dist/compiled/webpack/webpack' | ||
import type { RawSourceMap } from 'next/dist/compiled/source-map' | ||
|
||
const cache = new WeakMap<webpack.sources.Source, webpack.sources.Source>() | ||
|
||
const devtoolWarningMessage = `/* | ||
* ATTENTION: An "eval-source-map" devtool has been used. | ||
* This devtool is neither made for production nor for readable output files. | ||
* It uses "eval()" calls to create a separate source file with attached SourceMaps in the browser devtools. | ||
* If you are trying to read the output file, select a different devtool (https://webpack.js.org/configuration/devtool/) | ||
* or disable the default devtool with "devtool: false". | ||
* If you are looking for production-ready output files, see mode: "production" (https://webpack.js.org/configuration/mode/). | ||
*/ | ||
` | ||
|
||
// @ts-expect-error -- can't compare `string` with `number` in `version`Ï | ||
interface SourceMap extends RawSourceMap { | ||
ignoreList?: number[] | ||
version: number | ||
} | ||
|
||
export interface EvalSourceMapDevToolPluginOptions | ||
extends SourceMapDevToolPluginOptions { | ||
// Fork | ||
shouldIgnorePath?: (modulePath: string) => boolean | ||
} | ||
|
||
// Fork of webpack's EvalSourceMapDevToolPlugin with support for adding `ignoreList`. | ||
// https://github.com/webpack/webpack/blob/e237b580e2bda705c5ab39973f786f7c5a7026bc/lib/EvalSourceMapDevToolPlugin.js#L37 | ||
export default class EvalSourceMapDevToolPlugin { | ||
sourceMapComment: string | ||
moduleFilenameTemplate: NonNullable< | ||
EvalSourceMapDevToolPluginOptions['moduleFilenameTemplate'] | ||
> | ||
namespace: NonNullable<EvalSourceMapDevToolPluginOptions['namespace']> | ||
options: EvalSourceMapDevToolPluginOptions | ||
shouldIgnorePath: (modulePath: string) => boolean | ||
|
||
/** | ||
* @param {SourceMapDevToolPluginOptions|string} inputOptions Options object | ||
*/ | ||
constructor(inputOptions: EvalSourceMapDevToolPluginOptions) { | ||
let options: EvalSourceMapDevToolPluginOptions | ||
if (typeof inputOptions === 'string') { | ||
options = { | ||
append: inputOptions, | ||
} | ||
} else { | ||
options = inputOptions | ||
} | ||
this.sourceMapComment = | ||
options.append && typeof options.append !== 'function' | ||
? options.append | ||
: '//# sourceURL=[module]\n//# sourceMappingURL=[url]' | ||
this.moduleFilenameTemplate = | ||
options.moduleFilenameTemplate || | ||
'webpack://[namespace]/[resource-path]?[hash]' | ||
this.namespace = options.namespace || '' | ||
this.options = options | ||
|
||
// fork | ||
this.shouldIgnorePath = options.shouldIgnorePath ?? (() => false) | ||
} | ||
|
||
/** | ||
* Apply the plugin | ||
* @param compiler the compiler instance | ||
*/ | ||
apply(compiler: webpack.Compiler): void { | ||
const options = this.options | ||
compiler.hooks.compilation.tap( | ||
'NextJSEvalSourceMapDevToolPlugin', | ||
(compilation) => { | ||
const { JavascriptModulesPlugin } = compiler.webpack.javascript | ||
const { RawSource, ConcatSource } = compiler.webpack.sources | ||
|
||
const devtoolWarning = new RawSource(devtoolWarningMessage) | ||
|
||
const hooks = JavascriptModulesPlugin.getCompilationHooks(compilation) | ||
|
||
new SourceMapDevToolModuleOptionsPlugin(options).apply(compilation) | ||
const matchModule = ModuleFilenameHelpers.matchObject.bind( | ||
ModuleFilenameHelpers, | ||
options | ||
) | ||
|
||
hooks.renderModuleContent.tap( | ||
'NextJSEvalSourceMapDevToolPlugin', | ||
(source, m, { chunk, runtimeTemplate, chunkGraph }) => { | ||
const cachedSource = cache.get(source) | ||
if (cachedSource !== undefined) { | ||
return cachedSource | ||
} | ||
|
||
const result = ( | ||
r: webpack.sources.Source | ||
): webpack.sources.Source => { | ||
cache.set(source, r) | ||
return r | ||
} | ||
|
||
if (m instanceof NormalModule) { | ||
const module = m | ||
if (!matchModule(module.resource)) { | ||
return result(source) | ||
} | ||
} else if (m instanceof ConcatenatedModule) { | ||
const concatModule = m | ||
if (concatModule.rootModule instanceof NormalModule) { | ||
const module = concatModule.rootModule | ||
if (!matchModule(module.resource)) { | ||
return result(source) | ||
} | ||
} else { | ||
return result(source) | ||
} | ||
} else { | ||
return result(source) | ||
} | ||
|
||
const namespace = compilation.getPath(this.namespace, { | ||
chunk, | ||
}) | ||
let sourceMap: SourceMap | ||
let content | ||
if (source.sourceAndMap) { | ||
const sourceAndMap = source.sourceAndMap(options) | ||
sourceMap = sourceAndMap.map as SourceMap | ||
content = sourceAndMap.source | ||
} else { | ||
sourceMap = source.map(options) as SourceMap | ||
content = source.source() | ||
} | ||
if (!sourceMap) { | ||
return result(source) | ||
} | ||
|
||
// Clone (flat) the sourcemap to ensure that the mutations below do not persist. | ||
sourceMap = { ...sourceMap } | ||
const context = compiler.options.context! | ||
const root = compiler.root | ||
const modules = sourceMap.sources.map((sourceMapSource) => { | ||
if (!sourceMapSource.startsWith('webpack://')) | ||
return sourceMapSource | ||
sourceMapSource = makePathsAbsolute( | ||
context, | ||
sourceMapSource.slice(10), | ||
root | ||
) | ||
const module = compilation.findModule(sourceMapSource) | ||
return module || sourceMapSource | ||
}) | ||
let moduleFilenames = modules.map((module) => | ||
ModuleFilenameHelpers.createFilename( | ||
module, | ||
{ | ||
moduleFilenameTemplate: this.moduleFilenameTemplate, | ||
namespace, | ||
}, | ||
{ | ||
requestShortener: runtimeTemplate.requestShortener, | ||
chunkGraph, | ||
// @ts-expect-error -- Original code | ||
hashFunction: compilation.outputOptions.hashFunction, | ||
} | ||
) | ||
) | ||
moduleFilenames = ModuleFilenameHelpers.replaceDuplicates( | ||
moduleFilenames, | ||
(filename, _i, n) => { | ||
for (let j = 0; j < n; j++) filename += '*' | ||
return filename | ||
} | ||
) | ||
sourceMap.sources = moduleFilenames | ||
sourceMap.ignoreList = [] | ||
for (let index = 0; index < moduleFilenames.length; index++) { | ||
if (this.shouldIgnorePath(moduleFilenames[index])) { | ||
sourceMap.ignoreList.push(index) | ||
} | ||
} | ||
if (options.noSources) { | ||
sourceMap.sourcesContent = undefined | ||
} | ||
sourceMap.sourceRoot = options.sourceRoot || '' | ||
const moduleId = | ||
/** @type {ModuleId} */ | ||
chunkGraph.getModuleId(m) | ||
sourceMap.file = | ||
typeof moduleId === 'number' ? `${moduleId}.js` : moduleId | ||
|
||
const footer = `${this.sourceMapComment.replace( | ||
/\[url\]/g, | ||
`data:application/json;charset=utf-8;base64,${Buffer.from( | ||
JSON.stringify(sourceMap), | ||
'utf8' | ||
).toString('base64')}` | ||
)}\n//# sourceURL=webpack-internal:///${moduleId}\n` // workaround for chrome bug | ||
|
||
return result( | ||
new RawSource( | ||
`eval(${ | ||
compilation.outputOptions.trustedTypes | ||
? `${RuntimeGlobals.createScript}(${JSON.stringify( | ||
content + footer | ||
)})` | ||
: JSON.stringify(content + footer) | ||
});` | ||
) | ||
) | ||
} | ||
) | ||
hooks.inlineInRuntimeBailout.tap( | ||
'EvalDevToolModulePlugin', | ||
() => 'the eval-source-map devtool is used.' | ||
) | ||
hooks.render.tap( | ||
'EvalSourceMapDevToolPlugin', | ||
(source) => new ConcatSource(devtoolWarning, source) | ||
) | ||
hooks.chunkHash.tap('EvalSourceMapDevToolPlugin', (_chunk, hash) => { | ||
hash.update('EvalSourceMapDevToolPlugin') | ||
hash.update('2') | ||
}) | ||
if (compilation.outputOptions.trustedTypes) { | ||
compilation.hooks.additionalModuleRuntimeRequirements.tap( | ||
'EvalSourceMapDevToolPlugin', | ||
(_module, set, _context) => { | ||
set.add(RuntimeGlobals.createScript) | ||
} | ||
) | ||
} | ||
} | ||
) | ||
} | ||
} |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we also check if it's starting with
next/dist
or there's a slash at the beginning? like the regext\/?next\/dist
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What would that do?
This is also an outdated version. We really just need this when
next
is symlinked i.e. local development.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
techinically it wil match sth
foo-next/dist
, but i realize it would also matchmy-lib/next/dist
. If we can check if it's including the path of nextjs package would be more ideal but not critical for now