Feature: Add webpack plugin to externalize and extract script depende…

…ncies (#14869)

* Add package

* Add docs manifest

* Replace hard-coded externals with webpack plugin

* Extract default functions

* Prepare options

* Handle useDefaults for externals

* Handle custom requestToExternal function

* Handle custom dependency functions

* Save dev dependency in README

* Use prose package name as README title

* Use ^ version ranges

* Update return description

* Use boolean value directly

* Implement injectPolyfill

* Invert yoda conditional

Co-Authored-By: sirreal <jon.surrell@automattic.com>

* Restore regeneratorRuntime external

* Extern lodash-es

* Improve clarity

* Handle @babel/runtime/regenerator

* Copy camelCaseDash from scripts

See #14869 (comment)

* Add tests

* Add unit tests and restructure to facilitate

Move default transform functions to a until module.
Add unit tests for transform functions.

* Test external modules and dependencies file snapshot

* Align adjacent `=` in README

* Add options documentation to the README

* Test externals conflict

* Add a note about conflicts with externals

* Add test case for requires

* Add changelog for new package

* Add @wordpress/scripts changelog and bump version

* Remove describe wrapper and simplify test name

* Try: add package to root devDeps

* Update @wordpress/scripts README

* Add bundling documentation to scripts

* Add rimraf before tests

* Revert @wordpress/scripts version bump

* Use alpha version suffix

* Use "unreleased" versions in changelogs

Co-Authored-By: sirreal <jon.surrell@automattic.com>

* Use the plugin explicitly

* Use describe.each over explicit loop

* Use before/afterEach, add comment about cleanup

* Update README href

Co-Authored-By: sirreal <jon.surrell@automattic.com>

* Add externals info to scripts README

* Add default module request handling table to README

* Update package-lock

* Use this externalType and array object path

* Update tests to use array paths

* Add dynamic-import test

docs/contributors/scripts.md

@@ -65,3 +65,15 @@ It is recommended to use the main `wp-polyfill` script handle which takes care o
 | [Formdata Polyfill](https://www.npmjs.com/package/formdata-polyfill) | wp-polyfill-formdata| Polyfill conditionally replaces the native implementation |
 | [Node Contains Polyfill](https://polyfill.io) | wp-polyfill-node-contains |Polyfill for Node.contains |
 | [Element Closest Polyfill](https://www.npmjs.com/package/element-closest) | wp-polyfill-element-closest| Return the closest element matching a selector up the DOM tree |
+
+## Bundling and code sharing
+
+When using a JavaScript bundler like [webpack](https://webpack.js.org/), the scripts mentioned here
+can be excluded from the bundle and provided by WordPress in the form of script dependencies [(see
+`wp_enqueue_script`)][https://developer.wordpress.org/reference/functions/wp_enqueue_script/#default-scripts-included-and-registered-by-wordpress].
+
+The
+[`@wordpress/dependency-extraction-webpack-plugin`](https://github.com/WordPress/gutenberg/tree/master/packages/dependency-extraction-webpack-plugin)
+provides a webpack plugin to help extract WordPress dependencies from bundles. `@wordpress/scripts`
+[`build`](https://github.com/WordPress/gutenberg/tree/master/packages/scripts#build) script includes
+the plugin by default.

docs/manifest.json

@@ -629,6 +629,12 @@
 		"markdown_source": "https://raw.githubusercontent.com/WordPress/gutenberg/master/packages/date/README.md",
 		"parent": "packages"
 	},
+	{
+		"title": "@wordpress/dependency-extraction-webpack-plugin",
+		"slug": "packages-dependency-extraction-webpack-plugin",
+		"markdown_source": "https://raw.githubusercontent.com/WordPress/gutenberg/master/packages/dependency-extraction-webpack-plugin/README.md",
+		"parent": "packages"
+	},
 	{
 		"title": "@wordpress/deprecated",
 		"slug": "packages-deprecated",

package.json

@@ -66,6 +66,7 @@
 		"@wordpress/babel-preset-default": "file:packages/babel-preset-default",
 		"@wordpress/browserslist-config": "file:packages/browserslist-config",
 		"@wordpress/custom-templated-path-webpack-plugin": "file:packages/custom-templated-path-webpack-plugin",
+		"@wordpress/dependency-extraction-webpack-plugin": "file:packages/dependency-extraction-webpack-plugin",
 		"@wordpress/docgen": "file:packages/docgen",
 		"@wordpress/e2e-test-utils": "file:packages/e2e-test-utils",
 		"@wordpress/e2e-tests": "file:packages/e2e-tests",

packages/dependency-extraction-webpack-plugin/CHANGELOG.md

@@ -0,0 +1,5 @@
+## Unreleased
+
+### New Feature
+
+- Introduce the `@wordpress/dependency-extraction-webpack-plugin` package.

packages/dependency-extraction-webpack-plugin/README.md

@@ -0,0 +1,196 @@
+# Dependency Extraction Webpack Plugin
+
+This webpack plugin serves two purposes:
+
+- Externalize dependencies that are available as script dependencies on modern WordPress sites.
+- Add a JSON file for each entrypoint that declares the WordPress script dependencies for the
+  entrypoint.
+
+This allows JavaScript bundles produced by webpack to leverage WordPress style dependency sharing
+without an error-prone process of manually maintaining a dependency list.
+
+Consult the [webpack website](https://webpack.js.org) for additional information on webpack concepts.
+
+## Installation
+
+Install the module
+
+```bash
+npm install @wordpress/dependency-extraction-webpack-plugin --save-dev
+```
+
+## Usage
+
+### Webpack
+
+Use this plugin as you would other webpack plugins:
+
+```js
+// webpack.config.js
+const WordPressExternalDependenciesPlugin = require( '@wordpress/dependency-extraction-webpack-plugin' );
+
+module.exports = {
+  // …snip
+  plugins: [
+    new WordPressExternalDependenciesPlugin(),
+  ]
+}
+```
+
+Each entrypoint in the webpack bundle will include JSON file that declares the WordPress script dependencies that should be enqueued.
+
+For example:
+
+```
+// Source file entrypoint.js
+import { Component } from '@wordpress/element';
+
+// Webpack will produce the output output/entrypoint.js
+/* bundled JavaScript output */
+
+// Webpack will also produce output/entrypoint.deps.json declaring script dependencies
+['wp-element']
+```
+
+By default, the following module requests are handled:
+
+| Request | Global | Script handle |
+| --- | --- | --- |
+| `@babel/runtime/regenerator` | `regeneratorRuntime` | `wp-polyfill` |
+| `@wordpress/*` | `wp['*']` | `wp-*` |
+| `jquery` | `jQuery` | `jquery` |
+| `jquery` | `jQuery` | `jquery` |
+| `lodash-es` | `lodash` | `lodash` |
+| `lodash` | `lodash` | `lodash` |
+| `moment` | `moment` | `moment` |
+| `react-dom` | `ReactDOM` | `react-dom` |
+| `react` | `React` | `react` |
+
+**Note:** This plugin overlaps with the functionality provided by [webpack
+`externals`](https://webpack.js.org/configuration/externals). This plugin is intended to extract
+script handles from bundle compilation so that a list of script dependencies does not need to be
+manually maintained. If you don't need to extract a list of script dependencies, use the `externals`
+option directly.
+
+This plugin is compatible with `externals`, but they may conflict. For example, adding
+`{ externals: { '@wordpress/blob': 'wp.blob' } }` to webpack configuration will effectively hide the
+`@wordpress/blob` module from the plugin and it will not be included in dependency lists.
+
+#### Options
+
+An object can be passed to the constructor to customize the behavior, for example:
+
+```js
+module.exports = {
+  plugins: [
+    new WordPressExternalDependenciesPlugin( { injectPolyfill: true } ),
+  ]
+}
+```
+
+##### `useDefaults`
+
+- Type: boolean
+- Default: `true`
+
+Pass `useDefaults: false` to disable the default request handling.
+
+##### `injectPolyfill`
+
+- Type: boolean
+- Default: `false`
+
+Force `wp-polyfill` to be included in each entrypoint's dependency list. This would be the same as
+adding `import '@wordpress/polyfill';` to each entrypoint.
+
+##### `requestToExternal`
+
+- Type: function
+
+`requestToExternal` allows the module handling to be customized. The function should accept a
+module request string and may return a string representing the global variable to use. An array of
+strings may be used to access globals via an object path, e.g. `wp.i18n` may be represented as `[
+'wp', 'i18n' ]`.
+
+`requestToExternal` provided via configuration has precedence over default external handling.
+Unhandled requests will be handled by the default unless `useDefaults` is set to `false`.
+
+```js
+/**
+ * Externalize 'my-module'
+ *
+ * @param {string} request Requested module
+ *
+ * @return {(string|undefined)} Script global
+ */
+function requestToExternal( request ) {
+
+  // Handle imports like `import myModule from 'my-module'`
+  if ( request === 'my-module' ) {
+    // Expect to find `my-module` as myModule in the global scope:
+    return 'myModule';
+  }
+}
+
+module.exports = {
+  plugins: [
+    new WordPressExternalDependenciesPlugin( { requestToExternal } ),
+  ]
+}
+```
+
+##### `requestToHandle`
+
+- Type: function
+
+All of the external modules handled by the plugin are expected to be WordPress script dependencies
+and will be added to the dependency list. `requestToHandle` allows the script handle included in the dependency list to be customized.
+
+If no string is returned, the script handle is assumed to be the same as the request.
+
+`requestToHandle` provided via configuration has precedence over the defaults. Unhandled requests will be handled by the default unless `useDefaults` is set to `false`.
+
+```js
+/**
+ * Map 'my-module' request to 'my-module-script-handle'
+ *
+ * @param {string} request Requested module
+ *
+ * @return {(string|undefined)} Script global
+ */
+function requestToHandle( request ) {
+
+  // Handle imports like `import myModule from 'my-module'`
+  if ( request === 'my-module' ) {
+    // Expect to find `my-module` as myModule in the global scope:
+    return 'my-module-script-handle';
+  }
+}
+
+module.exports = {
+  plugins: [
+    new WordPressExternalDependenciesPlugin( { requestToExternal } ),
+  ]
+}
+```
+
+##### `requestToExternal` and `requestToHandle`
+
+The functions `requestToExternal` and `requestToHandle` allow this module to handle arbitrary
+modules. `requestToExternal` is necessary to handle any module and maps a module request to a global
+name. `requestToHandle` maps the same module request to a script handle, the strings that will be
+included in the `entrypoint.deps.json` files.
+
+### WordPress
+
+Enqueue your script as usual and read the script dependencies dynamically:
+
+```php
+$script_path         = 'path/to/script.js';
+$script_deps_path    = 'path/to/script.deps.json';
+$script_dependencies = file_exists( $script_deps_path )
+	? json_decode( file_get_contents( $script_deps_path ) )
+	: array();
+$script_url = plugins_url( $script_path, __FILE__ );
+wp_enqueue_script( 'script', $script_url, $script_dependencies );
+```