Feature: Add webpack plugin to externalize and extract script dependencies

[sirreal]

Description

Add a webpack plugin that externalizes WordPress script dependencies and adds a dependency manifest per entrypoint. The goal is to eliminate error-prone process of manually creating and maintaining script dependencies.

A JSON file is output alongside each entrypoint. That looks like (webpack output):

             ./build/annotations/index.js    9.17 KiB      39  [emitted]         annotations
                    ./build/a11y/index.js    1.78 KiB      40  [emitted]         a11y
             ./build/a11y/index.deps.json    16 bytes      40  [emitted]         a11y
      ./build/annotations/index.deps.json    56 bytes      39  [emitted]         annotations

More detail in the README.

Closes Part of #14837

To do

• Add tests
• Update README. Document options with examples.
• CHANGELOG - this package and scripts
• Fix CI test failures
• Document in handbook: # #

How has this been tested?

The plugin is put to use in the Gutenberg webpack config. It can be tested using npm run build.

Plugin output results

a11y

["wp-dom-ready"]

annotations

["lodash","wp-data","wp-hooks","wp-i18n","wp-rich-text"]

api-fetch

["wp-i18n","wp-url"]

autop

[]

blob

[]

block-editor

["lodash","react","wp-a11y","wp-blob","wp-blocks","wp-components","wp-compose","wp-core-data","wp-data","wp-deprecated","wp-dom","wp-element","wp-hooks","wp-html-entities","wp-i18n","wp-is-shallow-equal","wp-keycodes","wp-rich-text","wp-token-list","wp-url","wp-viewport","wp-wordcount"]

block-library

["lodash","moment","wp-a11y","wp-api-fetch","wp-autop","wp-blob","wp-block-editor","wp-blocks","wp-components","wp-compose","wp-core-data","wp-data","wp-date","wp-deprecated","wp-editor","wp-element","wp-i18n","wp-is-shallow-equal","wp-keycodes","wp-rich-text","wp-url","wp-viewport"]

block-serialization-default-parser

[]

block-serialization-spec-parser

[]

blocks

["lodash","wp-autop","wp-blob","wp-block-serialization-default-parser","wp-compose","wp-data","wp-dom","wp-element","wp-hooks","wp-html-entities","wp-i18n","wp-is-shallow-equal","wp-shortcode"]

components

["lodash","moment","react","react-dom","wp-a11y","wp-api-fetch","wp-compose","wp-dom","wp-element","wp-hooks","wp-i18n","wp-is-shallow-equal","wp-keycodes","wp-rich-text","wp-url"]

compose

["lodash","wp-element","wp-is-shallow-equal"]

core-data

["lodash","wp-api-fetch","wp-data","wp-deprecated","wp-url"]

data

["lodash","wp-compose","wp-deprecated","wp-element","wp-is-shallow-equal","wp-priority-queue","wp-redux-routine"]

date

["moment"]

deprecated

["wp-hooks"]

dom

["lodash"]

dom-ready

[]

edit-post

["lodash","wp-a11y","wp-api-fetch","wp-block-editor","wp-block-library","wp-blocks","wp-components","wp-compose","wp-core-data","wp-data","wp-editor","wp-element","wp-hooks","wp-i18n","wp-keycodes","wp-notices","wp-nux","wp-plugins","wp-url","wp-viewport"]

edit-widgets

["wp-element"]

editor

["lodash","react","wp-api-fetch","wp-autop","wp-blob","wp-block-editor","wp-blocks","wp-components","wp-compose","wp-core-data","wp-data","wp-date","wp-deprecated","wp-element","wp-hooks","wp-html-entities","wp-i18n","wp-keycodes","wp-notices","wp-nux","wp-rich-text","wp-url","wp-viewport","wp-wordcount"]

element

["lodash","react","react-dom","wp-escape-html"]

escape-html

[]

format-library

["lodash","wp-block-editor","wp-components","wp-element","wp-i18n","wp-keycodes","wp-rich-text","wp-url"]

hooks

[]

html-entities

[]

i18n

[]

is-shallow-equal

[]

keycodes

["lodash","wp-i18n"]

list-reusable-blocks

["lodash","wp-api-fetch","wp-components","wp-compose","wp-element","wp-i18n"]

notices

["lodash","wp-a11y","wp-data"]

nux

["lodash","wp-components","wp-compose","wp-data","wp-element","wp-i18n"]

plugins

["lodash","wp-compose","wp-element","wp-hooks"]

priority-queue

[]

redux-routine

["lodash"]

rich-text

["lodash","wp-compose","wp-data","wp-element","wp-escape-html","wp-hooks"]

shortcode

["lodash"]

token-list

["lodash"]

url

[]

viewport

["lodash","wp-compose","wp-data"]

wordcount

["lodash"]

Types of changes

New feature:
Adds a new package @wordpress/dependency-extraction-webpack-plugin.

New feature:
Uses the plugin in @wordpress/scripts (will require version bump, not included initially). This can be part of another PR but is useful for testing.

Checklist:

• My code is tested.
• My code follows the WordPress code style.
• My code follows the accessibility standards.
• My code has proper inline documentation.
• I've included developer documentation if appropriate.

[nerrad]

Nice!

I wonder, instead of hardcoding the externals and the handle ref to use, could we make that configurable via an options array? This would allow WordPress plugins to use this for their own externals as well.

So the options config would be an array with two callbacks. The first callback would be used for mapping externals, the second callback would be for mapping to the json output. It could have the shape:

const options = [
    ( request ) => { /* eternalized dependencies callback for mapping external to */ },
    ( request ) => { /* dependencies map callback */ }
];

This way it's somewhat extensible by WP plugins that might want to use this package for non WP externals.

This is just an idea, the actual shape of the configuration option for callbacks could likely be iterated on a bit (eg - should callbacks always be provided and the application merged? Should the wp externals be hardcoded/permanent or can they be overridden?)

[sirreal]

I wonder, instead of hardcoding the externals and the handle ref to use, could we make that configurable via an options array?

Extensibility and configuration is missing at this time. I had in mind a few options that might be helpful.

Configuration for plugin extensibility allowing plugins provide script dependencies and build bundles that externalize them.

• requestToExternal: A function that receives module requests (@wordpress/element) and returns a string to externalize the module (wp.element):

  const options = {
    requestToExternal: request => request === '@wordpress/element' ? 'wp.element' : undefined,
  }

• requestToDependency: A function that receives module requests (@wordpress/element) and returns the WordPress script dependency to include in the manifest list (wp-element):

  const options = {
    requestToDependency: request => request === '@wordpress/element' ? 'wp-element' : undefined,
  }

I can imagine cases where options like these might be helpful as well:

• useDefaults - include the built-in externals like jquery, lodash, and @wordpres/*. This option would allow this behavior to be enabled/disabled (true/false).
• injectPolyfill - An option to include wp-polyfill as a dependency for all scripts.

[aduth]

Related issue: #14837

Am I correct in thinking this should close the issue once merged?

[sirreal]

Related issue: #14837

Am I correct in thinking this should close the issue once merged?

Yes, thanks. I've updated the description to reflect that.

[gziolo]

Am I correct in thinking this should close the issue once merged?

I don't see the following file removed in this PR:
https://github.com/WordPress/gutenberg/blob/master/lib/packages-dependencies.php

If that can be easily squeezed in then sure, we can consider it as closing the issue :)

[aduth]

Am I correct in thinking this should close the issue once merged?

I don't see the following file removed in this PR:
https://github.com/WordPress/gutenberg/blob/master/lib/packages-dependencies.php

If that can be easily squeezed in then sure, we can consider it as closing the issue :)

I had wondered that as well, whether this would allow for this file to be removed.

[sirreal]

I've rebased, addressed some feedback, and implemented the configuration outlined in #14869 (comment)

I haven't had an opportunity to test the new functionality yet.

[aduth]

For those following along, this pull request was discussed during today's JavaScript Chat (link requires registration):

https://wordpress.slack.com/archives/C5UNMSU4R/p1554818548169400