Use webpack DLL plugin for performance improvement

[thangngoc89]

I read a new article on using DLL plugin to speed up Webpack initial build speed.

You can read it here : https://medium.com/@soederpop/webpack-plugins-been-we-been-keepin-on-the-dll-cdfdd6cb8cd7#.9fzlbqz0m

It's quite simple to implement (I think). But the problem is how can we detect if the user changed the dependencies to remove the old cache and build a new one ?

[MoOx]

how can we detect if the user changed the dependencies to remove the old cache and build a new one ?

Just run the build of the dll in npm postinstall hook. Should be good right?

---

Here are some ressources:

• https://webpack.github.io/docs/list-of-plugins.html#dllplugin
• https://gist.github.com/robertknight/058a194f45e77ff95fcd
• https://github.com/mxstbr/react-boilerplate/pull/495/files
• https://github.com/FrendEr/webpack-optimize-example/tree/master/dll-bundles

[DavidWells]

I set this up for a project and it cut the build time in half.

Where would this be implemented in the project? In the boilerplate?

[thangngoc89]

Could you inject it in the core? We want to keep the changes in boilerplate as small as possible.

See config.*.js in this folder https://github.com/MoOx/phenomic/tree/master/src/builder/webpack

[MoOx]

@DavidWells can you show us a diff as an example so I can investigate this a little?

[DavidWells]

@MoOx this is how I hooked it up: https://gist.github.com/DavidWells/e886bff1fdb543a707b39451535b1369

[DavidWells]

Webpack.config.dll.js https://gist.github.com/DavidWells/e886bff1fdb543a707b39451535b1369#file-webpack-dll-config-js gets run from the npm postinstall hook https://gist.github.com/DavidWells/e886bff1fdb543a707b39451535b1369#file-package-json-L2, which only runs in the context of local dev environment and not when the folder is nested inside the node_modules folder as a dependency.

Then in the index.html file you include the vendorPackages.dll.js which has all the non frequently changing files https://gist.github.com/DavidWells/e886bff1fdb543a707b39451535b1369#file-index-html-L7

And the last piece is providing the manifest to your development config: https://gist.github.com/DavidWells/e886bff1fdb543a707b39451535b1369#file-webpack-dev-config-js-L31-L33

[MoOx]

I implemented this, but the bad part is that enabling this seems to disable hard source plugin (which is far better than dll). I opened an issue so I hope we could benefit from hard source AND dllplugin at the same time. mzgoddard/hard-source-webpack-plugin#18

Meanwhile I will ship the hard source option (eg: 20s to 4s for the docs).

[MoOx]

Compared to the cache we have (easy to setup, just a flag) dll will be a pain so closing since I don't plan to spend more time on this and nobody really asked for this.

[DavidWells]

Looks like @mzgoddard is working on this! mzgoddard/hard-source-webpack-plugin#18 🎉

I have a DLL setup ready to go once hard-source gets updated.

It should speed up hot-module reloading quite a bit =)

[MoOx]

The problem I see is: how we decide what dep to put in the dll bundle?

[DavidWells]

all dev deps https://github.com/serverless/site/blob/master/webpack.DLL.babel.js#L14

Then you need to generate the DLL on the postinstall npm hook to update the DLL everytime npm install whatever is run https://github.com/serverless/site/blob/master/package.json#L31

[DavidWells]

As long as the hard-source plugin gets updated and allows for DLL. People can do this in userland. I'm cool with that =)

[MoOx]

I am afraid that dev deps can be quickly too much. Importing babel, webpack etc seems like overkill. I will update the package but I am not sure we can by default use dll plugin.

[DavidWells]

just adding react and react-dom would speed up the webpack compilation time for faster npm starts

[bloodyowl]

Going to be fixed with #925 (moving to user-land)

[MoOx]

Especially make sense as webpack won't be a core piece.

[loopmode]

@DavidWells About using a DLL plugin for "all dev deps" as you described..
So we explicitly put all the direct dependencies into the bundle (Object.keys(pkg.dependencies)).
What happens with dependencies of the dependencies? For example, looking into my node_modules folder, there are dozens of lodash variations - none of which I have in my dependencies in package.json.

Should they still be included because they are part of the dependency tree?
Or do I need to explicitly add them to my vendors dll config? (Which would be kinda tedious micro-managment)

Edit: A naive test I just did suggests one should add indirect dependencies by hand.
I just tried with and without, and when manually adding all the lodash variants to the config, the dll bundle is roughly 2mb bigger, so I guess it's worth it.

[MoOx]

@loopmode from my experience (& if my memory is correct), it will automatically handle deps of deps.

[loopmode]

@MoOx I haven't done benchmarks of any kind yet, but as I said, it did make quite a difference in bundle size when I manually included them. But also as I said, it was quite a naive quick shot. I do not know whether, for example, most of these lodash dependencies I was seeing were coming from devDependencies like webpack loaders etc, or actual frontend runtime dependencies.

In case I find out something useful, I'll post it here.