Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds brotli support for modern javascript #674

Merged
merged 36 commits into from
Apr 5, 2019
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
0507885
WIP adding sw as a webpack dependency
prateekbh Nov 20, 2018
c5d5246
Merge branch 'babel-bug-fix' into sw-webpack
prateekbh Nov 20, 2018
5a37b9f
adding sw-config
prateekbh Nov 20, 2018
65a507c
Brotli support for modern javascript
prateekbh Nov 27, 2018
02498fe
adding brotli plugin to webpack
prateekbh Nov 27, 2018
375c545
fixing conditional plugins
prateekbh Nov 27, 2018
9c34521
lint fixes
prateekbh Nov 28, 2018
e9e1cdc
fixing error
prateekbh Nov 29, 2018
8b84b4d
bug-fix
prateekbh Nov 29, 2018
cc29725
WIP fix watch mode
prateekbh Dec 5, 2018
1500698
Merge branch 'next' of https://github.com/developit/preact-cli into s…
prateekbh Dec 5, 2018
86f0f78
fix watch mode
prateekbh Dec 5, 2018
3d1ac3a
add the capability to build sw from user land
prateekbh Dec 5, 2018
1369f2c
fixing comments
prateekbh Dec 10, 2018
3135b3b
fixing spacing
prateekbh Dec 10, 2018
c1fd332
fixing in place mutation
prateekbh Dec 10, 2018
8aec475
no multi compilers
prateekbh Dec 19, 2018
8bc9521
Update run-webpack.js
prateekbh Dec 19, 2018
dcc8f41
removing unwanted files
prateekbh Dec 19, 2018
434a813
Merge branch 'sw-webpack' of https://github.com/developit/preact-cli …
prateekbh Dec 19, 2018
9ed7049
Update sw-plugin.js
prateekbh Dec 21, 2018
fb28138
Update sw-plugin.js
prateekbh Dec 21, 2018
81bb193
Merge branch 'next' into sw-webpack
ForsakenHarmony Jan 26, 2019
dc64998
making changes for workbox v4
prateekbh Jan 31, 2019
9a04494
fixes for regexp
prateekbh Jan 31, 2019
95d3ccb
addressing comments
prateekbh Feb 20, 2019
400fdb2
precacing only index.html
prateekbh Mar 6, 2019
7f21e24
Merge branch 'next' into sw-webpack
prateekbh Mar 6, 2019
fdf81b4
Update run-webpack.js
prateekbh Mar 6, 2019
37d9ec7
Update run-webpack.js
prateekbh Mar 6, 2019
9054e9b
fixing kluer dep
prateekbh Mar 6, 2019
126cd49
fixing package.json
prateekbh Mar 6, 2019
c95b6dc
fixing unhashed bundle bug
prateekbh Mar 7, 2019
4302de4
no json parse
prateekbh Mar 7, 2019
8935426
fixing comments
prateekbh Apr 5, 2019
b12603b
Merge branch 'master' of https://github.com/developit/preact-cli into…
prateekbh Apr 5, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions packages/cli/lib/commands/build.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ const rimraf = require('rimraf');
const { resolve } = require('path');
const { promisify } = require('bluebird');
const { isDir, error } = require('../util');
const { yellow } = require('chalk');
const runWebpack = require('../lib/webpack/run-webpack');

const toBool = val => val === void 0 || (val === 'false' ? false : val);
Expand All @@ -19,6 +20,11 @@ module.exports = async function (src, argv) {
return error('No `node_modules` found! Please run `npm install` before continuing.', 1);
}

if (argv.brotli) {
console.log(yellow("⚛️ ATTENTION! You have enabled BROTLI support. "
+ "In order for this to work correctly, make sure .js.br files are served with 'content-encoding: br' header."));
}

if (argv.clean === void 0) {
let dest = resolve(cwd, argv.dest);
await promisify(rimraf)(dest);
Expand Down
1 change: 1 addition & 0 deletions packages/cli/lib/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ prog
.option('--prerenderUrls', 'Path to pre-rendered routes config', 'prerender-urls.json')
.option('-c, --config', 'Path to custom CLI config', 'preact.config.js')
.option('--esm', 'Builds ES-2015 bundles for your code.', true)
.option('--brotli', 'Adds brotli redirects to the service worker.', false)
.option('--inline-css', 'Adds critical css to the prerendered markup.', true)
.action(commands.build);

Expand Down
3 changes: 0 additions & 3 deletions packages/cli/lib/lib/entry.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,6 @@ if (process.env.NODE_ENV==='development') {
if (!process.env.ADD_SW && 'serviceWorker' in navigator) {
// eslint-disable-next-line no-undef
navigator.serviceWorker.register(__webpack_public_path__ + 'sw-debug.js');
} else if (process.env.ADD_SW && 'serviceWorker' in navigator) {
// eslint-disable-next-line no-undef
navigator.serviceWorker.register(__webpack_public_path__ + (process.env.ES_BUILD ? 'sw-esm.js' : 'sw.js'));
}
}
else if (process.env.ADD_SW && 'serviceWorker' in navigator) {
Expand Down
54 changes: 54 additions & 0 deletions packages/cli/lib/lib/sw.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
self.__precacheManifest = [].concat(self.__precacheManifest || []);

/* global workbox */
workbox.precaching.suppressWarnings();
/** We are sure brotli is enabled for browsers supporting script type=module
* so we do brotli support only for them.
* We can do brolti support for other browsers but there is no good way of
* feature detect the same at the time of pre-caching.
*/
if (process.env.ENABLE_BROTLI && process.env.ES_BUILD) {
// Alter the precache manifest to precache brotli files instead of gzip files.
self.__precacheManifest = self.__precacheManifest.map(asset => {
if (/.*.js$/.test(asset.url)) {
asset.url = asset.url.replace(/.esm.js$/, ".esm.js.br");
}
return asset;
});

class BrotliRedirectPlugin {
// Before saving the response in cache, we need to treat the headers.
async cacheWillUpdate({request, response}) {
if (/.js.br$/.test(request.url)) {
const headers = response.headers;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this work instead of manually cloning?

response = response.clone();
response.headers.set('content-type', 'application/javascript')
return response;

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should – it's what my service workers are doing, tho I just pray that they work

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

am i doing something wrong here?

screen shot 2018-12-07 at 3 51 38 pm

Copy link
Member

@developit developit Feb 13, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my bad - this would work though:

const headers = new Headers(clonedResponse.headers);
newHeaders.set('content-type', 'application/javascript');
return new Response(await clonedResponse.text(), { headers })

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

const newHeaders = {};
for (let key of headers.keys()) {
newHeaders[key] = headers.get(key);
}
// make sure the content type is compatible
newHeaders["content-type"] = "application/javascript";
return new Response((await response.text()), {
headers: newHeaders,
});
}
return response;
}
}
workbox.precaching.addPlugins(new BrotliRedirectPlugin());
}

const precacheOptions = {};
if (process.env.ENABLE_BROTLI) {
developit marked this conversation as resolved.
Show resolved Hide resolved
precacheOptions['urlManipulation'] = ({url}) => {
if (/.esm.js$/.test(url.href)) {
url.href = url.href + ".br";
}
return [url];
};
}

workbox.precaching.precacheAndRoute(self.__precacheManifest, precacheOptions);

workbox.routing.registerNavigationRoute("index.html", {
whitelist: [/^(?!\/__).*/],
});
16 changes: 9 additions & 7 deletions packages/cli/lib/lib/webpack/run-webpack.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,14 @@ const { error, isDir, warn } = require('../../util');

async function devBuild(env) {
let config = clientConfig(env);
const webpackClientConfig = config[config.length - 1];
// client config will always be the last config in array, irrespective of sw is enabled or not..
await transformConfig(env, webpackClientConfig);

await transformConfig(env, config);

let userPort = parseInt(process.env.PORT || config.devServer.port, 10) || 8080;
let userPort = parseInt(process.env.PORT || webpackClientConfig.devServer.port, 10) || 8080;
let port = await getPort(userPort);

let compiler = webpack(config);
let compiler = webpack(webpackClientConfig);
return new Promise((res, rej) => {
compiler.plugin('emit', (compilation, callback) => {
let missingDeps = compilation.missingDependencies;
Expand All @@ -34,7 +35,7 @@ async function devBuild(env) {
});

compiler.plugin('done', stats => {
let devServer = config.devServer;
let devServer = webpackClientConfig.devServer;
let protocol = (process.env.HTTPS || devServer.https) ? 'https' : 'http';

let host = process.env.HOST || devServer.host || 'localhost';
Expand Down Expand Up @@ -63,7 +64,7 @@ async function devBuild(env) {

compiler.plugin('failed', rej);

let c = Object.assign({}, config.devServer, {
let c = Object.assign({}, webpackClientConfig.devServer, {
stats: { colors:true }
});

Expand All @@ -75,7 +76,8 @@ async function devBuild(env) {

async function prodBuild(env) {
let config = clientConfig(env);
await transformConfig(env, config);
const webpackClientConfig = config[config.length - 1];
await transformConfig(env, webpackClientConfig);

if (env.prerender) {
let ssrConfig = serverConfig(env);
Expand Down
79 changes: 35 additions & 44 deletions packages/cli/lib/lib/webpack/webpack-client-config.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ const { existsSync } = require('fs');
const merge = require('webpack-merge');
const { filter } = require('minimatch');
const CopyWebpackPlugin = require('copy-webpack-plugin');
const SWPrecacheWebpackPlugin = require('sw-precache-webpack-plugin');
const TerserPlugin = require('terser-webpack-plugin');
const OptimizeCssAssetsPlugin = require('optimize-css-assets-webpack-plugin');
const { BundleAnalyzerPlugin } = require('webpack-bundle-analyzer');
Expand All @@ -13,7 +12,10 @@ const RenderHTMLPlugin = require('./render-html-plugin');
const PushManifestPlugin = require('./push-manifest');
const baseConfig = require('./webpack-base-config');
const BabelEsmPlugin = require('babel-esm-plugin');
const { InjectManifest } = require('workbox-webpack-plugin');
const BrotliPlugin = require('brotli-webpack-plugin');
const { normalizePath } = require('../../util');
const swWebPackConfig = require('./webpack-sw-config');

const cleanFilename = name => name.replace(/(^\/(routes|components\/(routes|async))\/|(\/index)?\.js$)/g, '');

Expand Down Expand Up @@ -114,7 +116,7 @@ function isProd(config) {
'process.env.ADD_SW': config.sw,
'process.env.ES_BUILD': false,
'process.env.ESM': config.esm,
})
}),
],

optimization: {
Expand Down Expand Up @@ -146,26 +148,6 @@ function isProd(config) {
},
};

if (config.sw) {
prodConfig.plugins.push(
new SWPrecacheWebpackPlugin({
filename: 'sw.js',
navigateFallback: 'index.html',
navigateFallbackWhitelist: [/^(?!\/__).*/],
minify: true,
stripPrefix: config.cwd,
staticFileGlobsIgnorePatterns: [
/\.esm\.js$/,
/polyfills(\..*)?\.js$/,
/\.map$/,
/push-manifest\.json$/,
/.DS_Store/,
/\.git/
]
}),
);
}

if (config.esm) {
prodConfig.plugins.push(
new BabelEsmPlugin({
Expand Down Expand Up @@ -193,26 +175,23 @@ function isProd(config) {
},
}),
);
config['sw'] && prodConfig.plugins.push(
new InjectManifest({
swSrc: resolve(config.dest, 'sw-esm.js'),
include: [/\.html$/, /\.esm.js$/, /\.css$/, /\.(png|jpg)$/],
precacheManifestFilename: 'precache-manifest.[manifestHash].esm.js'
}),
);
}

if (config.sw) {
prodConfig.plugins.push(
new SWPrecacheWebpackPlugin({
filename: 'sw-esm.js',
navigateFallback: 'index.html',
navigateFallbackWhitelist: [/^(?!\/__).*/],
minify: true,
stripPrefix: config.cwd,
staticFileGlobsIgnorePatterns: [
/(\.[\w]{5}\.js)/,
/polyfills(\..*)?\.js$/,
/\.map$/,
/push-manifest\.json$/,
/.DS_Store/,
/\.git/
]
}),
);
}
if (config['sw']) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason this isn't config.sw?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nups, fixed

prodConfig.plugins.push(
new InjectManifest({
swSrc: resolve(config.dest, 'sw.js'),
include: [/\.html$/, /\.js$/, /\.css$/, /\.(png|jpg)$/],
exclude: [/\.esm\.js$/]
})
);
}

if (config['inline-css']) {
Expand All @@ -229,6 +208,14 @@ function isProd(config) {
);
}

if (config.brotli) {
prodConfig.plugins.push(
new BrotliPlugin({
test: /\.esm\.js$/,
})
);
}

return prodConfig;
}

Expand Down Expand Up @@ -273,9 +260,13 @@ function isDev(config) {
}

module.exports = function (env) {
return merge(
const config = [merge(
baseConfig(env),
clientConfig(env),
(env.isProd ? isProd : isDev)(env)
);
};
)];
if (env.sw) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the Array return here done with the intention of moving to a MultiCompiler? This seems like it'll break in future Workbox versions that use child compilation.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is the child compiler affected by the its parent compilation mode?

config.unshift(swWebPackConfig(env));
}
return config;
};
62 changes: 62 additions & 0 deletions packages/cli/lib/lib/webpack/webpack-sw-config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
const webpack = require('webpack');
const merge = require('webpack-merge');
const baseConfig = require('./webpack-base-config');
const BabelEsmPlugin = require('babel-esm-plugin');
const fs = require('fs');
const {yellow} = require('chalk');
const { resolve } = require('path');

function swConfig(config) {
const { dest, src, brotli, esm } = config;

const plugins = [
new webpack.DefinePlugin({
'process.env.ES_BUILD': false,
'process.env.ENABLE_BROTLI': brotli,
'process.env.NODE_ENV': 'production',
}),
];

esm && plugins.push(
new BabelEsmPlugin({
filename: '[name]-esm.js',
beforeStartExecution: (plugins) => {
plugins.forEach(plugin => {
if (plugin.constructor.name === 'DefinePlugin' && plugin.definitions) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reason for only mutating values in-place here - does this fix DefinePlugin definitions leaking into the parent compiler?

Otherwise this could just be

plugins.forEach(plugin => {
  if (plugin.constructor.name === 'DefinePlugin') {
    if (!plugin.definitions) throw Error('ESM Error:  DefinePlugin found without definitions.');
    plugin.definitions['process.env.ES_BUILD'] = 'true';
  }
});

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point, i can chk that

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

works, fixed

for (const definition in plugin.definitions) {
if (definition === 'process.env.ES_BUILD') {
plugin.definitions[definition] = true;
}
}
} else if (plugin.constructor.name === 'DefinePlugin' && !plugin.definitions) {
throw new Error('WebpackDefinePlugin found but not `process.env.ES_BUILD`.');
}
});
}
})
);

let swSrc = resolve(__dirname, './../sw.js');
if (fs.existsSync(resolve(`${src}/sw.js`))) {
console.log(yellow('⚛️ Info - CLI found sw.js in source root folder, using that to compile the final service worker instead'));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: can we make this a different color like blue? yellow seems like a warning, this is informational.

Also maybe shorten the message since it's a heads-up that will be shown every build? Or only show the message when things change (first build or sw.js added/removed):

  // only warn on first build, OR on rebuild if sw.js is added/removed:
  let swSrc = resolve(__dirname, './../sw.js');
  const exists = fs.existsSync(resolve(`${src}/sw.js`));
  if (exists !== global.hasSeenSw) {
    global.hasSeenSw = exists;
    if (exists) {
      console.log(blue('⚛️ Detected custom sw.js: compiling instead of default Service Worker.'));
    }
    else {
      console.log(blue('⚛️ No custom sw.js detected: compiling default Service Worker.'));
    }
  }

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

swSrc = resolve(`${src}/sw.js`);
}

return {
context: src,
entry: {
sw: swSrc,
},
target: 'webworker',
output: {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.. i guess?

path: dest,
publicPath: '/',
filename: '[name].js',
},
plugins,
};
}

module.exports = function (env) {
return merge(baseConfig(env), swConfig(env));
};
6 changes: 4 additions & 2 deletions packages/cli/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -58,12 +58,13 @@
"@babel/register": "^7.0.0-beta.51",
"@preact/async-loader": "next",
"autoprefixer": "^8.4.1",
"babel-esm-plugin": "0.2.7",
"babel-esm-plugin": "^0.2.9",
"babel-loader": "^8.0.0-beta.3",
"babel-plugin-jsx-pragmatic": "^1.0.2",
"babel-plugin-transform-export-extensions": "^6.22.0",
"babel-plugin-transform-react-remove-prop-types": "^0.4.5",
"bluebird": "^3.5.0",
"brotli-webpack-plugin": "^1.0.0",
"chalk": "^2.4.1",
"console-clear": "^1.0.0",
"copy-webpack-plugin": "^4.1.0",
Expand Down Expand Up @@ -117,6 +118,7 @@
"webpack-dev-server": "^3.1.3",
"webpack-merge": "^4.1.0",
"webpack-plugin-replace": "^1.1.1",
"which": "^1.2.14"
"which": "^1.2.14",
"workbox-webpack-plugin": "^3.6.3"
}
}