WIP: Add support for WebWorker with worker-loader

[iansu]

This is an updated version of #3934

Use worker-loader to turn any file that ends with .worker.js into a WebWorker.

Closes #3660

Here is the sample WebWorker code I used to test this:

// hello.worker.js

let helloInterval;

const sayHello = () => {
  self.postMessage({ message: 'Hello' });
};

self.addEventListener('message', event => {
  if (event.data.run === true) {
    self.postMessage({ status: 'Worker started' });
    helloInterval = setInterval(sayHello, 1000);
  }

  if (event.data.run === false) {
    self.postMessage({ status: 'Worker stopped' });
    clearInterval(helloInterval);
  }
});

// index.js

import React from 'react';
import ReactDOM from 'react-dom';
import './index.css';
import App from './App';
import * as serviceWorker from './serviceWorker';
import HelloWorker from './hello.worker.js';

ReactDOM.render(<App />, document.getElementById('root'));

// If you want your app to work offline and load faster, you can change
// unregister() to register() below. Note this comes with some pitfalls.
// Learn more about service workers: http://bit.ly/CRA-PWA
serviceWorker.unregister();

const helloWorker = new HelloWorker();
let messageCount = 0;

helloWorker.postMessage({ run: true });

helloWorker.onmessage = event => {
  if (event.data.status) {
    console.log('STATUS', event.data.status);
  }

  if (event.data.message) {
    messageCount += 1;
    console.log('MESSAGE', event.data.message);

    if (messageCount >= 5) {
      helloWorker.postMessage({ run: false });
    }
  }
}

To Do

• Make ESLint ignore references to self in workers
• Add TypeScript support

[iansu]

This is required due to what seems to be a bug/regression in webpack 4. More details can be found here: webpack/webpack#6642

[r17x]

Hello @iansu , can i ask a question . Why globalObject value is this , not self ?

[developit]

+1

[ortonomy]

@ri7nz -- as far as I can tell, the eslint config inside create-react-app bugs out on self keyword saying it's invalid, and hence the empty production build when using worker-loader yourself.

[edtz]

This is dope, but FYI eslint marks any use of "self" inside workers as error because of the rule "no-restricted-globals".

[iansu]

Yes, that's true. If we decide to go forward with this we'll want to relax that rule (hopefully only in .worker.js files). I'll look into that.

[0xcaff]

Also, we'd probably want to do something similar for typescript?

Some Ideas of how to do this.

Idea 1

When compiling typescript for a webworker, add the "webworker" to the lib field in compilerOptions. https://stackoverflow.com/questions/38715001/how-to-make-web-workers-with-typescript-and-webpack

I'm not sure how well this would work if both the worker and main bundle require the same file. They would probably be compiled twice. Maybe the default tsconfig.json should have webworker typings enabled everywhere.

Idea 2

Make the user explicitly reference the ambient declaration file for workers at the top of .worker.ts files.

This would look something like:

/// <reference lib="webworker" />

self.addEventListener((message) => console.log(message));

I think idea 2 is better but it does push some work onto the end user of create react app. What do you think?

[stale]

This pull request has been automatically marked as stale because it has not had any recent activity. It will be closed in 5 days if no further activity occurs.

[ghost]

Any updates for this PR?

[iansu]

No updates at the moment. There's still some more work to do to get this ready. I've added a couple of to do items to the description.

[ghost]

@iansu Right, thanks 👍

[j-lee8]

Thanks for all your hard work guys. Any idea of date/time when this will be released? We've had to revert back to an older Webpack 3 non-PWA version of our product as the worker worked flawlessly then.

Thanks

[stale]

This pull request has been automatically marked as stale because it has not had any recent activity. It will be closed in 5 days if no further activity occurs.

[mishawakerman]

Is this PR still being worked on?

[iansu]

I'm not actively working on it at the moment but I am still planning on getting it into Create React App one of these days.

[mishawakerman]

@iansu, I've taken a look at your PR and am not seeing any ESLint issues relating to self in the workers (after creating a dummy app like yours). Is this now solved? If not, can you confirm what you mean by this?

I'm also happy to take a look at TypeScript support (though this isn't an area I know very well).

[jzhng]

Thx for the pr and I stealed it to my project first. To whom may use worker-loader in CRA, please be aware of this issue webpack-contrib/worker-loader#176, yarn build to generate production build may fail.

[T89L]

I see them building up here in the console if I don't terminate them. My use case is a little weird/complex so maybe this is out of scope for the typical implementation.

[kentcdodds]

Gotcha. Yeah, I think the way you get rid of a worker is to let it get garbage collected, but I don't think that's possible with workerize. You might ask in the issues on workerize.

[ortonomy]

Worker.terminate()

https://developer.mozilla.org/en-US/docs/Web/API/Worker/terminate

[kentcdodds]

😅

[mattdarveniza]

Thanks for the merge @iansu. What's the status of this PR now, given workerize-loader now works with CRA. Is the intent to still support workers "natively" in react-scripts at some point? Or will this be dropped in favour of using workerize-loader?

[akuji1993]

Would be great if the actual description of what you did wouldn't be paywalled / restricted to your workshop @kentcdodds

[mattdarveniza]

@kentcdodds nice work with the workerize workaround. Seems to work well in dev, however when including a worker, the production build doesn't seem to output any files. Have you seen this? Might raise an issue in CRA about this. I've got a reproducible case here if anyone wants to take a look https://github.com/mattdarveniza/worker-cra-test

[serdaroquai]

here is a workaround developit/workerize-loader#48 (comment) for unit testing react components that import workers using workerize-loader

[kentcdodds]

@mattdarveniza, I'm not sure what's wrong with yours (didn't have time to look at it), but I can tell you that mine works fine in production: https://react-performance.netlify.com/isolated/final/02.extra-1.js

[thasmin]

I copied the diff into my source tree and was able to use the workers with Typescript just fine.

[nickretallack]

Still hoping this gets merged eventually.

I'm working on a project that uses worker-loader. I patched it into the webpack configuration using patch-package so I wouldn't have to eject. It has some issues with eslint though:

Since I'm using self, it fails the no-restricted-globals check. However, react-scripts build silently ignores the error, pretends to succeed, and doesn't produce a build. I can see the error by adding the console.log described here, and I can fix this by putting eslint-ignore-next-line on every line that uses self.

I tried to ignore it using an override in my .eslintrc like so:

{
  "extends": "react-app",
  "overrides": [
    {
      "files": ["*.worker.js"],
      "rules": {
        "no-restricted-globals": "off"
      }
    }
  ]
}

This override works if I'm ejected but it doesn't work if I'm un-ejected, even if I have EXTEND_ESLINT=true in my .env file. I know I'm properly extending eslint because I can put other overrides in my .eslintrc file that get respected regardless of whether I'm ejected or not, but when I'm un-ejected they only seem to work if I use simple file extensions like so:

{
  "extends": "react-app",
  "overrides": [
    {
      "files": ["*.js"],
      "rules": {
        "no-restricted-globals": "off"
      }
    }
  ]
}

Any other type of pattern in files just gets ignored.

The error I see get when I add in the console.log is:

    ERROR in ./src/webWorkers/my.worker.js (./node_modules/react-scripts/node_modules/babel-loader/lib??ref--7-oneOf-1!./node_modules/eslint-loader/dist/cjs.js??ref--5-0!./src/webWorkers/my.worker.js)
    Module Error (from ./node_modules/eslint-loader/dist/cjs.js):
    
      Line 24:1:  Unexpected use of 'self'  no-restricted-globals

Here's my patch-package patch:

index 350d424..7afc84b 100644
--- a/node_modules/react-scripts/config/webpack.config.js
+++ b/node_modules/react-scripts/config/webpack.config.js
@@ -307,6 +307,7 @@ module.exports = function(webpackEnv) {
           'scheduler/tracing': 'scheduler/tracing-profiling',
         }),
         ...(modules.webpackAliases || {}),
+        'src': paths.appSrc,
       },
       plugins: [
         // Adds support for installing with Plug'n'Play, leading to faster installs and adding
@@ -375,6 +376,13 @@ module.exports = function(webpackEnv) {
           ],
           include: paths.appSrc,
         },
+        {
+          test: [/\.worker\.(js|ts)$/],
+          include: paths.appSrc,
+          use: [
+            { loader: 'worker-loader' }
+          ]
+        },
         {
           // "oneOf" will traverse all following loaders until one will
           // match the requirements. When no loader matches it will fall
@@ -668,6 +676,9 @@ module.exports = function(webpackEnv) {
       // https://github.com/jmblog/how-to-optimize-momentjs-with-webpack
       // You can remove this if you don't use Moment.js:
       new webpack.IgnorePlugin(/^\.\/locale$/, /moment$/),
+      // dtrace-provider is an optional dependency that gives us build time warnings
+      // this makes webpack ignore this file, so we don't see unnecessary warnings.
+      new webpack.IgnorePlugin(/dtrace-provider/),
       // Generate a service worker script that will precache, and keep up to date,
       // the HTML & assets that are part of the webpack build.
       isEnvProduction &&
@@ -692,6 +703,7 @@ module.exports = function(webpackEnv) {
           typescript: resolve.sync('typescript', {
             basedir: paths.appNodeModules,
           }),
+          memoryLimit: 99999,
           async: isEnvDevelopment,
           useTypescriptIncrementalApi: true,
           checkSyntacticErrors: true,
diff --git a/node_modules/react-scripts/scripts/build.js b/node_modules/react-scripts/scripts/build.js
index fa30fb0..fe09f9f 100644
--- a/node_modules/react-scripts/scripts/build.js
+++ b/node_modules/react-scripts/scripts/build.js
@@ -163,6 +163,7 @@ function build(previousFileSizes) {
   const compiler = webpack(config);
   return new Promise((resolve, reject) => {
     compiler.run((err, stats) => {
+      console.log(stats.toString())
       let messages;
       if (err) {
         if (!err.message) {

[christian-valadez]

+1 on this PR, would love to see this become something that's supported by default

[NicolasRannou]

Wrote a small post for myself if you want to use web-workers in CRA without unmounting - you may find it interesting -> https://dev.to/nicolasrannou/web-workers-in-create-react-app-cra-without-unmounting-4865

[LarsKoelpin]

Is this PR still beeing implemented or is it aborted already?

[seedy]

Wrote a small post for myself if you want to use web-workers in CRA without unmounting - you may find it interesting -> https://dev.to/nicolasrannou/web-workers-in-create-react-app-cra-without-unmounting-4865

@NicolasRannou Nice post man, thanks !

I was also thinking about integrating comlink.

For those who handle the harsh IE11 support, you might need some polyfills on the way

• comlink requires proxy-polyfill
• if you opt for workerize-loader, it requires promise-polyfill (handled by react-app-polyfill)

Edit : I am trying comlink-loader, which ships all my requirements together 😍

[developit]

FWIW Webpack 5 has added support for bundling workers that are initialized with a module-relative path:

new Worker(new URL ('./foo.js', import.meta.url))

[Lagicrus]

Any updates on this PR? 😅

[alvaroschipper]

+1 would be very nice to have indeed.

[mattdarveniza]

Hola, back in this thread after a successful year using workerize-loader just with the workerize-loader!./path/to/worker syntax. This has worked relatively well but fast-refresh throws a spanner in the works as it needs to be disabled as a plugin for anything that runs through a worker as per here.

It looks like the most sensible solution here would be a separate loader for workers in the webpack.config.js, at which point worker-loader or workerize-loader might as well be supported natively if we can iron out all the kinks.

This seems to have dropped off the list of things you're (understandably) prioritising @iansu, I might have a crack at this fresh given the changes in react-scripts@4, although I suppose if there's native worker support in webpack 5 and that's in the pipeline for 4.1 or something, it might just be easier to wait?

[Timer]

Just leaving this review since I saw it in my pending requests! Best of luck with the PR! Seems to be a GitHub bug since I no longer have rights.