next dev fails with webpack error when importing functions inside web worker

[paescuj]

What version of Next.js are you using?

10.2.3

What version of Node.js are you using?

16.2.0

What browser are you using?

Firefox

What operating system are you using?

macOS

How are you deploying your application?

next dev

Describe the Bug

Using a web worker, similar to the with-web-worker example. The web worker file (located under worker/search.js) imports functions from another local file like this:

import { getIndexRange, getTextItemWithNeighbors } from '../lib/search';

Which then results in the following error with Next.js >=10.2.1:

error - webpack/runtime/compat
The "path" argument must be of type string. Received undefined

I'm able to resolve this error by in-lining the functions directly in the web worker file (worker/search.js) instead of importing them.

This issue appeared with v10.2.1-canary.9, so probably introduced by #25035. It used to work before.

Expected Behavior

I would like to be able to import the functions in the web worker file without an error, as it used to work in the past.

To Reproduce

Honestly, it's pretty hard to reproduce this issue... I've tried to apply a similar structure as in my project to the with-web-worker example, but wasn't able to reproduce it.
Maybe @nemanja-tosic (#25276 (comment)) has some clues about it...

[nemanja-tosic]

Through trial and error, the error goes away after i comment out a spread expression, i.e:

return { type: 'Message', ...options };

Cannot reproduce in the example though. That being said, we have a monorepo with typescript where we import the function that contains the spread from another package. I'll try and set that up when i have the time, dropping a line in case someone finds it useful.

[mdirolf]

Saw this as well and reverted to 10.1 for the time being. Thanks for looking into it!

[Qeldrona]

I'm also running into this error on 10.2.3 and 11. This error is definitely related to the only web worker in our project (follows the with-web-worker example), commenting it out allows the build to run successfully.

[Qeldrona]

Just tried this on next@11.0.1-canary.5 and it is still an issue. I know that contained the PR fixing some other web worker issues so I just wanted to check on this one too.

[timneutkens]

Just tried this on next@11.0.1-canary.5 and it is still an issue. I know that contained the PR fixing some other web worker issues so I just wanted to check on this one too.

Please provide a reproduction so that we can have a look.

[Qeldrona]

I've been trying to reproduce all morning in a clean project to no avail, unfortunately I can't share the project where the error is occurring. I have found a workaround and a possible lead:

I paired down my web worker to the basic example from Webpack's documentation https://webpack.js.org/guides/web-workers/ and that one was working. Then I slowly started adding imports to resemble the existing worker. I found when I imported a basic GUID function from its current location then I got an error, but if I split it out into it's own file then everything built fine. So My guess is that the import chain is dying when file sizes get too large.

Looking at the splitChunks option in next.config.js, I found that using these basic settings from Webpack documentation resolved the error https://webpack.js.org/plugins/split-chunks-plugin/

config.optimization.splitChunks = {
      chunks: 'async',
      minSize: 20000,
      minRemainingSize: 0,
      minChunks: 1,
      maxAsyncRequests: 30,
      maxInitialRequests: 30,
      enforceSizeThreshold: 50000,
      cacheGroups: {
        defaultVendors: {
          test: /[\\/]node_modules[\\/]/,
          priority: -10,
          reuseExistingChunk: true,
        },
        default: {
          minChunks: 2,
          priority: -20,
          reuseExistingChunk: true,
        },
      },
    }

So I suppose the error is something in webpack-config.ts in Next.js but there is so much going on in there I am not sure if this really narrows it down.

[Qeldrona]

Comparing against the splitChunks config for prodGranular in the current version of Next vs 10.0.0 (last good version we were on), I was able to pull these settings into mine and still compile successfully:

    config.optimization.splitChunks = {
      chunks: (chunk) => !/^(polyfills|main|pages\/_app)$/.test(chunk.name),
      minSize: 20000,
      minRemainingSize: 0,
      minChunks: 1,
      maxAsyncRequests: 30,
      maxInitialRequests: 25,
      enforceSizeThreshold: 50000,
      cacheGroups: {
        defaultVendors: false,
        default: false,
        framework: {
          chunks: 'all',
          name: 'framework',
          test: /(?<!node_modules.*)[\\/]node_modules[\\/](react|react-dom|scheduler|prop-types|use-subscription)[\\/]/,
          priority: 40,
          enforce: true,
        },
      },
    },

Unfortunately the rest are in TypeScript and there is little I can do on those ones. I tried pulling "commons" over and just setting minChunks: 1 and that threw the same error: The "path" argument must be of type string. Received undefined

webpack: (config, options) => {
    config.optimization.splitChunks = {
      chunks: (chunk) => !/^(polyfills|main|pages\/_app)$/.test(chunk.name),
      minSize: 20000,
      minRemainingSize: 0,
      minChunks: 1,
      maxAsyncRequests: 30,
      maxInitialRequests: 25,
      enforceSizeThreshold: 50000,
      cacheGroups: {
        defaultVendors: false,
        default: false,
        framework: {
          chunks: 'all',
          name: 'framework',
          test: /(?<!node_modules.*)[\\/]node_modules[\\/](react|react-dom|scheduler|prop-types|use-subscription)[\\/]/,
          priority: 40,
          enforce: true,
        },
        commons: {
          name: 'commons',
          minChunks: 1,
          priority: 20,
        },
      },
    }

So I moved the config over to my sample project and reproduced the error there:
https://github.com/Qeldrona/prototype

If you comment out the web worker useEffect hook in index.js OR remove the commons object in next.config.js then npm run build will be successful. If you have them together the build will throw the error.

Alternatively, raising minChunks manually based on project size seems to also work. e.g. in our project 10 breaks but 50 compiles.

[lagz0ne]

OMG, unbelieve that I was able to reproduce this. It happens when there are 2 different webworker sharing same dependencies.

https://github.com/Lagz0ne/web-worker-build-error

Those backgrounds are sharing those imports and those imports exceed a certain size threshold (the data file is pretty big). Only then the yarn build yells

webpack/runtime/compat
The "path" argument must be of type string. Received undefined

It won't happen if we remove that data from the background bundle.

And, yarn dev works. yarn build yells that error

Next version is 11.0.1

[nemanja-tosic]

For what it's worth, we only had the one webworker in the app. Good thing you managed to reproduce it though!

[Qeldrona]

We also only had the one worker. The import size being an issue would be consistent with ours though.

[Qeldrona]

After spending some more time on this, I noticed that on the webpack documentation there is a property on commons for chunks: "initial" which is missing from next by default as far as I can tell and after adding that to my configuration I was able to add commons back into our config without the error. Curious if this works for others as well (note that minChunks number should be balanced based on your project size to get the initial bundle sizes that work for you)

cacheGroups: {
   defaultVendors: false,
   default: false,
   framework: {
     chunks: "all",
     name: "framework",
     test: /(?<!node_modules.*)[\\/]node_modules[\\/](react|react-dom|scheduler|prop-types|use-subscription)[\\/]/,
     priority: 40,
     enforce: true,
   },
   commons: {
     name: "commons",
     chunks: "initial",
     minChunks: 20,
     priority: 20
   },
 }, 

[lagz0ne]

Do you guys have any workaround? This bug really blocking us from moving to Nextjs. It is a pity because now working with version 10, we'll see how slow it was

[Qeldrona]

@lagz0ne Does my workaround with the splitchunks config in next.config.js not work for you? We were able to move forward with it while this gets fixed.

[lagz0ne]

@Qeldrona thanks. I can confirm it works using your patch

others can find the reference here https://github.com/lagz0ne/web-worker-build-error/tree/bump-nextjs-version

Cheers

[anonymouscatcher]

Same issue when using web workers. working fine on dev but while building the app I also get same error. This happens after migrating to next@11.01, going back to use ^10.02

[colmanhumphrey]

Confirming same error also on next@11.1 after adding a second worker (works on dev, fails on build). Using the patch/config from @Qeldrona and @lagz0ne for now

[dontsave]

is this issue being addressed? it's still a problem in Next 12 and the above workaround does solve it, however it adds 200K to our total gzipped bundle size

edit: it turns out this was only an issue for us on the server, so we were able to flag the workaround conf against isServer and keep our client bundle using the default Next conf. hope this helps anyone running into this

[njarraud]

I am running into the same issue on build (works on dev) with Nextjs 12.0.2 for client side webworkers.

If a second worker imports the same file as the first one, build fails.
If a worker imports file from external packages used somewhere else as well, build fails.

@Qeldrona It works - Size is roughly similar in my case but build time is x2

[eschaefer]

Confirmed @Qeldrona 's fix works for us.

Basically just added this to next.config.js

const _ = require('lodash');

const nextConfig = {
  webpack: (config) => {
    _.set(
      config,
      'optimization.splitChunks.cacheGroups.commons.chunks',
      'initial'
    );

    return config;
  },
};

module.exports = nextConfig;

[yakovenkoroman1993]

I confirm that fix which suggested by @eschaefer works and more simple

[talaikis]

config.optimization.splitChunks = { solution works for Next 12+, just build process is much longer and pages are much bigger now.

[andreypopp]

On latest next 12 and the workaround doesn't work for me...

@talaikis any chance you could share the whole snippet which fixes the issue?

[balazsorban44]

I was unable to reproduce this using the with-web-worker example as the OP suggested. It also uses local imports:

next.js/examples/with-web-worker/worker.js

Lines 1 to 2 in c74e4f2

	// This is a module worker, so we can use imports (in the browser too!)
	import pi from './utils/pi'

Anyone else having similar/different problems, please open a separate issue with an attached reproduction.

[andreypopp]

Sure, submitted #34064 with the repro.

[Qeldrona]

@balazsorban44 You've got to be kidding, we've got several reproductions and this is impacting lots of people in production. I've brought this up with our company's account rep repeatedly and been ignored, and here it also seems to be something the team would rather ignore than seriously look at our reproductions and solve.

[lagz0ne]

@balazsorban44 I don't think this is how you are representing Vercel, if you follow the content of the issue before even too eager to just close issues, you should figure out that there's a BIG ASS REPRODUCIBLE repo using create-next-app right here #25484 (comment). The repo is using nextjs: latest and IT IS STILL ERROR when you do yarn build.

To be very honest with Vercel, this bug fucks us quite a bit for a while and now a random guy just steps in and closes this as unreproducible. Are you freaking serious, @timneutkens

[colmanhumphrey]

Hey @lagz0ne this bug frustrates me too, been waiting on a fix for a while, and it does seem like a somewhat inconsiderate oversight from @balazsorban44. So I totally understand where you're coming from. But I think it's worth expressing our frustrations as Nextjs users without getting too personal.

[lagz0ne]

thanks, @colmanhumphrey, I think that makes sense, hot heated got me. I updated the repo with the upgrade https://github.com/Lagz0ne/web-worker-build-error to the latest nextjs, somehow the error displayed twice now

[rauchg]

Hey folks. We are definitely paying attention here and we will take action if we closed by mistake. Thanks for all the feedback and effort into giving us further reproductions 🙏

[balazsorban44]

Thanks for the reminders everyone, and thank you @rauchg for stepping in!

I would like to apologize if someone felt ignored. I did - to the best of my abilities - try to reproduce the issue based on what the OP suggested (using with-web-worker), but I could not.

I also tried @lagz0ne's repro #25484 (comment), and could not reproduce the problem with next dev, which is what this issue was about, according to my understanding. I should have been more clear in my closing comment though because I did see the issue while running yarn build, maybe I could have created the new issue myself and referred to that specific comment.

Once again, I do apologize, and I would like to assure everyone that I look at all comments in this repository, even on closed issues, and I would have eventually circled back on this if there were new comments/concerns.

That said, we took this issue up immediately with the rest of the team once I got available, and @sokra has already opened a PR at #34087 that should fix this issue.

[talaikis]

In my case it was/ is not Next.js related: serverless-nextjs/serverless-next.js#2220

[Qeldrona]

@balazsorban44 Thanks for circling back and fully explaining, and for the team getting a potential fix pushed. We really appreciate you picking this up and getting some traction on it. I can't wait to try out the fix.

[lagz0ne]

Hi guys, thanks @rauchg for stepping in. I do apologize for all of my harsh words, just a matter of fact, words mean everything, we all definitely do better with our wordings. I guess the takeaway here at the end of the day is all about empathy.

I would make my statement less aggressive if I stand from the point of view of @balazsorban44, try to understand a little better about the rationale he takes.

Also, from @balazsorban44, the outcome will be quite different if he spent time to stand from our point of view, as end-users to decide what action to take. The opaque message comes with rushy action to close the issue didn't do good there.

All in all, I do appreciate prompt actions, responses from Vercel, and eager to look for the fix, can't wait to remove the ugly hack in the config file.

Cheers

[Qeldrona]

The fix is in the current canary branch and at least from my initial testing it appears to resolve the issue for us. Hope it does so for everyone else as well. I'll be doing further testing and deploys to make sure but so far so good!

[github-actions]

This closed issue has been automatically locked because it had no new activity for a month. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.