Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"/node_modules/cesium/Source/ThirdParty/zip.js" this file can't not be packed #9790

Closed
ydyvip opened this issue Sep 3, 2021 · 34 comments
Closed

Comments

@ydyvip
Copy link

ydyvip commented Sep 3, 2021

Module parse failed: Unexpected token (1565:71)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
| if (!workerData.interface) {
| try {

                  workerData.worker = new Worker(new URL(workerData.scripts[0], import.meta.url));

| } catch (error) {
| workerData.worker = new Worker(new URL(workerData.scripts[0], import.meta.url), { type: "module" });

@puxiao
Copy link
Contributor

puxiao commented Sep 3, 2021

Yes, I also encountered this problem when I upgraded to version 1.85.

Other people also suffered.

#9789

@bruce6135
Copy link

As a temp resolution, you can edit your package.json as following:
"dependencies": { "cesium": "1.80.0" }
This will precise install the 1.80.0 version

@puxiao
Copy link
Contributor

puxiao commented Sep 3, 2021

As a temp resolution, you can edit your package.json as following:
"dependencies": { "cesium": "1.80.0" }
This will precise install the 1.80.0 version

The 1.84 version I used before has no problems.

@ebogo1
Copy link
Contributor

ebogo1 commented Sep 3, 2021

Thanks all for catching this and opening the issue. We'll look into this asap.

@srothst1 This looks like it might be a regression with webpack after 1.85..

@srothst1
Copy link
Contributor

srothst1 commented Sep 3, 2021

@puxiao @ydyvip thank you for sharing this with us. Can you please provide a little bit more information on your development environment? What version of webpack, etc.

@puxiao
Copy link
Contributor

puxiao commented Sep 3, 2021

@srothst1


The following is my project.

  1. use create-react-app 4.0.3, yarn create react-app react-cesium-demo --template typescript

  2. use @craco/craco to add some webpack config

    carco.config.js

    const path = require('path');
    module.exports = {
        webpack: {
            configure: (config) => {
                //remove cesium warning
                config.module.unknownContextCritical = false
                config.module.unknownContextRegExp = /\/cesium\/cesium\/Source\/Core\/buildModuleUrl\.js/
                return config
            },
            output: {
                amd: {
                    // Enable webpack-friendly use of require in Cesium
                    toUrlUndefined: true
                }
            }
        }
    };
    
  3. package.json

    {
        "name": "react-cesium-demo",
        "version": "0.1.0",
        "private": true,
        "dependencies": {
            "@craco/craco": "^6.2.0",
            "@testing-library/jest-dom": "^5.11.4",
            "@testing-library/react": "^11.1.0",
            "@testing-library/user-event": "^12.1.10",
            "@types/jest": "^26.0.15",
            "@types/node": "^12.0.0",
            "@types/react": "^17.0.0",
            "@types/react-dom": "^17.0.0",
            "cesium": "^1.85.0",
            "node-sass": "5.0.0",
            "react": "^17.0.2",
            "react-dom": "^17.0.2",
            "react-scripts": "4.0.3",
            "typescript": "^4.3.5",
            "web-vitals": "^1.0.1"
        },
        "scripts": {
            "start": "craco start",
            "build": "craco build",
            "test": "craco test",
            "eject": "react-scripts eject"
        },
        "eslintConfig": {
            "extends": [
                "react-app",
                "react-app/jest"
            ]
        },
        "browserslist": {
            "production": [
                ">0.2%",
                "not dead",
                "not op_mini all"
            ],
            "development": [
                "last 1 chrome version",
                "last 1 firefox version",
                "last 1 safari version"
            ]
        },
        "devDependencies": {}
    }
    

    Note that the version used is 1.85.0.


When I returned the version to 1.84.0, the problem disappeared.

@meschg
Copy link

meschg commented Sep 4, 2021

Hello,
looks like I am having this issue only with 1.85.0. My project run every cesium version since over 2 years.

Failed to compile.

./node_modules/cesium/Source/ThirdParty/zip.js 1565:71
Module parse failed: Unexpected token (1565:71)
File was processed with these loaders:
 * ./node_modules/strip-pragma-loader/index.js
You may need an additional loader to handle the result of these loaders.
| 	if (!workerData.interface) {
| 		try {
> 			workerData.worker = new Worker(new URL(workerData.scripts[0], import.meta.url));
| 		} catch (error) {
| 			workerData.worker = new Worker(new URL(workerData.scripts[0], import.meta.url), { type: "module" });

I having a "template project" (https://github.com/meschg/vue-stack-cesium) for testing updates before updating my main application. I know it is not a "minimal sample" but checking multiple packages together is for me important. Might it helps you to find this error

Steps to reproduce:

  1. Checkout https://github.com/meschg/vue-stack-cesium
  2. npm install
  3. npm run serve -> should run :)
  4. npm install cesium@1.85.0
  5. npm run serve -> leads to the error above :(

I did not have time yet to look into more yet, not sure when I have time to do so. More input is welcome. Thanks to all investigating/working on this!

Kind regards,
Gerhard

@lilleyse
Copy link
Contributor

lilleyse commented Sep 7, 2021

This seems to be a regression from #9706. I marked this issue as priority - next release.

@OCISHY
Copy link

OCISHY commented Sep 15, 2021

nuxt.js meet this error

@ebogo1 ebogo1 self-assigned this Sep 20, 2021
@ebogo1
Copy link
Contributor

ebogo1 commented Sep 20, 2021

@puxiao - I tried creating a react project with the same configs as the ones you posted on Node 14 (LTS) and 16. I was unable to reproduce the error when building the project.

I was able to fix another issue with the 1.85 version of zip.js by making the following change in https://github.com/CesiumGS/cesium/blob/main/ThirdParty/npm/zip.js:

- import * as zip from "@zip.js/zip.js/lib/zip.js";
+ import * as zip from "@zip.js/zip.js/lib/zip-no-worker.js";
+ zip.configure({
+  useWebWorkers: false
+ });

To test this with your app locally, you can run npm install && npm run build && npm run makeZipFile from your CesiumJS repo folder and link to the CesiumJS package in your react app's package.json with

"dependencies": {
  "cesium": "file:/../cesium/cesium-1.85.0.tgz",
}

I'll keep digging into this but if anyone is able to try the above workaround please do post a comment!

@puxiao
Copy link
Contributor

puxiao commented Sep 21, 2021

@ebogo1

I found out the cause of the problem.

gildas-lormeau/zip.js#258 (comment)

create-react-appuses webpack 4 under the hood to build the application and webpack 4 does not fully support web workers without an additional loader.

To fix this issue, you must include the loader@open-wc/webpack-import-meta-loader in your project. You can do this by using for examplereact-app-rewiredand following these steps:

.......

@puxiao
Copy link
Contributor

puxiao commented Sep 21, 2021

My cesium.js 1.85 can be compiled normally in webpack4.


Apart from react-app-rewired, if you use create-react-app, i think @craco/craco is more suitable for cesium.

  1. run yarn add @craco/craco --dev

  2. run yarn add @open-wc/webpack-import-meta-loader --dev

  3. create a file named craco.config.js in the root folder of your project and paste the following content:

    module.exports = {
        webpack: {
            configure: (config) => {
                // remove cesium warning
                config.module.unknownContextCritical = false
                config.module.unknownContextRegExp = /\/cesium\/cesium\/Source\/Core\/buildModuleUrl\.js/
    
                // remove zip.js error in webpack4
                config.module.rules.push({
                    test: /\.js$/,
                    use: { loader: require.resolve('@open-wc/webpack-import-meta-loader') }
                });
                return config
            }
    };
    
  4. edit you package.json file and replace:

    - "start": "react-scripts start",
    - "build": "react-scripts build",
    - "test": "react-scripts test",
    
    + "start": "craco start",
    + "build": "craco build",
    + "test": "craco test",

By the way, today is China's Mid-Autumn Festival. I wish everyone:

中秋节快乐!

Happy Mid-Autumn Festival !

…… ^_^ ……

@meschg
Copy link

meschg commented Sep 21, 2021

@ebogo1
I tried to apply your workaround in my project https://github.com/meschg/vue-stack-cesium. I also created a branch but did have no useful commits yet.

I run a build with your changes but instead of using the cesium-1.85.0.tgz I copy everything over into the node-modules folder because the other rules I also have in place.

Now it get a different error in the KmlDataSource.js which seems to use a similar import.

ERROR in ./node_modules/cesium/Source/ThirdParty/zip.js 6357:57
Module parse failed: Unexpected token (6357:57)
File was processed with these loaders:
 * ./node_modules/strip-pragma-loader/index.js
You may need an additional loader to handle the result of these loaders.
| 
|       function getWorker(options = {}) {
>               return new Worker(new URL(workerData.scripts[0], import.meta.url), options);
|       }
| 
 @ ./node_modules/cesium/Source/DataSources/KmlDataSource.js 44:0-39 394:32-35 406:32-35 3168:19-22 3168:37-40
 @ ./node_modules/cesium/Source/
Widgets/Viewer/viewerDragDropMixin.js
 @ ./node_modules/cesium/Source/Cesium.js
 @ ./src/plugins/cesium.ts
...
 @ multi (webpack)-dev-server/client?http://192.168.178.70:8080&sockPath=/sockjs-node (webpack)/hot/dev-server.js ./src/main.ts

I hope it helps. Not sure if the problem is related to the strip-pragma-loader in my project.


I wasn't able to adapt the other changes with @open-wc/webpack-import-meta-loader to my project which was proposed by @puxiao. I am just at beginner level on all the webpack stuff :/

@puxiao
Copy link
Contributor

puxiao commented Sep 21, 2021

@meschg

After my attempt, I have fixed the problem you encountered.
I submitted a PR to you meschg/vue-stack-cesium#7
@open-wc/webpack-import-meta-loader also applies to Vue.

  1. run

    yarn add @open-wc/webpack-import-meta-loader --dev
    
    or
    
    npm i  @open-wc/webpack-import-meta-loader --save-dev
    
  2. edit you vue.config.js file.

    module.exports = {
        ...
        
        module: {
          ...
          rules: [{
            ...
    +      },{
    +        test: /\.js$/,
    +        use: { loader: require.resolve('@open-wc/webpack-import-meta-loader') }
    +      }
          ]
        }
      },
      ...
    }

@meschg
Copy link

meschg commented Sep 21, 2021

@puxiao workaround with the @open-wc/webpack-import-meta-loader works in with vue. If this is a good/intended dependency for using cesium for webpack is probably up to the cesium team to decided. Thanks again to puxiao!

@ebogo1 I could provide a branch with my template if required but it seems to be identical for each webpack project.

@blissvisitor
Copy link

@puxiao but when I build It cause a error : Syntax Error: Thread Loader (Worker 0)
Cannot read property 'length' of undefined; do you meet this question?

@puxiao
Copy link
Contributor

puxiao commented Sep 23, 2021

@blissvisitor
Copy link

@blissvisitor You can try this demo: https://github.com/puxiao/cesium-issues-9790

I tried @open-wc/webpack-import-meta-loader ,It can run nornal,but my problem is when I build my project with 'yarn run build' It comes a error Syntax Error: Thread Loader (Worker 0)

@blissvisitor
Copy link

@blissvisitor You can try this demo: https://github.com/puxiao/cesium-issues-9790

I tried @open-wc/webpack-import-meta-loader ,It can run nornal,but my problem is when I build my project with 'yarn run build' It comes a error Syntax Error: Thread Loader (Worker 0)

I find the answer, { test: /\.js$/, include: path.resolve(__dirname, 'node_modules/cesium/Source'), use: { loader: require.resolve('@open-wc/webpack-import-meta-loader') } }

add the include to ignore other files, make it only affect Cesium.

@blissvisitor
Copy link

{
test: /.js$/,
include: path.resolve(__dirname, 'node_modules/cesium/Source'),
use: { loader: require.resolve('@open-wc/webpack-import-meta-loader') }
}

this can temp resolve the problem

@ebogo1
Copy link
Contributor

ebogo1 commented Oct 14, 2021

Hi all - we're going to close this issue and for now, we recommend that for webpack 4 you use @open-wc/webpack-import-meta-loader as recommended above.

Alternatively, I have had success with using rollup-plugin-modify to find and replace import.meta.url with undefined in the zip.js file during the gulpfile's "buildThirdParty" task. This works because zip.js only uses import.meta.url as the second argument when constructing a URL to the z-worker-pako.js file. It might be possible to avoid this because we get the file path from buildModuleUrl so the second argument in the URL constructor might not be necessary. We're still looking into this approach, but are not sure it should make it into main just yet.

Thanks again to all who reported this and helped out!

@ebogo1 ebogo1 closed this as completed Oct 14, 2021
@puxiao
Copy link
Contributor

puxiao commented Oct 26, 2021

@ebogo1

Good news!

The author of zip.js said:

" I fixed this issue in the version v2.3.12, see https://github.com/gildas-lormeau/zip.js/releases/tag/v2.3.12 "

Fix issues when building zip.js with angular-cli 12.2.x (see #267) and create-react-app (see #262)

@ebogo1
Copy link
Contributor

ebogo1 commented Oct 27, 2021

@puxiao Ah great - thanks for posting about it here! We just merged the update to 2.3.12 to main - if anyone is interested in testing this out let us know how it goes!

@wikiNed
Copy link

wikiNed commented Nov 12, 2021

@puxiao Ah great - thanks for posting about it here! We just merged the update to 2.3.12 to main - if anyone is interested in testing this out let us know how it goes!

I tried version1.87.0,which used zip.js 2.3.12,but it has same problem.

@ebogo1
Copy link
Contributor

ebogo1 commented Nov 12, 2021

@wikiNed Could you please share details of what you're doing to reproduce the issue?

@wikiNed
Copy link

wikiNed commented Nov 15, 2021

@wikiNed Could you please share details of what you're doing to reproduce the issue?

My project:
(1)create-react-app version 4.0.3
(2)use @craco/craco
The following is my package.json:

{
  "name": "react-app",
  "version": "0.1.0",
  "private": true,
  "dependencies": {
    "@craco/craco": "^5.6.2",
    "@hot-loader/react-dom": "16.11.0",
    "@material-ui/core": "^4.8.1",
    "@material-ui/icons": "^4.5.1",
    "@material-ui/lab": "^4.0.0-alpha.37",
    "@testing-library/jest-dom": "^4.2.4",
    "@testing-library/react": "^9.3.2",
    "@testing-library/user-event": "^7.1.2",
    "@turf/turf": "^5.1.6",
    "@types/echarts": "^4.6.4",
    "@types/react-router-dom": "^5.1.3",
    "@types/yaml": "^1.2.0",
    "@umijs/hooks": "^1.6.1",
    "axios": "^0.19.2",
    "cesium": "^1.87.0",
    "cesium-sensors-es6": "^0.2.0",
    "craco-cesium": "^1.2.0",
    "craco-plugin-react-hot-reload": "^0.1.0",
    "crypto-js": "^3.1.9-1",
    "echarts": "^4.8.0",
    "file-replace-loader": "^1.1.0",
    "jpeg-js": "^0.4.2",
    "query-string": "^6.11.0",
    "react": "16.11.0",
    "react-color": "^2.18.1",
    "react-dnd": "^10.0.2",
    "react-dnd-html5-backend": "^10.0.2",
    "react-dom": "16.11.0",
    "react-fastclick": "^3.0.2",
    "react-hot-loader": "^4.12.18",
    "react-router-dom": "^5.1.2",
    "react-scripts": "3.4.0",
    "regenerator-runtime": "^0.13.7",
    "resium": "^1.10.0",
    "typescript": "^3.7.4",
    "yaml": "^1.7.2",
    "yarn": "^1.22.10"
  },
  "scripts": {
    "start": "craco start",
    "build": "craco build",
    "copy": "node copy-cesium-type-file.js",
    "test": "craco test",
    "eject": "react-scripts eject"
  },
  "eslintConfig": {
    "extends": "react-app",
    "rules": {
      "no-var": "warn",
      "no-console": "warn",
      "no-extra-boolean-cast": "warn",
      "no-extra-parens": "warn",
      "no-extra-semi": "warn",
      "no-func-assign": "warn",
      "no-irregular-whitespace": "warn",
      "accessor-pairs": "warn",
      "block-scoped-var": "warn",
      "complexity": [
        "warn",
        46
      ],
      "curly": "warn",
      "eqeqeq": "warn",
      "no-alert": "warn",
      "no-empty-function": "warn",
      "no-useless-return": "warn",
      "require-await": "warn",
      "getter-return": "warn",
      "no-use-before-define": "warn",
      "camelcase": "warn",
      "eol-last": "warn",
      "indent": [
        "warn",
        2
      ],
      "multiline-comment-style": [
        "warn",
        "starred-block"
      ],
      "multiline-ternary": [
        "warn",
        "never"
      ],
      "new-cap": "warn",
      "new-parens": "warn",
      "no-inline-comments": "warn",
      "no-mixed-spaces-and-tabs": "warn"
    }
  },
  "browserslist": {
    "production": [
      ">0.2%",
      "not dead",
      "not op_mini all"
    ],
    "development": [
      "last 1 chrome version",
      "last 1 firefox version",
      "last 1 safari version"
    ]
  },
  "homepage": "/gis1/tswly",
  "devDependencies": {
    "@babel/plugin-syntax-import-meta": "^7.10.4",
    "cpr": "^3.0.1",
    "craco-fast-refresh": "^1.0.5"
  }
}

The following is my craco.config.js

module.exports = {
  webpack: {
    alias: {
      "react-dom": "@hot-loader/react-dom",
    },
  },
  devServer: {
    proxy: {
      [api_prefix]: {
        target: back_end_server,
        changeOrigin: true,
        pathRewrite: {[`^${api_prefix}`]: ''},
      }
    },
    port: dev_server_port,
    contentBasePublicPath: homepage,
  },
  plugins: [
    {plugin: require("craco-plugin-react-hot-reload")},
    {plugin: require("craco-fast-refresh")},
    {
      plugin: require("craco-cesium")({
        loadPartially: true,
      })
    },
    {
      plugin: replace_file({
        replacement_array: [
          [/Ion\.js$/, path.resolve("src/mock/Ion.js")],
        ]
      })
    }]
};

when I run my project,the same problem:

SyntaxError: D:\Code\Sandi-React\SandiGIS\node_modules\cesium\Source\ThirdParty\zip.js: Support for the experimental syntax 'importMeta' isn't currently enabled (6357:59):

  6355 | 
  6356 |        function getWorker(options = {}) {
> 6357 |                return new Worker(new URL(workerData.scripts[0], import.meta.url), options);
       |                                                                        ^
  6358 |        }
  6359 | 
  6360 |        async function initAndSendMessage(message) {

Add @babel/plugin-syntax-import-meta (https://git.io/vbKK6) to the 'plugins' section of your Babel config to enable parsing.

@GimpMaster
Copy link

I've created my own webpack config based loosely off of create react app. I have not yet tried yarn add @open-wc/webpack-import-meta-loader.

But just out of the box Cesium 1.87.1 fails to start with the same error given above by others. I will try adding webpack-import-meta-loader and adjust my test.

@jony89
Copy link
Contributor

jony89 commented Nov 28, 2021

Not sure why this is closed, @open-wc/webpack-import-meta-loader by the author definition is not ideal and assumed to be legacy. either way it can't be an official solution. also security wise, this package is barely used by others.

ATM, we are prevented from upgrading cesium.

@ebogo1

@ebogo1
Copy link
Contributor

ebogo1 commented Nov 29, 2021

@jony89 For now, since this is an issue with the zip.js library we use in CesiumJS, our best workaround is to use the @open-wc/webpack-import-meta-loader if possible. At some point we may revisit this and possibly switch to a new third party zip library if this becomes a bigger blocker, in which case we could reopen this issue.

In the meantime, if anyone finds a better workaround that could work for CesiumJS we'd be happy to take a look at a pull request!

@yonzmeer
Copy link
Contributor

yonzmeer commented Dec 6, 2021

@open-wc/webpack-import-meta-loader does not work in web-workers - as discussed here.

So If anyone has a different loader it will be great.

(Or for the cesium team to use a different zip library)

@an501920078
Copy link

@open-wc/webpack-import-meta-loader 会导致我的vue项目打包有问题
Thread Loader (Worker 0)
Cannot read property 'length' of undefined
实际我并没有写length属性

@puxiao
Copy link
Contributor

puxiao commented Dec 18, 2021

If you use react,the latest version of create-react-app 5.0 uses webpack 5, and this problem will not occur again.

https://github.com/facebook/create-react-app/releases/tag/v5.0.0

@ShareQiu1994
Copy link

at cesium1.89 , vueCli4 + webpack

vue.config.js addLoader after

         {
            test: /.js$/,
            use: { loader: require.resolve('@open-wc/webpack-import-meta-loader') }
          }

development run success, but build error

Syntax Error: Thread Loader (Worker 0)
Cannot read property 'length' of undefined

@ShareQiu1994
Copy link

{ test: /.js$/, include: path.resolve(__dirname, 'node_modules/cesium/Source'), use: { loader: require.resolve('@open-wc/webpack-import-meta-loader') } }

this can temp resolve the problem

at cesium1.89 , vueCli4 + webpack , This method does not work..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
Archived in project
Development

No branches or pull requests