Fix backend webpack output and watching #12902

msujew · 2023-09-13T10:07:13Z

What it does

Apparently, using tapAsync to hook into the webpack generator was incorrect. I should've use tapPromise instead...

As the output is now correctly printed, I was also able to identify what the issue with the watch build was. The intermittently created files were deleted too fast, resulting in a failing watch build. They are now alive longer on the disk.

How to test

Run the yarn browser/electron build script.
Webpack should print output (i.e. what has been generated/bundled)
Run the yarn browser/electron watch script.
Perform a change in the server part of the app (i.e. add a print statement somewhere)
Start the app and confirm that the changes are active.

Review checklist

As an author, I have thoroughly tested my changes and carefully followed the review guidelines

Reminder for reviewers

As a reviewer, I agree to behave in accordance with the review guidelines

kittaakos

a3b24f3 works for me. Thank you!

yarn browser build

yarn run v1.22.19
$ yarn -s --cwd examples/browser build
$ theia rebuild:browser --cacheRoot ../..
Reverted "node-pty"
Reverted "keytar"
Reverted "native-keymap"
Reverted "drivelist"
Reverted "nsfw"
Reverted "find-git-repositories"
Reverted "cpu-features"
Reverted "ssh2"
assets by status 1.93 MiB [cached] 16 assets
assets by path *.js 32.7 MiB
  assets by chunk 12.9 MiB (id hint: vendors)
    asset vendors-node_modules_theia_monaco-editor-core_esm_vs_editor_editor_main_js.js 8.46 MiB [compared for emit] (id hint: vendors) 2 related assets
    asset vendors-node_modules_highlight_js_lib_index_js-node_modules_markdown-it-anchor_index_js.js 1.36 MiB [compared for emit] (id hint: vendors) 2 related assets
    + 19 assets
  + 121 assets
assets by path ../webview/pre/ 38.8 KiB
  assets by path ../webview/pre/*.js 38.1 KiB 3 assets
  assets by path ../webview/pre/*.html 726 bytes
    asset ../webview/pre/index.html 480 bytes [compared for emit] [from: ../../packages/plugin-ext/src/main/browser/webview/pre/index.html] [copied] 1 related asset
    asset ../webview/pre/fake.html 246 bytes [compared for emit] [from: ../../packages/plugin-ext/src/main/browser/webview/pre/fake.html] [copied] 1 related asset
asset context/plugin-vscode-init-fe.js 1.52 KiB [compared for emit] [from: ../../packages/plugin-ext-vscode/lib/node/context/plugin-vscode-init-fe.js] [copied] 1 related asset
asset secondary-window.html 869 bytes [compared for emit] [from: src-gen/frontend/secondary-window.html] [copied] 1 related asset
runtime modules 13.7 KiB 26 modules
orphan modules 14.9 KiB [orphan] 7 modules
javascript modules 24.9 MiB
  modules by path ../../ 24.8 MiB 2919 modules
  modules by path ../api-samples/ 92.9 KiB 21 modules
  + 6 modules
json modules 1.28 MiB 70 modules
asset modules 15.6 KiB (javascript) 1.93 MiB (asset)
  modules by path ../../ 11.4 KiB (javascript) 1.91 MiB (asset) 20 modules
  modules by path ../api-samples/ 740 bytes (javascript) 27.2 KiB (asset) 2 modules
  modules by mime type image/svg+xml 3.17 KiB 2 modules
  + 1 module
  
webpack 5.88.1 compiled successfully in 34073 ms

assets by status 1.45 MiB [cached] 13 assets
assets by status 17.9 MiB [compared for emit]
  asset secondary-window.js 17.5 MiB [compared for emit] (name: secondary-window) 1 related asset
  asset secondary-window.css 406 KiB [compared for emit] (name: secondary-window) 1 related asset
Entrypoint secondary-window 17.9 MiB (19.5 MiB) = secondary-window.css 406 KiB secondary-window.js 17.5 MiB 15 auxiliary assets
runtime modules 79.8 KiB 359 modules
orphan modules 952 KiB (javascript) 1.45 MiB (asset) [orphan] 143 modules
javascript modules 14.8 MiB
  modules by path ../../node_modules/ 12.2 MiB 1126 modules
  modules by path ../../packages/ 2.6 MiB 372 modules
  modules by path ../../dev-packages/ 13.2 KiB 5 modules
  + 3 modules
css modules 358 KiB
  modules by path ../../node_modules/ 251 KiB 89 modules
  modules by path ../../packages/ 107 KiB 28 modules
json modules 1.25 MiB
  optional modules 154 KiB [optional] 47 modules
  modules by path ../../packages/core/node_modules/iconv-lite/ 86.7 KiB 8 modules
  modules by path ../../node_modules/ 31.6 KiB 3 modules
  ../../packages/core/src/common/i18n/nls.metadata.json 1000 KiB [built] [code generated]
  
webpack 5.88.1 compiled successfully in 15175 ms

assets by path *.js 11 MiB
  assets by status 6.52 MiB [compared for emit]
    assets by chunk 3.79 MiB (id hint: vendors) 8 assets
    + 12 assets
  assets by status 4.48 MiB [emitted]
    asset vendors-node_modules_drivelist_build_Release_drivelist_node-node_modules_stroncium_procfs_lib-cf49d8.js 2.9 MiB [emitted] (id hint: vendors) 1 related asset
    asset main.js 1.58 MiB [emitted] (name: main) 1 related asset
assets by path native/*.node 626 KiB
  asset native/nsfw.node 182 KiB [compared for emit] (auxiliary id hint: vendors)
  asset native/keytar.node 106 KiB [compared for emit] (auxiliary id hint: vendors)
  asset native/sshcrypto.node 102 KiB [compared for emit] (auxiliary id hint: vendors)
  asset native/drivelist.node 72.4 KiB [compared for emit] (auxiliary id hint: vendors)
  asset native/cpufeatures.node 66 KiB [compared for emit] (auxiliary id hint: vendors)
  + 2 assets
asset worker/conoutSocketWorker.js 3.33 KiB [compared for emit] (name: worker/conoutSocketWorker) 1 related asset
runtime modules 27.5 KiB 65 modules
modules by path ../../node_modules/ 5.68 MiB
  javascript modules 5.3 MiB 998 modules
  json modules 394 KiB 17 modules
modules by path ../../packages/ 4 MiB
  javascript modules 2.66 MiB 505 modules
  json modules 1.34 MiB 23 modules
modules by path ../../dev-packages/ 77.4 KiB 23 modules
modules by path ../api-samples/lib/ 17.6 KiB
  modules by path ../api-samples/lib/node/*.js 15.1 KiB 4 modules
  modules by path ../api-samples/lib/common/vsx/*.js 2.5 KiB 2 modules
modules by path ./ 5.52 KiB
  modules by path ./src-gen/backend/*.js 5.19 KiB 2 modules
  modules by path ./lib/backend/native-webpack-plugin/*.js 341 bytes 2 modules
+ 33 modules
webpack 5.88.1 compiled successfully in 19105 ms

✨  Done in 44.13s.

yarn electron rebuild && yarn electron build

yarn run v1.22.19
$ yarn -s --cwd examples/electron rebuild
Processed "node-pty"
Processed "native-keymap"
Processed "keytar"
Processed "drivelist"
Processed "find-git-repositories"
Processed "nsfw"
Processed "cpu-features"
Processed "ssh2"
⠙ Building module: cpu-features, Completed: 1  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_aarch64_linux_or_android.o
⠴ Building module: cpu-features, Completed: 1  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_arm_linux_or_android.o
⠋ Building module: cpu-features, Completed: 1  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_mips_linux_or_android.o
⠸ Building module: cpu-features, Completed: 1  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_ppc_linux.o
⠧ Building module: cpu-features, Completed: 1  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_x86_freebsd.o
⠙ Building module: cpu-features, Completed: 1  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_x86_linux_or_android.o
⠴ Building module: cpu-features, Completed: 1  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_x86_macos.o
⠼ Building module: cpu-features, Completed: 1  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_x86_windows.o
⠇ Building module: cpu-features, Completed: 1  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/filesystem.o
⠸ Building module: cpu-features, Completed: 1  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/stack_line_reader.o
⠧ Building module: cpu-features, Completed: 1  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/string_view.o
⠸ Building module: cpu-features, Completed: 1  LIBTOOL-STATIC Release/cpu_features.a
⠧ Building module: cpu-features, Completed: 1  CXX(target) Release/obj.target/cpufeatures/src/binding.o
⠼ Building module: cpu-features, Completed: 1  SOLINK_MODULE(target) Release/cpufeatures.node
⠦ Building module: drivelist, Completed: 2  CXX(target) Release/obj.target/drivelist/src/drivelist.o
⠧ Building module: drivelist, Completed: 2  CXX(target) Release/obj.target/drivelist/src/device-descriptor.o
⠇ Building module: drivelist, Completed: 2  CXX(target) Release/obj.target/drivelist/src/darwin/list.o
⠇ Building module: drivelist, Completed: 2  CXX(target) Release/obj.target/drivelist/src/darwin/REDiskList.o
⠧ Building module: drivelist, Completed: 2../src/darwin/REDiskList.m:29:13: warning: 'NSArray' may not respond to 'sortUsingSelector:'
    [_disks sortUsingSelector:@selector(localizedCaseInsensitiveCompare:)];
     ~~~~~~ ^
1 warning generated.
⠏ Building module: drivelist, Completed: 2  SOLINK_MODULE(target) Release/drivelist.node
⠙ Building module: find-git-repositories, Completed: 3  CC(target) Release/obj.target/openpa/openpa/src/opa_primitives.o
⠴ Building module: find-git-repositories, Completed: 3  CC(target) Release/obj.target/openpa/openpa/src/opa_queue.o
⠏ Building module: find-git-repositories, Completed: 3  LIBTOOL-STATIC Release/openpa.a
⠹ Building module: find-git-repositories, Completed: 3  CXX(target) Release/obj.target/findGitRepos/src/FindGitRepos.o
⠴ Building module: find-git-repositories, Completed: 3../src/FindGitRepos.cpp:270:28: warning: 'Call' is deprecated [-Wdeprecated-declarations]
  baton->progressCallback->Call(1, argv);
                           ^
../../nan/nan.h:1745:3: note: 'Call' has been explicitly marked deprecated here
  NAN_DEPRECATED inline v8::Local<v8::Value>
  ^
../../nan/nan.h:110:40: note: expanded from macro 'NAN_DEPRECATED'
# define NAN_DEPRECATED __attribute__((deprecated))
                                       ^
../src/FindGitRepos.cpp:294:13: warning: 'Call' is deprecated [-Wdeprecated-declarations]
  callback->Call(1, argv);
            ^
../../nan/nan.h:1745:3: note: 'Call' has been explicitly marked deprecated here
  NAN_DEPRECATED inline v8::Local<v8::Value>
  ^
../../nan/nan.h:110:40: note: expanded from macro 'NAN_DEPRECATED'
# define NAN_DEPRECATED __attribute__((deprecated))
                                       ^
⠙ Building module: find-git-repositories, Completed: 32 warnings generated.
⠸ Building module: find-git-repositories, Completed: 3  CXX(target) Release/obj.target/findGitRepos/src/Queue.o
⠼ Building module: find-git-repositories, Completed: 3  SOLINK_MODULE(target) Release/findGitRepos.node
⠦ Building module: keytar, Completed: 4  CXX(target) Release/obj.target/keytar/src/async.o
⠸ Building module: keytar, Completed: 4  CXX(target) Release/obj.target/keytar/src/main.o
⠇ Building module: keytar, Completed: 4  CXX(target) Release/obj.target/keytar/src/keytar_mac.o
⠼ Building module: keytar, Completed: 4  SOLINK_MODULE(target) Release/keytar.node
⠦ Building module: native-keymap, Completed: 5  CXX(target) Release/obj.target/keymapping/src/string_conversion.o
⠏ Building module: native-keymap, Completed: 5  CXX(target) Release/obj.target/keymapping/src/keymapping.o
⠸ Building module: native-keymap, Completed: 5  CXX(target) Release/obj.target/keymapping/src/keyboard_mac.o
⠙ Building module: native-keymap, Completed: 5  SOLINK_MODULE(target) Release/keymapping.node
⠴ Building module: node-pty, Completed: 6  CXX(target) Release/obj.target/pty/src/unix/pty.o
⠦ Building module: node-pty, Completed: 6  SOLINK_MODULE(target) Release/pty.node
⠇ Building module: node-pty, Completed: 6  CXX(target) Release/obj.target/spawn-helper/src/unix/spawn-helper.o
⠏ Building module: node-pty, Completed: 6  LINK(target) Release/spawn-helper
⠏ Building module: nsfw, Completed: 7  CXX(target) Release/obj.target/nsfw/src/NSFW.o
⠧ Building module: nsfw, Completed: 7  CXX(target) Release/obj.target/nsfw/src/Queue.o
⠴ Building module: nsfw, Completed: 7  CXX(target) Release/obj.target/nsfw/src/NativeInterface.o
⠸ Building module: nsfw, Completed: 7  CXX(target) Release/obj.target/nsfw/src/osx/RunLoop.o
⠋ Building module: nsfw, Completed: 7  CXX(target) Release/obj.target/nsfw/src/osx/FSEventsService.o
⠼ Building module: nsfw, Completed: 7  SOLINK_MODULE(target) Release/nsfw.node
✔ Rebuild Complete
✨  Done in 52.19s.
yarn run v1.22.19
$ yarn -s --cwd examples/electron build
$ theia rebuild:electron --cacheRoot ../..
native node modules are already rebuilt for electron
Downloaded ffmpeg shared library { version: "23.3.10", dist: "/Users/a.kitta/dev/git/theia/node_modules/electron/dist" }.
Successfully replaced "/Users/a.kitta/dev/git/theia/node_modules/electron/dist/Electron.app/Contents/Frameworks/Electron Framework.framework/Libraries/libffmpeg.dylib".
"/Users/a.kitta/dev/git/theia/node_modules/electron/dist/Electron.app/Contents/Frameworks/Electron Framework.framework/Libraries/libffmpeg.dylib" does not contain proprietary codecs (16 found).
assets by status 1.93 MiB [cached] 16 assets
assets by path *.js 32.4 MiB
  assets by chunk 12.8 MiB (id hint: vendors)
    asset vendors-node_modules_theia_monaco-editor-core_esm_vs_editor_editor_main_js.js 8.38 MiB [compared for emit] (id hint: vendors) 2 related assets
    asset vendors-node_modules_highlight_js_lib_index_js-node_modules_markdown-it-anchor_index_js.js 1.34 MiB [compared for emit] (id hint: vendors) 2 related assets
    + 19 assets
  + 124 assets
assets by path ../webview/pre/ 38.8 KiB
  assets by path ../webview/pre/*.js 38.1 KiB 3 assets
  assets by path ../webview/pre/*.html 726 bytes
    asset ../webview/pre/index.html 480 bytes [compared for emit] [from: ../../packages/plugin-ext/src/main/browser/webview/pre/index.html] [copied] 1 related asset
    asset ../webview/pre/fake.html 246 bytes [compared for emit] [from: ../../packages/plugin-ext/src/main/browser/webview/pre/fake.html] [copied] 1 related asset
asset context/plugin-vscode-init-fe.js 1.52 KiB [compared for emit] [from: ../../packages/plugin-ext-vscode/lib/node/context/plugin-vscode-init-fe.js] [copied] 1 related asset
asset secondary-window.html 869 bytes [compared for emit] [from: src-gen/frontend/secondary-window.html] [copied] 1 related asset
runtime modules 13.7 KiB 26 modules
orphan modules 14.1 KiB [orphan] 7 modules
javascript modules 24.6 MiB
  modules by path ../../ 24.5 MiB 2940 modules
  modules by path ../api-samples/ 100 KiB 24 modules
  + 6 modules
json modules 1.28 MiB 70 modules
asset modules 15.6 KiB (javascript) 1.93 MiB (asset)
  modules by path ../../ 11.4 KiB (javascript) 1.91 MiB (asset) 20 modules
  modules by path ../api-samples/ 740 bytes (javascript) 27.2 KiB (asset) 2 modules
  modules by mime type image/svg+xml 3.17 KiB 2 modules
  + 1 module
  
webpack 5.88.1 compiled successfully in 33893 ms

assets by status 1.45 MiB [cached] 13 assets
assets by status 17.9 MiB [compared for emit]
  asset secondary-window.js 17.5 MiB [compared for emit] (name: secondary-window) 1 related asset
  asset secondary-window.css 406 KiB [compared for emit] (name: secondary-window) 1 related asset
Entrypoint secondary-window 17.9 MiB (19.5 MiB) = secondary-window.css 406 KiB secondary-window.js 17.5 MiB 15 auxiliary assets
runtime modules 79.8 KiB 359 modules
orphan modules 952 KiB (javascript) 1.45 MiB (asset) [orphan] 143 modules
javascript modules 14.8 MiB
  modules by path ../../node_modules/ 12.2 MiB 1126 modules
  modules by path ../../packages/ 2.6 MiB 372 modules
  modules by path ../../dev-packages/ 13.2 KiB 5 modules
  + 3 modules
css modules 358 KiB
  modules by path ../../node_modules/ 251 KiB 89 modules
  modules by path ../../packages/ 107 KiB 28 modules
json modules 1.25 MiB
  optional modules 154 KiB [optional] 47 modules
  modules by path ../../packages/core/node_modules/iconv-lite/ 86.7 KiB 8 modules
  modules by path ../../node_modules/ 31.6 KiB 3 modules
  ../../packages/core/src/common/i18n/nls.metadata.json 1000 KiB [built] [code generated]
  
webpack 5.88.1 compiled successfully in 15153 ms

asset preload.js 53 KiB [compared for emit] (name: preload) 1 related asset
modules by path ../../packages/core/ 43.7 KiB
  modules by path ../../packages/core/lib/common/*.js 32.5 KiB
    ../../packages/core/lib/common/disposable.js 4.15 KiB [built] [code generated]
    ../../packages/core/lib/common/event.js 12.6 KiB [built] [code generated]
    ../../packages/core/lib/common/types.js 4.18 KiB [built] [code generated]
    + 3 modules
  ../../packages/core/lib/electron-browser/preload.js 7.37 KiB [built] [code generated]
  ../../packages/core/lib/electron-common/electron-api.js 3.79 KiB [built] [code generated]
  ../../packages/core/electron-shared/electron/index.js 61 bytes [built] [code generated]
modules by path ../../packages/filesystem/lib/ 2.64 KiB
  ../../packages/filesystem/lib/electron-browser/preload.js 1.56 KiB [built] [code generated]
  ../../packages/filesystem/lib/electron-common/electron-api.js 1.08 KiB [built] [code generated]
./src-gen/frontend/preload.js 146 bytes [built] [code generated]
external "electron" 42 bytes [built] [code generated]
../../packages/electron/shared/electron/index.js 38 bytes [built] [code generated]
  
webpack 5.88.1 compiled successfully in 4046 ms

assets by path *.js 11.6 MiB
  assets by status 7.56 MiB [compared for emit]
    assets by chunk 4.67 MiB (id hint: vendors) 11 assets
    + 14 assets
  assets by status 4.04 MiB [emitted]
    asset vendors-node_modules_drivelist_build_Release_drivelist_node-node_modules_stroncium_procfs_lib-f88532.js 2.45 MiB [emitted] (id hint: vendors) 1 related asset
    asset main.js 1.59 MiB [emitted] (name: main) 1 related asset
assets by path native/*.node 733 KiB
  asset native/nsfw.node 182 KiB [emitted] [compared for emit] (auxiliary id hint: vendors)
  asset native/keytar.node 106 KiB [emitted] [compared for emit] (auxiliary id hint: vendors)
  asset native/sshcrypto.node 102 KiB [compared for emit] (auxiliary id hint: vendors)
  asset native/drivelist.node 88.7 KiB [emitted] [compared for emit] (auxiliary id hint: vendors)
  asset native/cpufeatures.node 82.3 KiB [emitted] [compared for emit] (auxiliary id hint: vendors)
  + 3 assets
asset worker/conoutSocketWorker.js 3.33 KiB [compared for emit] (name: worker/conoutSocketWorker) 1 related asset
runtime modules 32.3 KiB 76 modules
modules by path ../../node_modules/ 6.05 MiB
  javascript modules 5.67 MiB 1114 modules
  json modules 397 KiB 19 modules
modules by path ../../packages/ 4.11 MiB
  javascript modules 2.77 MiB 535 modules
  json modules 1.34 MiB 23 modules
modules by path ../../dev-packages/ 77.4 KiB 23 modules
modules by path ../api-samples/lib/ 24.3 KiB 9 modules
modules by path ./ 9.07 KiB
  modules by path ./src-gen/backend/*.js 8.74 KiB 3 modules
  modules by path ./lib/backend/native-webpack-plugin/*.js 341 bytes 2 modules
+ 34 modules

WARNING in ../../node_modules/shell-env/node_modules/cross-spawn/index.js 32:26-47
Module not found: Error: Can't resolve 'spawn-sync' in '/Users/a.kitta/dev/git/theia/node_modules/shell-env/node_modules/cross-spawn'
 @ ../../node_modules/shell-env/node_modules/execa/index.js 4:19-41
 @ ../../node_modules/shell-env/index.js 2:14-30
 @ ../../node_modules/shell-path/index.js 2:17-37
 @ ../../node_modules/fix-path/index.js 2:18-39
 @ ./src-gen/backend/electron-main.js 9:0-19

1 warning has detailed information that is not shown.
Use 'stats.errorDetails: true' resp. '--stats-error-details' to show it.

webpack 5.88.1 compiled with 1 warning in 19878 ms

✨  Done in 41.55s.

Watching with yarn --cwd ./packages/core watch + yarn electron watch also works locally. I can modify both the electron and backend main modules during the watch mode, and the app picks it up after a restart.

I noticed this when watching the electron example app:

[bundle] WARNING in ../../node_modules/shell-env/node_modules/cross-spawn/index.js 32:26-47
[bundle] Module not found: Error: Can't resolve 'spawn-sync' in '/Users/a.kitta/dev/git/theia/node_modules/shell-env/node_modules/cross-spawn'

JonasHelming · 2023-10-10T13:00:28Z

@msujew Do you need a second review for this?

msujew · 2023-10-10T13:42:13Z

@JonasHelming I do. The current approving reviewer doesn't have write access to the repo, so I can't merge the PR right now.

martin-fleck-at

I also tested the changes and everything is working as expected, thank you very much @msujew!

Fix backend webpack output and watching

b9dff28

msujew added the bundle Related to webpack bundling label Sep 13, 2023

msujew mentioned this pull request Sep 13, 2023

Webpack does not generate feedback when the backend component is bundled #12899

Closed

Fix recursive mkdir bug

a3b24f3

kittaakos approved these changes Sep 13, 2023

View reviewed changes

Remove more warnings

4f9b7a9

msujew requested a review from vince-fugnitto September 25, 2023 10:51

martin-fleck-at approved these changes Oct 24, 2023

View reviewed changes

msujew merged commit a8c2a67 into master Oct 24, 2023
12 checks passed

msujew deleted the msujew/fix-backend-bundler-print branch October 24, 2023 10:27

github-actions bot added this to the 1.43.0 milestone Oct 24, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix backend webpack output and watching #12902

Fix backend webpack output and watching #12902

msujew commented Sep 13, 2023

kittaakos left a comment

JonasHelming commented Oct 10, 2023

msujew commented Oct 10, 2023

martin-fleck-at left a comment

Fix backend webpack output and watching #12902

Fix backend webpack output and watching #12902

Conversation

msujew commented Sep 13, 2023

What it does

How to test

Review checklist

Reminder for reviewers

kittaakos left a comment

Choose a reason for hiding this comment

JonasHelming commented Oct 10, 2023

msujew commented Oct 10, 2023

martin-fleck-at left a comment

Choose a reason for hiding this comment