Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: undeclared events dependency breaks vite builds #2110

Closed
weboko opened this issue Oct 2, 2023 · 8 comments · Fixed by #2134
Closed

bug: undeclared events dependency breaks vite builds #2110

weboko opened this issue Oct 2, 2023 · 8 comments · Fixed by #2134
Labels
need/author-input Needs input from the original author

Comments

@weboko
Copy link

weboko commented Oct 2, 2023

  • Version: 0.46.12

Severity:

Medium - A non-essential functionality does not work, performance issues, etc.

Description:

Since connectionManager/utils has events dependency to nodejs package I assume and there is no alias or guard for the browser it creates problems for consumers that are using the lib without polyfills.

In particular, it happens to vite https://vitejs.dev/ which by default does not any kinds of polyfills.

Steps to reproduce the error:

Take any project with vite and install to it libp2p, try building the project and observe the problem.
Additional context can be found in issue on our side - waku-org/js-waku#1564

✓ built in 4.98s
"setMaxListeners" is not exported by "__vite-browser-external", imported by "node_modules/.pnpm/libp2p@0.46.12/node_modules/libp2p/dist/src/connection-manager/utils.js".
file: /Users/okozlov/Status/tmp/draft/waku-objects-playground/node_modules/.pnpm/libp2p@0.46.12/node_modules/libp2p/dist/src/connection-manager/utils.js:1:9
1: import { setMaxListeners } from 'events';
            ^
2: import { logger } from '@libp2p/logger';
3: import { multiaddr } from '@multiformats/multiaddr';
[vite-plugin-sveltekit-compile] "setMaxListeners" is not exported by "__vite-browser-external", imported by "node_modules/.pnpm/libp2p@0.46.12/node_modules/libp2p/dist/src/connection-manager/utils.js".
file: /Users/okozlov/Status/tmp/draft/waku-objects-playground/node_modules/.pnpm/libp2p@0.46.12/node_modules/libp2p/dist/src/connection-manager/utils.js:1:9
1: import { setMaxListeners } from 'events';
            ^
2: import { logger } from '@libp2p/logger';
3: import { multiaddr } from '@multiformats/multiaddr';
✓ built in 9.44s
error during build:
RollupError: "setMaxListeners" is not exported by "__vite-browser-external", imported by "node_modules/.pnpm/libp2p@0.46.12/node_modules/libp2p/dist/src/connection-manager/utils.js".
    at error (file:///Users/okozlov/Status/tmp/draft/waku-objects-playground/node_modules/.pnpm/rollup@3.27.2/node_modules/rollup/dist/es/shared/node-entry.js:2245:30)
    at Module.error (file:///Users/okozlov/Status/tmp/draft/waku-objects-playground/node_modules/.pnpm/rollup@3.27.2/node_modules/rollup/dist/es/shared/node-entry.js:13604:16)
    at Module.traceVariable (file:///Users/okozlov/Status/tmp/draft/waku-objects-playground/node_modules/.pnpm/rollup@3.27.2/node_modules/rollup/dist/es/shared/node-entry.js:14029:29)
    at ModuleScope.findVariable (file:///Users/okozlov/Status/tmp/draft/waku-objects-playground/node_modules/.pnpm/rollup@3.27.2/node_modules/rollup/dist/es/shared/node-entry.js:12547:39)
    at FunctionScope.findVariable (file:///Users/okozlov/Status/tmp/draft/waku-objects-playground/node_modules/.pnpm/rollup@3.27.2/node_modules/rollup/dist/es/shared/node-entry.js:7082:38)
    at ChildScope.findVariable (file:///Users/okozlov/Status/tmp/draft/waku-objects-playground/node_modules/.pnpm/rollup@3.27.2/node_modules/rollup/dist/es/shared/node-entry.js:7082:38)
    at BlockScope.findVariable (file:///Users/okozlov/Status/tmp/draft/waku-objects-playground/node_modules/.pnpm/rollup@3.27.2/node_modules/rollup/dist/es/shared/node-entry.js:7082:38)
    at BlockScope.findVariable (file:///Users/okozlov/Status/tmp/draft/waku-objects-playground/node_modules/.pnpm/rollup@3.27.2/node_modules/rollup/dist/es/shared/node-entry.js:7082:38)
    at TrackingScope.findVariable (file:///Users/okozlov/Status/tmp/draft/waku-objects-playground/node_modules/.pnpm/rollup@3.27.2/node_modules/rollup/dist/es/shared/node-entry.js:7082:38)
    at BlockScope.findVariable (file:///Users/okozlov/Status/tmp/draft/waku-objects-playground/node_modules/.pnpm/rollup@3.27.2/node_modules/rollup/dist/es/shared/node-entry.js:7082:38)
 ELIFECYCLE  Command failed with exit code 1.
@weboko weboko added the need/triage Needs initial labeling and prioritization label Oct 2, 2023
@wemeetagain wemeetagain moved this to 🥞Weekly Candidates/Discuss🎙 in js-libp2p Oct 2, 2023
@maschad
Copy link
Member

maschad commented Oct 5, 2023

I don't think the steps to reproduce this issue are correct given we have multiple examples that use vite with libp2p, such as libp2p in the browser that don't have this problem, do you have another example of this occuring independently of this issue?

Our build tool builds browser bundles separately and so I think this may be related to how the consumer is using rollup's tree shaking, but I can't say definitively since they are also using svelte.

@maschad maschad added need/author-input Needs input from the original author and removed need/triage Needs initial labeling and prioritization labels Oct 5, 2023
@maschad maschad moved this from 🥞Weekly Candidates/Discuss🎙 to 🤨Needs Investigation in js-libp2p Oct 5, 2023
@wemeetagain
Copy link
Member

More plainly the problem is that we're using the nodejs build-in events package, but it isn't listed in our dependencies and its not clear how to handle that package or our usage of it in browser builds. (The answer is to hope that the consumer uses a bundler that magically resolves the events package)

As far as I know, the setMaxListeners is only used in nodejs because nodejs prints a warning if too many event listeners are registered.
It would be nicer to use a library that "works" in browsers that we use here instead of silently using the node library which requires bundler BS.

eg:

// add to `@libp2p/interface/events` ??

// noop if in browser
let setMaxListeners = (n?: number, ...eventTargets: (DOMEventTarget | EventEmitter)[]) => {}

try {
  const {setMaxListeners: s} = await import('node:events')
  setMaxListeners = s
} catch (e) {}

export {setMaxListeners}

then in libp2p

import {setMaxListeners} from '@libp2p/interface/events'

...

setMaxListeners(Infinity, signal)

@maschad
Copy link
Member

maschad commented Oct 5, 2023

More plainly the problem is that we're using the nodejs build-in events package, but it isn't listed in our dependencies and its not clear how to handle that package or our usage of it in browser builds. (The answer is to hope that the consumer uses a bundler that magically resolves the events package)

Normally one wouldn't list a nodejs built-in events package in one's dependencies, regardless, our bundles are generated by esbuild which outputs code for the browser by default so the consumer doesn't need to resolve that package. What's happening here is that there is extra-bundler configuration via Rollup that causing conflation, otherwise this issue would have shown up much earlier in one of our many vite projects.

@achingbrain
Copy link
Member

I think we need to add events as a dep to the modules that use it. It will not be imported in node since it doesn't allow overriding the names of built-in modules.

This problem doesn't occur when we build the browser bundle in the monorepo with esbuild because kad-dht declares events as a dependency so it is hoisted and as such present and resolvable in node_modules.

achingbrain added a commit that referenced this issue Oct 6, 2023
Bundlers do not include node core modules any more so we need to
depend on polyfills in order for downstream consumers to build their
apps successfully.

Fixes #2110
achingbrain added a commit that referenced this issue Oct 6, 2023
Bundlers do not include node core modules any more so we need to
depend on polyfills in order for downstream consumers to build their
apps successfully.

Fixes #2110
@github-project-automation github-project-automation bot moved this from 🤨Needs Investigation to 🎉Done in js-libp2p Oct 6, 2023
@maschad
Copy link
Member

maschad commented Oct 6, 2023

This problem doesn't occur when we build the browser bundle in the monorepo with esbuild because kad-dht declares events as a dependency so it is hoisted and as such present and resolvable in node_modules.

The examples are self contained, and as a sanity check I ran the webRTC example independently of the monorepo with the latest libp2p and this issue still doesn't occur, and kad-dht isn't a dependency of this example, so I still believe this is a bundler issue.

That being said, I can see the benefit of what @wemeetagain suggests in moving towards handling native ESM support on our side such that we wouldn't need to rely on bundlers with the ultimate goal of having greater portability across environments, but that would require a bit more of a discussion.

@achingbrain
Copy link
Member

achingbrain commented Oct 6, 2023

kad-dht isn't a dependency of this example, so I still believe this is a bundler issue.

Just do npm ls events to see which dependency is pulling it in.

@maschad
Copy link
Member

maschad commented Oct 6, 2023

kad-dht isn't a dependency of this example, so I still believe this is a bundler issue.

Just do npm ls events to see which dependency is pulling it in.

It's not being pulled in

js-libp2p-webrtc-private-to-private@1.0.0 /Users/horizon/Desktop/work/vite-project
└── (empty)

@achingbrain
Copy link
Member

With vite I see a warning in the browser console:

browser-external:events:9 Module "events" has been externalized for browser compatibility. Cannot access "events.setMaxListeners" in client code. See http://vitejs.dev/guide/troubleshooting.html#module-externalized-for-browser-compatibility for more details.

so I think they're doing a more gentle version of webpack 5's rug-pull.

This was referenced Jan 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
need/author-input Needs input from the original author
Projects
Archived in project
4 participants