MaxListenersExceededWarning in more complex dapps

[wbt]

Issue

In a more complex dapp with several smart contracts, truffle-contract v4.0.x produces a MaxListenersExceededWarning, reporting detection of a possible EventEmitter memory leak.
It appears the warning comes as result of a call to setProvider() on the TruffleContract object.

Steps to Reproduce

Note: Not all of these are strictly necessary to reproduce the issue, but I believe they help with clarity and explanation and confidence that other factors are not affecting the outcomes.

0. npm uninstall truffle -g.
1. Complete the Pet Shop tutorial as described, up through npm run dev. This is done with truffle 5.0.13 (now the latest) being installed in the first step. You can skip most of the MetaMask setup if you’ve previously done it.
2. Replicate the Adoption contract several times.
3. In each, change the filename and contract name to Adoption1, Adoption2, Adoption3, Adoption4,... going up through at least Adoption12. This simulates a more complex dapp with several smart contracts.
4. In migrations/2_deploy_contracts.js, add declaration lines at the top and deploy lines inside the function, for each of those contracts, following the example.
5. Open src/js/app.js for editing.
6. Add the parameter contractName to the function initContract. Log it as a new first line in the function execution.
7. In the call to $.getJSON, replace 'Adoption.json' with contractName+'.json'
8. Replace the line return App.initContract(); with:

      for (var i=1; i<=12; i++) { 
          App.initContract('Adoption'+i);
      }

9. In initContract, comment out the call to markAdopted().
10. Back in the console window, halt the server (CTRL+C).
11. truffle compile
12. truffle migrate --reset
13. npm run dev again.
14. Visit localhost:3000 in a web browser and open the console (CTRL+SHIFT+J).
15. Observe the log lines from step 6 showing the contract names being initiated.
16. Back in the terminal window, halt the server and run these commands, comparing against expected results (command syntax may vary slightly, depending on your interpreter):

>npm list truffle-contract
`-- (empty)
>npm list web3
`-- (empty)
> npm install truffle-contract
...
+ truffle-contract@4.0.12
...
>npm list truffle-contract
`-- truffle-contract@4.0.12
>npm list web3
`-- truffle-contract@4.0.12
  +-- truffle-interface-adapter@0.1.2
  | `-- web3@1.0.0-beta.37  deduped
  `-- web3@1.0.0-beta.37
>ls src/js
[format may differ]
app.js
bootstrap.min.js
truffle-contract.js
web3.min.js
>rm src/js/truffle-contract.js
>rm src/js/web3.min.js
>ls src/js
app.js
bootstrap.min.js
>cp node_modules/truffle-contract/dist/truffle-contract.js src/js/truffle-contract.js
>wget https://raw.githubusercontent.com/ethereum/web3.js/v1.0.0-beta.37/dist/web3.min.js -OutFile src/js/web3.min.js
>ls src/js
app.js
bootstrap.min.js
truffle-contract.js
web3.min.js
>npm run dev

17. Now go back and refresh the browser page.
    After the previously observed console outputs, there is a new warning:

inpage.js:1 MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 101 data listeners added. Use emitter.setMaxListeners() to increase limit
at f (chrome-extension://nkbihfbeogaeaoehlefnkodbefgpgknn/inpage.js:1:153644)
at Proxy.a.addListener (chrome-extension://nkbihfbeogaeaoehlefnkodbefgpgknn/inpage.js:1:155911)
at RequestManager.setProvider (http://localhost:3000/js/truffle-contract.js:43850:23)
at Eth.pkg.setProvider (http://localhost:3000/js/truffle-contract.js:44617:37)
at Eth.setProvider (http://localhost:3000/js/truffle-contract.js:49142:31)
at Web3Shim.Web3.setProvider (http://localhost:3000/js/truffle-contract.js:51002:18)
at Function.setProvider (http://localhost:3000/js/truffle-contract.js:58704:17)
at Object.success (http://localhost:3000/js/app.js:62:36)
at i (https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js:2:27449)
at Object.fireWith [as resolveWith] (https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js:2:28213)

18. Back in the terminal, halt the server.
19. npm uninstall truffle-contract
20. npm install truffle-contract@4.0.11
21. npm list web3 (optional, to verify web3 version remains 1.0.0-beta.37)
22. cp node_modules/truffle-contract/dist/truffle-contract.js src/js/truffle-contract.js
23. npm run dev
24. Repeat steps 17-23, reducing the version of truffle-contract by one each time. The same basic behavior is observed for 4.0.11, 4.0.10, 4.0.9, 4.0.8, 4.0.7, 4.0.6, 4.0.5, 4.0.4, and 4.0.3.
    In 4.0.2, 4.0.1, and 4.0.0, web3 goes to beta.52, but there is not a minified version of beta.38 or later, and Trufflesuite’s fork of web3 doesn’t have beta.37 or after. The specification is "^1.0.0-beta.37", so sticking with 37 seems best.
    In 4.0.0 and 4.0.1 there are various other errors instead, starting with “Uncaught TypeError: EthersAbi is not a constructor” (Issue #1619).
25. npm show truffle-contract@3.* version shows 3.0.{0-7} as options.
26. npm uninstall truffle-contract.
27. npm install truffle-contract@3.0.7.
28. cp node_modules/truffle-contract/dist/truffle-contract.js src/js/truffle-contract.js
29. npm list web3 shows 0.20.6.
30. wget https://raw.githubusercontent.com/ethereum/web3.js/v0.20.6/dist/web3.min.js -OutFile src/js/web3.min.js
31. npm run dev and now we’re back to step 15 without warning messages.

Expected Behavior

No MaxListenersExceededWarning and no memory leak. Clarity in the warning message about whether subsequent listeners are getting created/attached properly or what other consequences might be, since a programmer adapting prior code for compatibility with newer versions of Truffle is not explicitly using listeners, and may find this warning confusing.

Environment

• Operating System: Windows 10 Pro
• Ethereum client: Ganache
• Truffle version (truffle version): 5.0.13
• node version (node --version): 9.3.0
• npm version (npm --version): 6.9.0
• browser: Chrome 72.0.3626.121 (Official Build) (64-bit)

Additional information

Addressing issue 1938 could help make debugging this and similar future issues easier.

[gnidan]

Hm, it's tough to compare truffle-contract 4.x and 3.x, but it's useful information to know the problem goes back far and isn't something new we've broken.

Not sure what might be causing it. Perhaps something with setting the provider. It looks like web3 beta.53 is getting released soon, fixing a number of breaking changes for Truffle, so we might be able to upgrade finally! I might suggest waiting for then to see if this problem sorts itself out, assuming the beta.53 upgrade is painless, unless anyone has ideas on what the underlying culprit here might be.

Thanks for going through the effort to explain @wbt; your instructions will be very useful in validating a fix. It's hard to reliably reproduce this sort of thing.

[wbt]

assuming the beta.53 upgrade is painless

That is a big assumption and I am skeptical that it's warranted.

Issues like the one logged here are blocking us from upgrading to truffle-contract 4.x and the new web3. Every time we try, it's about a week of headaches and extra work, finding there are still serious issues in the underlying software that wasn't actually ready for release, and then reversion.

However, Beta.53 is now released, so the speculation of several hours ago is now a testable hypothesis. If Truffle upgrading to that can fix the issue, great!

Thanks for going through the effort to explain @wbt; your instructions will be very useful in validating a fix. It's hard to reliably reproduce this sort of thing.

Yes, I hope the instructions are helpful. That was multiple days of work, at the tail end of the most recent upgrade attempt.

[gnidan]

That is a big assumption and I am skeptical that it's warranted.

I'm more hopeful than previously because there has been a lot of active conversation over on web3's side lately. We've expressed the breaking changes we've identified since beta.37.

now a testable hypothesis

Here we go. Bet you ten finney that it fails the first time 😏 #1968

[wbt]

Here we go. Bet you ten finney that it fails the first time 😏 #1968

No way would I take the other side of that, even if I'd noticed before the test failure!

[gnidan]

What's going on here? Web3 beta.53 is clearly a no-go :)

Any ideas @wbt? Thanks!

[wbt]

I have no idea currently on what's going on to cause this bug, and am wanting to hold off on upgrading to any relatively recent version until it's fixed.

I agree that it needs investigation, but invested quite a bit into making a good report including carefully building the steps to reproduce noted above, which don't take that long to walk through. I'd be quite disappointed if this were closed especially on a needs-reproduced basis. To others who try, please actively write a comment reporting your +1 below, until needs-reproduced is removed.

[adrianmcli]

Does this have anything to do with this? trufflesuite/ganache#267 (comment)

[wbt]

@adrianmcli It might; nice find!

The stack trace differs after a few lines, and that issue reports that Truffle suppressed (all?) upstream warnings in this PR on July 6, 2018. That suppression appears to be in Truffle v4.0.14 and later (it's still there) with a note that it was intended to be temporary until Web3 issue 1648 was resolved; that issue was closed unresolved with redirection to the open Ganache issue you linked to.

The above was observed using MetaMask in a browser. I'm not an expert on how MetaMask selects a Provider, but I think it's an HTTP provider which happens to be pointing at an RPC interface on a Ganache node, rather than a Ganache-specific Provider being used.

One way to see if it's specific to Ganache or not would be to run the above Steps to Reproduce as described (to verify that the issue is observed), and then rerun the migration and access steps on a different network (e.g. local geth or a testnet). If the issue is still observed anywhere else, it's definitely not Ganache-specific.

[adrianmcli]

@wbt yes, if you look at what @seesemichaelj said in that issue, he mentions that the real problem is in Web3.js because of how they attach listeners.

Do check out this as well: web3/web3.js#1648 (comment)

I'm pretty confident it's a problem with Web3.js.

[wbt]

@adrianmcli I did read that and tentatively agree, but the Web3 issue was CLOSED WONTFIX by someone who's much more expert than I about that codebase. It remained closed even after comments by Levino et al. reporting that this was an issue with Web3 observed in the browser. So I suspect I might be missing something, or at least I'd want to spend more time diving into the codebase and getting more specific details before publicly asserting that @nivida is wrong on this topic where he is expert and I am not.

Also, even if it is definitely an issue with Web3, I'm not optimistic that it will be fixed there, based on the status of the tracked issue.

[adrianmcli]

@wbt please see my latest comment there: web3/web3.js#1648 (comment)

Since I am confident this is a Web3 problem, I will continue the investigation on that issue (even if it's closed). I can petition @nivida to open it up again.

[eggplantzzz]

@wbt Is this issue still relevant? Let me know, thanks!

[wbt]

@eggplantzzz I believe so. I don't see any particular reason to believe that it has been fixed, nor are there any plans to address a related issue from the Web3 side (web3#1648 remains CLOSED WONTFIX), and there is still a use case for having a dapp involving more than a few smart contracts.

[eggplantzzz]

@wbt Ok sounds good! I'll look into it and thanks for the detailed report!

[adrianmcli]

@eggplantzzz this is an on-going problem. I believe it has something to do with the organizational structure of Web3.js, and its heavy reliance on OOP objects + EventEmitters (i.e. assigning listeners unnecessarily).

[haltman-at]

@michaelsbradleyjr, do you know anything about this?

[michaelsbradleyjr]

@haltman-at I don't. But looking at the comment history of web3/web3.js#1648 it seems that issue may need to be reopened or a new one created.

[adrianmcli]

@michaelsbradleyjr @haltman-at Let's try to petition @nivida to re-open web3/web3.js#1648.

[eggplantzzz]

We are marking this as blocked by web3 stuff. :)

[fainashalts]

Hi @wbt is this issue still happening? We think it is probably resolved but wanted to check in. Thanks!

[wbt]

@fainashalts Yes I believe it is, though it's blended into the background and become harder to notice.

[wbt]

FYI I am still observing the issue today.

[eggplantzzz]

Thanks for the update! Always helpful to know how current these are.