Metamask updates seem to have broken some functionality in the Contract API in ethers

[cloudonshore]

I've created this codesandbox to reproduce (works with metamask on mainnet or rinkeby, all that is needed is a small ETH balance to test) https://ott4i.codesandbox.io/

The bug happens with all Contract interactions, but in the above test, the user tries to approve the null address to transfer 5 wei of WETH.

Expected behavior:
After submitting the transaction, you should see the step signature successful, awaiting mined transaction listed, and after mining, you should see the step transaction mined listed

Observed behavior:
After submitting the transaction, the promise for the successful signature doesn't resolve. However, when the transaction is mined both promises resolve at once and signature successful, awaiting mined transaction and transaction mined display simultaneously

You can change the ethers.js version in package.json, it seems to happen with all recent versions (including the new v5 beta). This started happening in one of our codebases that had no changes in the past couple weeks, so I believe it's because of updates that metamask has rolled out.

Please let me know if you have any other questions.

Thanks,
Sam

[ricmoo]

I've been chatting with MetaMask, as I think this is a change on their end.

I haven't had a chance to verify, but could you possibly try installing an old version of MM and see if (with the new code, either v4 or v5) this still happens? If it is a MM problem, it is something they said they would be happy to fix. I just haven't had time to check.

If you are busy though, I will be checking later this week.

Thanks!

[ricmoo]

This is happening to more and more people, I’ll be looking more into it tomorrow, but if people who have reproduced this could try out various versions of MetaMask and help isolate which versions are and are not affected, it would be a huge help.

Here are the MetaMask releases to try: https://github.com/MetaMask/metamask-extension/releases

And here is the ticket on MetaMask’s side of the investigation: MetaMask/metamask-extension#6704

Thanks! :)

[cloudonshore]

@ricmoo So I went and sequentially installed all the versions between 6.4.1 and 6.0.0 and none of them fixed the issue. I also tried using older versions of ethers.js. I then thought what the other variable in the system was and I decided to try switching out the default geth node and voila! That fixed it!

Instead of the default rinkeby provider in metamask I used the custom RPC https://geth-rinkeby.airswap-api.com which is our rinkeby geth node at AirSwap. That fixes the issue. I will also share this comment on the thread on their repo so they are aware.

[cloudonshore]

@ricmoo
Ok so the problem seems to stem from the face that the eth_getTransactionByHash RPC call that happens after a eth_sendRawTransaction submission has a JSON RPC id that is hardcoded to 42. INFURA doesn't like it, all other nodes seem to be fine with it.

AIRSWAP MAINNET NODE (https://geth-cluster.airswap-api.com)

eth_sendRawTransaction call and response

{"id":1561124604799611,"jsonrpc":"2.0","params":["0x6f14850165a0bc008302710094c02aaa3b223fe8d0a0e5c4f7ead9083c756cc265af3107a400084d0e30db026a04fb5a470f173e7046ac4b7f84f6d81ae64341cf1c0150562ae46bb92b9192eb1a004b9a082956197ad531501000ba193fd47e4fe6e4bf970ddb0b2ca59849bf81d"],"method":"eth_sendRawTransaction"}
{"jsonrpc":"2.0","id":1561124604799611,"result":"0x005de619d757908d075438762aa19b41c9a7d34d067a870d430ed663742a675a"}

subsequent eth_getTransactionByHash call and response

{"id":42,"jsonrpc":"2.0","params":["0x005de619d757908d075438762aa19b41c9a7d34d067a870d430ed663742a675a"],"method":"eth_getTransactionByHash"}
{"jsonrpc":"2.0","id":42,"result":{"blockHash":"0x0000000000000000000000000000000000000000000000000000000000000000","blockNumber":null,"from":"0x319a803235ed2bb5d3b2824b7bbc15aadab2b6d7","gas":"0x27100","gasPrice":"0x165a0bc00","hash":"0x005de619d757908d075438762aa19b41c9a7d34d067a870d430ed663742a675a","input":"0xd0e30db0","nonce":"0x14","to":"0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2","transactionIndex":"0x0","value":"0x5af3107a4000","v":"0x26","r":"0x4fb5a470f173e7046ac4b7f84f6d81ae64341cf1c0150562ae46bb92b9192eb1","s":"0x4b9a082956197ad531501000ba193fd47e4fe6e4bf970ddb0b2ca59849bf81d"}}

INFURA MAINNET NODE (https://mainnet.infura.io/8LNJeV3XEJUtC5YzpkF6)

eth_sendRawTransaction call and response

{"id":1561124193382843,"jsonrpc":"2.0","params":["0x6f13012a05f20083027100942aaa39b223fe8d0a0e5c4f27ea9083c756cc2865af3107a400084d0e30db025a0e1b9528d68c6ff96ff54a1f65894386a600fc107871c4f1a040cbf534e8e8e3aa0156d2f4b196ff1a058569690656c51e5445ebeb0272e0d99705b5ff5346c2069"],"method":"eth_sendRawTransaction"}
{"jsonrpc":"2.0","id":1561124193382843,"result":"0x50c3babfec0a7fa0b1c194c8ea2b5966bed4a81c0d2906b39640bbfee472e653"}

subsequent eth_getTransactionByHash call and response

{"id":42,"jsonrpc":"2.0","params":["0x50c3babfec0a7fa0b1c194c8ea2b5966bed4a81c0d2906b39640bbfee472e653"],"method":"eth_getTransactionByHash"}
{"jsonrpc":"2.0","id":42,"result":null}

The null result in INFURA's response to the eth_getTransactionByHash call is what causes the promise for the transaction submission to not resolve.

Also btw I generated the above behavior with just ethers.js and a ledger nano, so no metamask involved.

[ricmoo]

Interesting... I’ve removed the hard-coded 42 from JsonRpcProvider, but missed Web3Provider. I will make that change tonight regardless.

Maybe INFURA added some sort of new caching? Because that hasn’t changed in the Web3Provider. It also doesn’t explain why I can’t reproduce it. :p

I will reach out to INFURA to see if this makes sense to them.

[danfinlay]

It might be notable that geth recently made a change to how they report errors for some kinds of invalid requests, it just occurred to me that it might remotely be related:
ethereum/go-ethereum#18254

[ricmoo]

I've updated the Web3Provider to use a dynamic id. In v5, I'll be adding a random salt to the ID to mitigate response injection, which a lot of popular wallets seem to be susceptible to (MetaMask seems fine, but Trust Wallet is certainly vulnerable). But in v4, this is fine since it was already hard-coded previously.

@cloudonshore Can you try out 4.0.31 to see if that solves the problem?

[deacix]

Hi ricmoo,

is still not work with 4.0.31

[ricmoo]

@deacix Thanks. It doesn't have anything to do with the JSON-RPC id then.

I think I have a new theory. Can I get some help from people experiencing this?

Follow these instructions for "Background Logs". Basically, right click on the extension and enable "developer tools" for the background script.

Then just load MM as usual, looking at the logs, and note the URL it is using.

My MetaMask (which works fine) is using the INFURA v1 URL. My theory is that new installs (which would use the new v3), may be designed to not return transactions in the pending TX pool.

Still a theory, but lets find out. :)

[danfinlay]

. My theory is that new installs (which would use the new v3), may be designed to not return transactions in the pending TX pool.

It's also worth keeping in mind that Infura is a load-balanced service, and not every node necessarily knows about the full tx pool of every other Infura node.

If you want a fast response from the provider API, it probably is necessary that MetaMask locally cache these values and return them without hitting the network.

[ricmoo]

Absolutely, which is why ethers polls, (with exponential backoff) until it gets a response. :)

I do think it makes sense for MetaMask to locally cache transactions it signs and sends, but that may be a non-trivial change...

[danfinlay]

I do think it makes sense for MetaMask to locally cache transactions it signs and sends, but that may be a non-trivial change...

The basic implementation would actually be very trivial. We already cache all of this (and have to, for the sake of our own submission-with-exponential-backoff). It's just a matter of checking our local store as a middleware cache before hitting the network.

Hopefully this comment here can serve as an actionable summary when I return here, so we don't have to re-read the whole thread to get oriented ;)

[BigMurry]

Confirmed this issue on Ethereum mainnet, but on Ropsten it is fine.
It's happened not just on Metamask. Mobile wallets (Imtoken) have this issue too.

[danfinlay]

Hey, @cloudonshore, you have encountered a pretty strange MetaMask bug. That value should have been migrated away like 2 years ago. Is it possible you've been operating on the same installation for over 2 years? Would you mind emailing me your state logs? This thread is probably not the best place for debugging that particular issue, I don't think it's related to ethers.js

To help us diagnose your issue, you can download your "state logs" by following this guide, and then sending them to support@metamask.io .

[danfinlay]

Also if your install is very old, it's possible the easiest workaround is to :

1. Back up your seed phrase
2. Uninstall MetaMask
3. Reinstall MetaMask

Sorry that this will mean settings like token list and account nicknames would be lost (we're adding config backup soon!).

[cloudonshore]

@ricmoo @danfinlay the api endpoint it is hitting inside the background logs is https://api.infura.io/v1/jsonrpc/mainnet/.

So the issue seems to be caused by infura not returning a transaction when queried about a transactionHash that was returned by a "eth_sendRawTransaction" call. When using the default mainnet setting in metamask, the transaction hash doesn't return a result until the transaction is actually mined. Inside metamask, the eth_getTransactionByHash call seems to be implemented as a GET request to the following URL

https://api.infura.io/v1/jsonrpc/mainnet/eth_getTransactionByHash?params=%5B%220xafe553bc10693b3598f9edbc4eba367abbb7d1dec9924fb0e498131ab176adff%22%5D

and the response is

{"jsonrpc":"2.0","id":0,"result":null}

ethers continues polling until after the block for that transaction is mined, and then that same call returns a transaction

{"jsonrpc":"2.0","id":0,"result":{"blockHash":"0x4c21c60dec0e53bf06bd4a50cfa14701b6a81be492e5eb076f7709327ed0e4db","blockNumber":"0x7bdd31","from":"0xdead0717b16b9f56eb6e308e4b29230dc0eee0b6","gas":"0x30d40","gasPrice":"0x2540be400","hash":"0xafe553bc10693b3598f9edbc4eba367abbb7d1dec9924fb0e498131ab176adff","input":"0x1d4d691d0000000000000000000000001550d41be3651686e1aeeea073d8d403d0bd2e300000000000000000000000000000000000000000000000004563918244f4000000000000000000000000000089d24a6b4ccb1b6faa2625fe562bdd9a23260359000000000000000000000000dead0717b16b9f56eb6e308e4b29230dc0eee0b6000000000000000000000000000000000000000000000000003896f176a50f550000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005d24a1e600000000000000000000000000000000000000000000000000000000029c5627000000000000000000000000000000000000000000000000000000000000001c01bea1d2002f74943f831124ad3e04ebc0752fa330f6e7ff2bc2e9a1411e82d66dfed40026ab2ebf2b76e23fd44ca3db3f9cf914470ea4594a296446da6a2c40","nonce":"0x5bc","r":"0xe0b9afc9aceb4e9813e8fe851b21d2f8b33ab0f541e7c1c9d9e39eaece924dbe","s":"0x46dba30f3990a9fe9414fbff1aadbbead43ee1877b623e107bdb7f707415fd8a","to":"0x8fd3121013a07c57f0d69646e86e7a4880b467b7","transactionIndex":"0x53","v":"0x26","value":"0x3896f176a50f55"}}

When I change my RPC provider to the custom mainnet provider (https://geth-cluster.airswap-api.com), immediately after I submit the transaction, I see the ethers query for it show up in metamask, but instead of a GET request it's a POST request to the RPC url https://geth-cluster.airswap-api.com with the following payload:

{"method":"eth_getTransactionByHash","params":["0x00ca6c3e3923c03f070cff4b17daaa789135bdade8eb16613c992dde0dd5ab9c"],"id":4038002242,"jsonrpc":"2.0"}

and receive a response immediately:

{"jsonrpc":"2.0","id":4038002242,"result":{"blockHash":"0x0000000000000000000000000000000000000000000000000000000000000000","blockNumber":null,"from":"0xdead0717b16b9f56eb6e308e4b29230dc0eee0b6","gas":"0x30d40","gasPrice":"0x2540be400","hash":"0x00ca6c3e3923c03f070cff4b17daaa789135bdade8eb16613c992dde0dd5ab9c","input":"0x1d4d691d0000000000000000000000001550d41be3651686e1aeeea073d8d403d0bd2e300000000000000000000000000000000000000000000000004563918244f4000000000000000000000000000089d24a6b4ccb1b6faa2625fe562bdd9a23260359000000000000000000000000dead0717b16b9f56eb6e308e4b29230dc0eee0b600000000000000000000000000000000000000000000000000389c9328ebff5b0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005d249ff6000000000000000000000000000000000000000000000000000000000128fead000000000000000000000000000000000000000000000000000000000000001cd9353fc5a8d0ed8011cee029f65453bc3f4f8a9d6b65e855d94f52bd293f318c33be3739b9ff3da5d748c5afb5e9e8d12dbfe47ac77f00721c8ec0b9e8c3995a","nonce":"0x5ba","to":"0x8fd3121013a07c57f0d69646e86e7a4880b467b7","transactionIndex":"0x0","value":"0x389c9328ebff5b","v":"0x25","r":"0x6b3b025963281fef726a782bb3efbf3f51f5cc0ce1de2f6a6ff2b6bb8fe1992c","s":"0x1bab171f18aa33477c0c872aeed10dd3cd83d69a5036e5cdfa596f1a3e42e0"}}

This is definitely the root behavior that is causing the issue for me.

[cloudonshore]

@danfinlay I've forwarded the state logs to that support email

[ricmoo]

@cloudonshore Thanks! Yes, I've already sent a quick note regarding this to the MetaMask peeps last week, to see what they think. :)

I suspect this is not an issue with MetaMask at all, but something INFURA likely recently added (possibly as an anti-DDoS metric). Regardless, it is something MetaMask has tentatively agreed to address, since it is somewhat simpler (and efficient) for them to implement anyways.

[cloudonshore]

Seems like progress on it has kind of stalled, I was noticing that there is potentially another workaround from the ethers side.

ethers.js/src.ts/providers/json-rpc-provider.ts

Line 161 in 77c771b

return poll(() => {

If, instead of returning a "poll" for the transaction, this function returned the transactionHash immediately, then I could know that the transaction had been successfully submitted, and could poll for it myself. The way it is now, no response is given until it is mined. .

[ricmoo]

If you do not need the transaction (only the hash), you can use the UncheckedJsonRpcSigner. Keep in mind that when deploying contracts, you will need to wait until the transaction is mined before you can get the Contract Address, and things like that.

In v5 this class is built-in. To get an UncheckedSigner, use provider.getUncheckedSigner(addressOrIndex) instead of provider.getSigner(addressOrIndex). Or if you already have a JsonRpcSigner, you can call signer.connectUnchecked() which will return a new Signer which does not poll.

The TransactionResponse from an UncheckedSigner will only have the hash available, all other properties will be null, but the .wait() method will work, which will work as normal, returning the receipt.

[0xclem]

Any updates or workarounds regarding this issue? I'm using v4.0.37.

[ricmoo]

One of my recently released projects also suffers from this issue. I’ll be pinging the MM team again this week. :)

[0xclem]

@ricmoo same! Some users have been complaining about it. Did the MM team get back to you?

[danfinlay]

I'm checking in on our (MetaMask) side, looks like it got dropped somehow, sorry about that.

[ricmoo]

@danfinlay I have another related ask that could probably get wrapped into the same changes; when calling eth_getTransactionCount for an account in MetaMask with “pending”, can the response take into account the same pool of inflight transactions? :)

[ricmoo]

I believe this has long since been taken care of? I'm going to close it now, but if anyone continues to have a problem, please re-open or start another issue.

Thanks! :)