Nonce calculation is broken for private networks #1999
Comments
|
I'm struggling with the same issue while testing. |
|
Sorry for neglecting this over the long weekend, I'll be working to fix this soon. |
|
No problem Dan, thanks for taking the time. I investigated a little further, and it looks like this issue has to do with network ID specification for private networks. Account history is tied to network ID. So, when you switch to a private network with a different network ID, you will get a fresh account with the correct nonce. However, if you switch to a private network with the same network ID, the account history is the same and the nonce mismatch will occur. So one workaround for right now is to ensure that the various private networks you interact with have different network IDs. However, it would still be really nice to have the option to delete the account's history from the UI, forcing the correct nonce calculation. |
ghost
assigned
|
Oh, I may have misunderstood the nature of this problem. I think private networks should be responsible that they have unique IDs, and we shouldn't work too hard to ensure identically-identified networks both work with MetaMask. |
|
My guess in my case that the problem is not with the network id but the fact that I use the same accounts for different networks while testing.
|
Lots of people do that, so it seems unlikely that if this were the bug, it would affect so few people. I followed your reproduction steps (albeit with testrpc on both instances), and has no problem. To clarify: You need to specify a distinct networkId and distinct chainId for MetaMask to identify a distinct network. Could you try that and let me know if it works, @szerintedmi? |
|
I retried and I still had the issue with one of my accounts. All other accs work. I played a bit and I have a workaround. That's what I receive: When I send a tx without MetaMask it works. It seems the nonce increases with every tx I send from the non MetaMask window: Once the nonce reaches the MetaMask nonce I can send tx again through MetaMask. |
|
I'm very tempted to think that the one affected account is having problems because it specifically had transactions sent on a private chain with identical IDs, just because that's the simplest explanation I can think of. If that's not it, I'd love to cook up an edge case for our nonce-tracker that it's failing. You may have guessed that when we compose a transaction, we try to account for locally pending transactions, and so it's possible that on a new private chain with the same ID as another, we respect the highest nonce between the network-provided one and the locally-pending-tx derived one, and increment from there. Let me know if you come up with any theories on this, I'm chipping away on another issue that I understand at the moment, but would be happy to fix any incorrect behavior here if we could identify it. |
|
This is happening to me too. What's the alternative for using Truffle? |
|
happens to me from time to time. no idea why. i uninstall and reinstall metamask to fix it. |
…t get nonce tracking right with a fresh running testrpc when previous transactions had been performed with the same account and networkid. See discussion at MetaMask/metamask-extension#1999.
|
@danfinlay you said
How does one set a new chain id? I'm having this problem too. Using the same snapshot and account locally and on a dev server, the nonces get out of sync and I can't execute transactions on the local testrpc now. |
It is dependent on the client you are using. I believe the flag is |
|
I just got the same error message: I'm using
If I understand correctly making a couple of transactions against an ephemeral chain, then restarting such chain will lead to this situation. Reinstalling the MetaMask extension solved the issue "of course". |
|
There are two different issues people are reporting here:
|
|
To move this off of Twitter. :-) Switching between networks isn't working for me with Truffle 4's An example of a failed transaction in Firefox 57 with MetaMask 3.12.0 is: |
|
Reviewing the comments in here, this issue is actually a little bigger than what I was suggesting for switching networks and back again. Sorry about the skim. The problem here is that MetaMask calculates nonces locally on a per-network basis. That means if you connect to it to "the same network" (by ID) on two different endpoints, MetaMask will currently assume these are the same network, use its same history of successful transactions, check the nonce everywhere it can, assume the current node is behind (since MetaMask is aware of a newer tx), and it'll use the latest tx it knows of to calculate nonce. Current CausesWhat we need here is a way of detecting a new network, even when all the signs indicate it's the same network. Some of the signs that prevent MetaMask from noticing this today include:
Possible Solutions
The second one should only be used if no good solution can be found for the first one. Some detection strategies: Periodically check network and chain IDsThis can only partially alleviate cause #1, because if the networks share IDs, it would go undetected. Detect when checking nonceSince people usually experience this when sending a TX, nonce calculation time would seem like a good time, but I'm scratching for a good, definite set of indications that the chain is different. Tracking known blocksA method for identifying a new network on demand could be fairly reliably implemented (as long as new chains were not identical to previous ones):
Since this method requires re-checking known blocks on the node, the question rises of when we do this. Nonce calculation is a nice failsafe, since this is when the problem first burns people, but it would be better if we could detect immediately, so we don't show wrong balance, show incorrect tx history, etc. Re-Checking known successful transactionsThere's a tricky bit around "confirmed transactions". Part of why we use locally confirmed transactions, is because MetaMask is sometimes pointed at RPC endpoints that might not be fully synchronized, and in these cases, we still want to generate valid nonces. We could re-check our oldest known successful transaction by hash, and if it's unknown, that could also be used to signal a remote network change. Times we could perform this procedure
ConclusionI think I have a fairly actionable solution here, please add any improvements anyone can think of. It seems like we've had a big bump in developer experimentation recently, because this has definitely been behavior for the lifetime, but we've had a huge spike in actual complaints, so it seems like this feature is fairly important to some segment of users. |
|
When detecting a new network with an identical ID, some special things will need to be done:
Dealing with TX historyEither:
Short term easy solution is to trash that chain's tx history. |
|
My workaround for now is:
It takes around 30-60 seconds each time i do that... Wouldn't it be easiest to create a button |
|
We've published a new version that should auto-update soon, http://metamask.helpscoutdocs.com/article/36-resetting-an-account |
|
That's awesome @danfinlay ! Thank you! |
|
Brilliant! Reset account sounds scary though. Like a loser could lose some
important data if he clicks it which doesn't seem to be the case?
…On 2 Feb 2018 02:20, "Rob Myers" ***@***.***> wrote:
That's awesome @danfinlay <https://github.com/danfinlay> ! Thank you!
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#1999 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AC8oXwWet_7rMKuOvxbksBK1dtKLDj8vks5tQlTYgaJpZM4PJiMH>
.
|
That's right, we don't want normal people doing it. It's scary on purpose. Thanks to @brunobar79 for writing the change PR! |
The problem is that clearing a history with pending transactions can cause the user to submit transactions with an identical nonce, which can cause all kinds of confusion. It's better if users don't. We estimate nonces under normal conditions very well. |
I'm not quite sure of that. I am getting an error in which I use Metamask to attempt a transaction (against Ganache 1.0.1) that is (properly) reverted. The transaction count goes up one in Ganache but not in Metamask, and then the next time I attempt to send a transaction from MetaMask I get "Error: the tx doesn't have the correct nonce" where the account nonce is one larger than the transaction nonce. The workaround is to reset the network, rerun all transactions, and rerun useless transactions to get the count up to where Metamask thinks it should be (but not higher). I can then run transactions again, but I notice that any transaction replaces the one before it at the top of the list of history for that account, instead of adding a new element to the list as it used to. I also notice that now, even without reverted transactions, Metamask no longer increases the nonce when the account has had other non-Metamask transactions, resulting in nonce mismatch errors like this: I suspect this is a separate issue but when looking for possible duplicates, “Nonce calculation is broken for private networks” [this issue] seems to be a pretty strong candidate. All of this testing is without the newly contributed “Reset Account” button (thanks @danfinaly), so that patch is not the cause. |
@wbt - to clarify, you observe this without restarting Ganache? |
|
@benjamincburns Yes, that's without restarting Ganache. |
|
@wbt can you please try this with the current beta of If that works, then I assume you'll like running the next beta of the Ganache UI much better than the current beta or stable release. My suspicion is that in Ganache we report the transaction failure as an RPC error when the transaction is submitted, and MetaMask (correctly) interprets that as the transaction being rejected prior to it entering the transaction pool. If that's the case, I'd regard that as a separate issue from this one, and I wouldn't discredit MetaMask's nonce tracking because of it. We're the ones breaking the "standard," there. |
|
@danfinlay incidentally when that happens we break the JSON-RPC spec and return both an I wouldn't blame you if you don't want to do this, however. |
It's not so much that I don't want to, but that we are already stretched so thin, we greatly appreciate each non-breaking change. I would encourage Ganache to return success on submitting a tx w/ on-chain error, and allow clients to identify failure the usual way, by querying tx by hash. |
|
@danfinlay it's my intention to make that the default behavior in the next major release. |
|
Hi, xDai chain (web3.eth.getNetwork() == 100) (https://dai.poa.network) I've got nounce from Ethereum Main Network uninstalling metamask and installing again helped. |
|
I'm also getting this error ("[ethjs-rpc] rpc error with payload...") again today on a long-running private network that I can't reset. Resetting the account no longer helps. |
|
I'm going to close this in favour of the numerous other open tickets about improvements to this workflow (#4254, #5067, #6559, #8081). I think the "Reset Account" button does serve as a workaround for the time being. @wbt: The issue you're describing sounds distinct from the OP, so feel free to create a new ticket if you're still encountering it! Thanks |
and others
Currently, nonce calculation is broken for private networks. The main problem is that transaction history is retained for an account even after switching the RPC endpoint to a new network. This can cause the nonce on a new network to be too high, resulting in Metamask transactions never going through.
To reproduce:
I can see in app/scripts/lib/nonce-tracker.js that you're taking the max value between the locally calculated nonce and the transaction count for the address in question. This works for public/test networks where the expected nonce for an account can never go down, but that assumption doesn't hold for private networks when you can switch from a chain where an account's transaction count is 8 to one where the transaction count is 0.
One possible fix is clearing the "Private Network" transaction history when switching RPC endpoints. This isn't totally ideal, but does result in the correct nonce being calculated, and doesn't require you to redo any of the current nonce calculation functions.
The text was updated successfully, but these errors were encountered: