[R4R] Upgrade to 1.10.3

[yutianwu]

Description

Upgrade to go-ethereum 1.10.3.

Changes

Snapshots

Snapshots are an acceleration data structure on top of the Ethereum state, that allows reading accounts and contract storage significantly faster. The snapshot feature reduces the cost of accessing an account from O(logN) to O(1).

Though snapshots have enormous benefits, there are certain costs to them:
A snapshot is a redundant copy of the raw Ethereum state already contained in the leaves of the Merkle Patricia trie so it requires an additional disk overhead.
Since nobody has snapshots constructed in the network yet, nodes will initially need to bear the cost of iterating the state trie and creating the initial snapshot themselves. This might take a day to a week but you only need to do it once in the lifetime of your node.

Snapshot feature is enabled by default, you can disable it via --snapshot=false.

Snap sync

Now you have two different ways to synchronize the BSC network: full sync and fast sync. Full sync operated by downloading the entire chain and executing all transactions; fast sync placed an initial trust in a recent-ish block, and directly downloaded the state associated with it (after which it switched to block execution like full sync).

For fast sync, it needs to download the trie nodes one by one. So it will take time to download all the nodes if there are millions of nodes. And for the serving peers, they need to traverse all the nodes and it also takes time.

Snap sync is designed to solve the fast sync problems. The core idea is instead of downloading the trie node-by-node, snap sync downloads the contiguous chunks of useful state data, and reconstructs the Merkle trie locally.

You can manually enable snap sync via --syncmode=snap. Note: usually it will need one or two weeks until there is enough client upgrade to the version with snap enabled, you may use snap sync after that.

Offline pruning

If you have snapshots enabled and fully generated, Geth can use those as an acceleration structure to relatively quickly determine which trie nodes should be kept and which should be deleted. Pruning trie nodes based on snapshots does have the drawback that the chain may not progress during pruning. This means that you need to stop Geth, prune its database and then restart it.

Execution time wise, pruning takes a few hours (greatly depends on your disk speed and accumulated junk), one third of which is indexing recent trie nodes from snapshots, one third deleting stale trie nodes and the last third compacting the database to reclaim freed up space. At the end of the process, your disk usage should approximately be the same as if you did a fresh sync.

To prune your database, please run geth snapshot prune-state.

Note: You need to upgrade the client and run for a while before prune the data

Transaction unindexing

Geth no longer keeps transaction inclusion info for all transactions, and instead limits the storage of inclusion records to one year. For application developers, this change means that very old transactions can no longer be accessed by hash.
If you would like todisable this behavior and keep inclusion information for all historical transactions, you can re-enable indexing using the --txlookuplimit=0 command-line flag.

Database changes

There is an incompatible database change: adds a prefix for contract code in order to separate the codes and trie nodes.

So if you use the version of geth, you will not be able to rollback to the older version geth.

Geth command changes

• --rpc -> --http - Enable the HTTP-RPC server
• --rpcaddr -> --http.addr - HTTP-RPC server listening interface
• --rpcport -> --http.port - HTTP-RPC server listening port
• --rpccorsdomain -> --http.corsdomain - Domain from which to accept requests
• --rpcvhosts -> --http.vhosts - Virtual hostnames from which to accept requests
• --rpcapi -> --http.api - API’s offered over the HTTP-RPC interface
• --wsaddr -> --ws.addr - WS-RPC server listening interface
• --wsport -> --ws.port - WS-RPC server listening port
• --wsorigins -> --ws.origins - Origins from which to accept websockets requests
• --wsapi -> --ws.api - API’s offered over the WS-RPC interface
• --gpoblocks -> --gpo.blocks - Number of blocks to check for gas prices
• --gpopercentile -> --gpo.percentile - Percentile of recent txs to use as gas suggestion
• --graphql.addr -> --graphql - Enable GraphQL on the HTTP-RPC server
• --graphql.port -> --graphql - Enable GraphQL on the HTTP-RPC server
• --pprofport -> --pprof.port - Profiler HTTP server listening port
• --pprofaddr -> --pprof.addr - Profiler HTTP server listening interface
• --memprofilerate -> --pprof.memprofilerate - Turn on memory profiling with the given rate
• --blockprofilerate -> --pprof.blockprofilerate - Turn on block profiling with the given rate
• --cpuprofile -> --pprof.cpuprofile - Write CPU profile to the given file

For more details about the command changes, you can refer to: v1.10.0.

Rpc changes

BREAKING CHANGE: Non-EIP155 transactions (i.e. transactions which are not replay-protected) are now rejected by the RPC API. You can disable this restriction using the --rpc.allow-unprotected-txs command-line flag.
Since there are txs signed by old clients, we need to add this flag for all our nodes.

Response changes in eth_estimateGas, eth_call, eth_sendRawTransaction.

The response will now add a message field in addition to the data field if these RPCs cause the EVM to execute a REVERT operation. For example:

{
   "jsonrpc":"2.0",
   "id":1,
   "error":{
      "code":3,
      "message":"execution reverted: num_ is \u003c= 10",
      "data":"0x08c379a00000000000000000000000000000000000000000000000000000000000000020000000000000000000000000000000000000000000000000000000000000000d6e756d5f206973203c3d20313000000000000000000000000000000000000000"
   }
}

Response changes in eth_getTransactionByBlockHashAndIndex, eth_getTransactionByBlockNumberAndIndex, eth_getTransactionByHash.

The response now adds a type field to denote the transaction type. For example:

{
   "jsonrpc":"2.0",
   "id":1,
   "result":{
      "blockHash":"0x812619b0a7eb58de4f4c4fcf0b497567705f9182c2ad102745fa5ed87b3d96b5",
      "blockNumber":"0x95dc",
      "from":"0xef7368f755f5f551a602971b1f792b2737bc6e16",
      "gas":"0x2dc6c0",
      "gasPrice":"0x430e23400",
      "hash":"0xabbed20a397ff811fddf2b3f559e6291c1d14c6456bd1eca5fde82169e283857",
      "input":"0x29e99f07000000000000000000000000000000000000000000000000000000000000000c",
      "nonce":"0x55731a",
      "to":"0x6ac00c8b5514496aaae72cc2e6b8d4c29ee7d4ba",
      "transactionIndex":"0x0",
      "value":"0x0",
      "type":"0x0",
      "v":"0x1b",
      "r":"0x6e8c8af7e3266f0216b9a796eddd736a4f9bcdbfd6a47c49f566c187174d3268",
      "s":"0x5bf210f56d04126e0864643cb06670abe46fbf935446438490c2ff9ce6921adf"
   }
}

GraphQL changes

The GraphQL API is no longer available separately from the JSON-RPC HTTP server. If you want GraphQL, you need to enable it using the --http --graphql flag combination. The --graphql.port and --graphql.addr flags are no longer available.

Config changes

Remove whisper config.

[Shh]
MaxMessageSize = 1048576
RestrictConnectionBetweenLightClients = true

Remove GraphQL config.

GraphQLPort
GraphQLVirtualHosts

Rename DiscoveryURLs to EthDiscoveryURLs.

For more details about config changes, you can refer to go-ethereum v1.9.19, v1.9.21, v1.10.0

Preflight checks

• build passed (make build)
• tests passed (make test)
• manual transaction test passed

[unclezoro]

Please refine the description of the PR, and also change the version to v1.8.0

[spec42]

Hi, my database is 560G beacause of the bug fixexd in 1.0.7-hf1. Will the prune command be able to shrink this too? Or should i sync from scratch?

[gituser]

Hi, my database is 560G beacause of the bug fixexd in 1.0.7-hf1. Will the prune command be able to shrink this too? Or should i sync from scratch?

will be faster to sync from scratch

[jmouvill]

Hi, how/when do we know when validators will use this release ? I suppose it will be made as a requirement at some point to be elected ? thanks

[nilampatel-engineer]

Hi, running bsc full node with --gcmode = archive, syncmode=full, chain has stopped syncing with running out of space on device, if I add geth snapshot prune-state. it will prune database means will I loose complete historical data? means will not b able to see any chain history? what storage size is recommended? I have given 1TB and I ran out of space

[gituser]

@nilampatel-engineer for archive node --gcmode=archive you will need 15TB+ or maybe more.

You don't need archive node in order to get historical data unless you need to know exact balance at certain block.