v1.20.3

Release notes

Fixed the configuration issue in v1.20.2 which caused an error in engine JSON-RPC when executed by CL clients:

eth_chainId found but the containing module is disabled for the url 'http://localhost:8551', consider adding module in JsonRpcConfig.AdditionalRpcUrls for additional url, or to JsonRpcConfig.EnabledModules for default url

Please read the full v1.20.2 changelog here.

Changelog

• Change EngineEnabledModules to be comma separated by @kamilchodola in #6057

Thanks @antondlr for reporting the issue.

Full Changelog: 1.20.1...1.20.3

v1.20.2

Release notes

⚠️ This release is replaced with v1.20.3 because of the issue in the holesky.cfg and holesky_archive.cfg which caused issues with proper Engine JSON-RPC usage by CL.

This release is aimed for support of new testnet chain in space: Holesky

A few details about Holesky:

• Launch Date: Sept/15, 2023, 14:00 UTC
• Epoch time: 1694786400
• LTS: Dec/2027, EOL: Dec/2028
• Network ID: 17000

More details can be found here: https://github.com/eth-clients/holesky

To use Nethermind with this new chain, follow these steps:

1. Run nethermind with new config:

   • For a Snap Synced node, use the flag --config holesky.
   • For an archival mode setup, use the flag --config holesky_archive.

2. Run a Consensus Layer Client of your choice which supports holesky network.

3. Enjoy!

Changelog

Holesky

• Add holesky configs and chainspec by @smartprogrammer93 in #6034

Other changes

• Fix PPA package post-install script by @rubo in #6045

Full Changelog: 1.20.1...1.20.2

v1.20.1

Release notes

Fixed the regression in v1.20.0 which caused an error in Prysm:

Could not reconstruct full bellatrix block batch from blinded bodies

No re-sync is required.

Additionally, we added a support for ARM64 PPA package, so now Nethermind can be installed on this architecture as well.

Please read the full v1.20.0 changelog here.

Changelog

• Restore v in tx signature for Geth compatibility by @rubo in #5937

Thanks @nazar-kuzo for reporting the issue.

Full Changelog: 1.20.0...1.20.1

v1.20.0

Release notes

⚠️ This release is replaced with v1.20.1 because of the issue in the eth_getBlockByHash JSON-RPC method which cause problems on Prysm side.

Major highlights

• Logs readability improvements. Now with color support, improved log organization, and the prominent display of vital information related to block production, it's easier than ever to have a comprehensive understanding of the current head of the chain and all necessary details:

  Break down of new logs:

  

  • JsonRPC logs are turned off by default.
  • Users of older versions of Windows may need to set registry entry [HKEY_CURRENT_USER\Console\VirtualTerminalLevel] to DWORD 1 to enable colors if using cmd. If colors are too dark they can be adjusted in your terminal color settings.

• Boosted block processing priority to improve attestation rates during sync

• Improvements in sync (faster Snap Sync, Old Bodies and Old Receipts) + reduced SSD writes requirement

• Full Pruning improvements (better threads management, free disk space check for full pruning, pruning time reduced etc).

• This is the Shapella-ready release for the upcoming hard-fork on Gnosis!
  The Gnosis Shapella hard-fork is scheduled at 11:34:20 AM UTC on August 1, 2023.

Changelog

Gnosis

• Add Gnosis Shanghai hard-fork timestamp by @rubo in #5848
• Add more Gnosis bootnodes by @MarekM25 in #5910

Cancun

• Add RLP encoding and hashing for Blob txs by @flcl42 in #5593
• Add path to kzg setup file by @deffrian in #5797
• EIP-4844 V6 fixes and improvements by @flcl42 in #5745

Bug fixes and stability

• Improve peer discovery and connectivity by @asdacap in #5846
• Update DotNetty and RocksDB packages (fix issue with installation of Homebrew on MacOS AMD machines) by @rubo in #5883
• Remove v of non-legacy tx signature from RPC response (#5927) by @rubo in #5927
• Safety check for full pruning by @deffrian in #5550
  Example usage:
  Now the client will check available disk space to ensure that machine is capable to successfully execute full pruning. To disable this feature set --Pruning.AvailableSpaceCheckEnabled=false
• Don’t hide precompile contract exception by @asdacap in #5679
• Fixed pivot-related edge case that could lead to node corruption by @MarekM25 in #5724
• Copy timestamp from json by @deffrian in #5731
• Fix stuck state from period offline by @benaadams in #5710
• Fix point evaluation precompile by @deffrian in #5739
• Fixed eip2537 activation by @MarekM25 in #5824
• Quick fix for FullPruner by @marcindsobczak in #5809
• Bugfix - exchangeCapabilites based on ChainSpec by @MarekM25 in #5801
• Fix estimate gas by @deffrian in #5800
• Fix gas fees accounting in TransactionProcessor.Trace by @deffrian in #5743
• Fix KeccaksIterator not working when some keccak zero prefix is not i… by @asdacap in #5780
• Fixed fail on precompile dll import error by @Demuirgos in #5725.
• MaxDegreeOfParallelism defaults for full pruning by @MarekM25 in #5662
  • Introduced few more logs to better navigate through Full pruning process.
  • Revised parameters for --Pruning.FullPruningMaxDegreeOfParallelism parameter
    • -1 - number of logical processors
    • 0 - (DEFAULT) 25% of logical processors. This would reduce overhead of hardware and will give better results of attestations during that process.
    • 1 - In this example, 1 logical processor will be used. Set any value higher than 0
  • Revised parameter --Pruning.FullPruningMemoryBudgetMb and changed the default to 4000 MB

Logging

• Improved colour logs + block tx gas gwei range by @benaadams in #5770
• Move detailed peer logging to debug, add diversity by @benaadams in #5795
• JSON-RPC logs from Info to Debug by @MarekM25 in #5782

Performance

• Boost Block Processor priority to improve attestations during syncing by @benaadams in #5764
• Apply cache miss hint which improves full pruning perf by @asdacap in #5723
• Improve db priorities which improves snap sync perf by @asdacap in #5722
• Apply auto tune to compressed db by @asdacap in #5771
• Perf/valuekeccak snap code by @asdacap in #5689
• Optimise Account access and contract detection patterns in Evm by @benaadams in #5646
• Feature/disable WAL flag which will reduce SSD write requirements during SnapSync phase by @asdacap in #5702
• Perf/keccak with valuekeccak which results with reduced memory and CPU usage by @asdacap in #5715
• Perf/faster block load for receipt by @asdacap in #5708
• Reduce exceptions in Evm by @benaadams in #5727
• Keccaks: Skip unneeded array allocations by @benaadams in #5735
• Less contention for OldReceipts and OldBodies by @benaadams in #5737
• Add option to have multiple network processing thread by @asdacap in #5749
  Example usage:
  In case internet speed is higher than 500Mbps, there is option to set flag --Network.ProcessingThreadCount 32 which may reduce Old Bodies sync phase time
• Less aggressive peer discovery by @benaadams in #5747
• Reuse batch Dictionary and skip empty batches by @benaadams in #5802
• Don't add Keccaks during cache pruning by @benaadams in #5781
• Optimize Snap sync + Sort DB Batches by @benaadams in #5789
• Old Bodies sync performance improvement with blobfiles by @asdacap in #5785
• Add fast test for balance < tx.Value + gas by @benaadams in #5783
• Remove from ConcurrentDict directly rather than building secondary list by @benaadams in #5779
• Improve peering management strategy which results with faster sync time by @asdacap in #5772
• Refactor/Split dispatcher and downloader by @asdacap in #5778
• Reduce datetime calls in peer discovery by factor of 100 by @benaadams in #5751
• More efficient EvmPooledMemory by @benaadams in #5752

Other changes

• refactor state and storage behind one interface by @tanishqjasoria in #5659
• Address build warnings in Nethermind.Serialization.Ssz.Test.csproj by @MarekM25 in #5695
• Add support for specifying address in CLI by @emlautarom1 in #5690
• Move some logic from blocktree into chain level info by @asdacap in #5705
• Refactor/extract block store from blocktree by @asdacap in #5704
• One more Engine API test: Blocks_before_pivots_should_not_be_added by @MarekM25 in #5719
• [README] Add GitPOAP Badge to Display Number of Minted GitPOAPs for Contributors by @kayla-henrie in #4278
• Revise README by @rubo in #5728
• Update Nethermind.Crypto.SecP256k1 package by @rubo in #5733
• Update Nethermind.Crypto.Pairings package by @rubo in #5742
• Cleanup/remove keccak key by @asdacap in #5748
• Fix/inconsistently failing tests by @asdacap in #5773
• Added GnosisSpecProvider and gnosis_loads_properly test by @MarekM25 in #5775
• Rename BlockChain -> Bl...

v1.19.3

Release notes

For the Gnosis network operators, this is a mandatory update because of the upcoming Shapella hard-fork. We urge you to update promptly to ensure seamless network functioning post-fork.

The Gnosis Shapella hard-fork is scheduled at 11:34:20 AM UTC on August 1, 2023.

Changes

• Add Gnosis Shanghai hard-fork timestamp by @rubo in #5848

Full Changelog: 1.19.2...1.19.3

v1.20.0-rc

Release notes

Major highlights

• Logs readability improvements. Now with color support, improved log organization, and the prominent display of vital information related to block production, it's easier than ever to have a comprehensive understanding of the current head of the chain and all necessary details:

  Break down of new logs:

  

  • JsonRPC logs are turned off by default.
  • Users of older versions of Windows may need to set registry entry [HKEY_CURRENT_USER\Console\VirtualTerminalLevel] to DWORD 1 to enable colors if using cmd. If colors are too dark they can be adjusted in your terminal color settings.

• Boosted block processing priority to improve attestation rates during sync

• Improvements in sync (faster Snap Sync, Old Bodies and Old Receipts) + reduced SSD writes requirement

• Full Pruning improvements (better threads management, free disk space check for full pruning, pruning time reduced etc).

• This is the Shapella-ready release for the upcoming hard-fork on Gnosis!
  The Gnosis Shapella hard-fork is scheduled at 11:34:20 AM UTC on August 1, 2023.

Changelog

Gnosis

• Add Gnosis Shanghai hard-fork timestamp by @rubo in #5848

Cancun

• Add RLP encoding and hashing for Blob txs by @flcl42 in #5593
• Add path to kzg setup file by @deffrian in #5797
• EIP-4844 V6 fixes and improvements by @flcl42 in #5745

Bug fixes and stability

• Safety check for full pruning by @deffrian in #5550
  Example usage:
  Now the client will check available disk space to ensure that machine is capable to successfully execute full pruning. To disable this feature set --Pruning.AvailableSpaceCheckEnabled=false
• Don’t hide precompile contract exception by @asdacap in #5679
• Fixed pivot-related edge case that could lead to node corruption by @MarekM25 in #5724
• Copy timestamp from json by @deffrian in #5731
• Fix stuck state from period offline by @benaadams in #5710
• Fix point evaluation precompile by @deffrian in #5739
• Fixed eip2537 activation by @MarekM25 in #5824
• Quick fix for FullPruner by @marcindsobczak in #5809
• Bugfix - exchangeCapabilites based on ChainSpec by @MarekM25 in #5801
• Fix estimate gas by @deffrian in #5800
• Fix gas fees accounting in TransactionProcessor.Trace by @deffrian in #5743
• Fix KeccaksIterator not working when some keccak zero prefix is not i… by @asdacap in #5780
• Fixed fail on precompile dll import error by @Demuirgos in #5725.
• MaxDegreeOfParallelism defaults for full pruning by @MarekM25 in #5662
  • Introduced few more logs to better navigate through Full pruning process.
  • Revised parameters for --Pruning.FullPruningMaxDegreeOfParallelism parameter
    • -1 - number of logical processors
    • 0 - (DEFAULT) 25% of logical processors. This would reduce overhead of hardware and will give better results of attestations during that process.
    • 1 - In this example, 1 logical processor will be used. Set any value higher than 0

Logging

• Improved colour logs + block tx gas gwei range by @benaadams in #5770
• Move detailed peer logging to debug, add diversity by @benaadams in #5795
• JSON-RPC logs from Info to Debug by @MarekM25 in #5782

Performance

• Boost Block Processor priority to improve attestations during syncing by @benaadams in #5764
• Apply cache miss hint which improves full pruning perf by @asdacap in #5723
• Improve db priorities which improves snap sync perf by @asdacap in #5722
• Apply auto tune to compressed db by @asdacap in #5771
• Perf/valuekeccak snap code by @asdacap in #5689
• Optimise Account access and contract detection patterns in Evm by @benaadams in #5646
• Feature/disable WAL flag which will reduce SSD write requirements during SnapSync phase by @asdacap in #5702
• Perf/keccak with valuekeccak which results with reduced memory and CPU usage by @asdacap in #5715
• Perf/faster block load for receipt by @asdacap in #5708
• Reduce exceptions in Evm by @benaadams in #5727
• Keccaks: Skip unneeded array allocations by @benaadams in #5735
• Less contention for OldReceipts and OldBodies by @benaadams in #5737
• Add option to have multiple network processing thread by @asdacap in #5749
  Example usage:
  In case internet speed is higher than 500Mbps, there is option to set flag --Network.ProcessingThreadCount 32 which may reduce Old Bodies sync phase time
• Less aggressive peer discovery by @benaadams in #5747
• Reuse batch Dictionary and skip empty batches by @benaadams in #5802
• Don't add Keccaks during cache pruning by @benaadams in #5781
• Optimize Snap sync + Sort DB Batches by @benaadams in #5789
• Old Bodies sync performance improvement with blobfiles by @asdacap in #5785
• Add fast test for balance < tx.Value + gas by @benaadams in #5783
• Remove from ConcurrentDict directly rather than building secondary list by @benaadams in #5779
• Improve peering management strategy which results with faster sync time by @asdacap in #5772
• Refactor/Split dispatcher and downloader by @asdacap in #5778
• Reduce datetime calls in peer discovery by factor of 100 by @benaadams in #5751
• More efficient EvmPooledMemory by @benaadams in #5752

Other changes

• refactor state and storage behind one interface by @tanishqjasoria in #5659
• Address build warnings in Nethermind.Serialization.Ssz.Test.csproj by @MarekM25 in #5695
• Add support for specifying address in CLI by @emlautarom1 in #5690
• Move some logic from blocktree into chain level info by @asdacap in #5705
• Refactor/extract block store from blocktree by @asdacap in #5704
• One more Engine API test: Blocks_before_pivots_should_not_be_added by @MarekM25 in #5719
• [README] Add GitPOAP Badge to Display Number of Minted GitPOAPs for Contributors by @kayla-henrie in #4278
• Revise README by @rubo in #5728
• Update Nethermind.Crypto.SecP256k1 package by @rubo in #5733
• Update Nethermind.Crypto.Pairings package by @rubo in #5742
• Cleanup/remove keccak key by @asdacap in #5748
• Fix/inconsistently failing tests by @asdacap in #5773
• Added GnosisSpecProvider and gnosis_loads_properly test by @MarekM25 in #5775
• Rename BlockChain -> Blockchain in Merge tests by @MarekM25 in #5806
• Add a master build docker image for each commit on master branch by @kamilchodola in #5777
• Add tests validating a slot by timestamp by @rubo in #5792
• Refactoring of chainspec tests & handle chainspec exceptions by @MarekM25 in #5819
• Transaction processor refactor by @jmederosalvarado in #5799

New Contributors

• @emlautarom1 made their first contribution in #5690
• @kayla-henrie made their first contribution in #4278

**F...

v1.19.2

Release notes

This is an important release for Teku users.

In order to ensure proper functionality with the upcoming Teku release 23.6.0, it is necessary to upgrade Nethermind from version 1.19.x to version 1.19.2. Nethermind 1.19.2 or higher will be compatible with future Teku releases. If you are using a different consensus client or an older Teku version, there is no need to update your node.

This release addresses an issue with engine_exchange capabilities (PR #5787 by @flcl42).

Nethermind versions 1.17 and 1.18 do not require this fix. However, we recommend updating them to the newest version.

v1.19.1

Release notes

Fixed the regression in v1.19.0 where eth_getLogs requests weren't returning correct results for topics with leading zeroes.
The fix is important for the Rocket Pool community as their integration uses topics with leading zeroes.

Doesn't require resync since the database was not corrupted.

Changelog

• Fix Keccak iterator not correctly detecting more items when another item is zero prefixed by @asdacap in #5780

Big thanks to @jclapis from Rocket Pool for reporting.

Full changelog: 1.19.0...1.19.1

v1.19.0

Release notes

⚠️ This release is replaced with v1.19.1 because of the issue in the eth_getLogs JSON-RPC method.

Major highlights

• Significant storage reduction

  ⚠️
  Downgrading from this version to an earlier one is not possible because of the new database format.
  Resync is not required to update to this version. However, you will get the full benefits of disk space optimization only after fresh sync.

• Faster sync due to the new auto-pivot approach

Details

• Storage reduction

  	v1.18.0		v1.19.0
  		Ancient barriers		Ancient barriers
  State	166 GB	170 GB	161 GB	161 GB
  Receipts	477 GB	269 GB	152 GB	104 GB
  Blocks	334 GB	222 GB	334 GB	222 GB
  Other	...	...	...	...
  Total	965 GB	678 GB	662 GB	504 GB

• Receipts DB size reduction

  • Significantly reduced database size by using a different encoding.
    You can’t downgrade from this version without a full db drop and resync. To get the full benefit of the DB size reduction a resync is needed as it will only apply to new receipts. A node will still work fine on 1.19 version without resyncing.
    You can also call the RPC method debug_migrateReceipts(20000000) to rewrite receipts but it only reduces receipt size partially and a resync tends to be faster.
  • Significantly reduced receipt database size by limiting transaction lookup via transaction hash to past year only (similar to Geth). If you need an older lookup, you may keep the old transaction hash index via --Receipts.TxLookupLimit 0.

• Reduced state DB size by about 5% by not storing commonly occurring patterns.

• Lowered memory consumption. On the Mainnet, reduces memory usage was reduced by about 1 GB after resync or full pruning.

• Faster sync due to auto-pivot

  From initialization to snap sync
  v1.17.4	1h 7m 30s
  v1.18.0	28m 22s
  v1.19.0 without auto-pivot	1m 30s
  v1.19.0 with auto-pivot	10s

  State sync will start almost immediately after starting Nethermind. The pivot block will be updated to the one close to the chain head based on the message from the consensus layer. Thus, there’s no need to download a significant amount of the newest blocks before starting state sync.
  By default, auto-pivot functionality in this version will wait for ~15 minutes for CL to send us FCU based on which Pivot will be selected (in the future will be extended). If for any reason FCU will not arrive at Nethermind in that time(no checkpoint sync on the CL side, issue with CL configuration, etc.) you can increase this value by adding the flag --Sync.MaxAttemptsToUpdatePivot=1800 (900 is the default value - 1 attempt = ~1 second, so for 1800 it will try to update the pivot for a total of 30 minutes).

Changelog

Cancun

• Add JSON-RPC endpoints for EIP-4844, needed to exchange blobs with the consensus layer by @flcl42 in #5558
• Refactor transaction broadcasting for a needs of EIP-4844 by @marcindsobczak in #5485 #5619
• Change blob transaction type value to 3 by @flcl42 in #5597

Metrics

• Prometheus metrics improvements by @deffrian in #5570
• Add tags to every metric by @deffrian in #5625
• Report metrics for DB memory size by @benaadams in #5636

Bug fixes and stability

• Health Checks now will give more useful information about the current syncing status, possible problems during the sync process, and better recognition if a node is healthy by @deffrian in #5630
• Allow specifying concurrent new connection count and connect timeout by @asdacap in #5676
• Fix/transactionhash not threadsafe by @asdacap in #5634
• Use Environment.TickCount64 rather than StopWatch.GetTimestamp by @benaadams in #5575
• Zero total difficulty in FindBlock by @deffrian in #5581
• Modernise C# by @benaadams in #5607
• Set pivot to null if snap and fast sync are disabled by @deffrian in #5454
• Fix negative blockNumber exception by @deffrian in #5609
• Shouldn't Wait() on timer threads by @benaadams in #5497
• Allow specifying local ip. And use any by default instead of loopback. by @asdacap in #5635
• Add config for eth_getLogs max block depth by @asdacap in #5652
• Fix eth capabilities for archive nodes by @marcindsobczak in #5655
• Fix misplaced arguments by @deffrian in #5668

Logging

• Revise RocksDB loading failure message by @rubo in #5568
• Enr log Warn -> Debug by @MarekM25 in #5623
• Align only in peer report by @deffrian in #5667

Performance

• Perf/eth getlogs with compact encoding by @asdacap in #5569
• Make sure compact receipt does not get registered as global receipt decoder. by @asdacap in #5586
• Return a couple additional ArrayPoolLists by @benaadams in #5599
• Make some structs readonly by @benaadams in #5604
• Use direct properties rather than Linq by @benaadams in #5605
• Feature/low priority write by @asdacap in #5588
• Refactor HeaderDecoder by @deffrian in #5488
• Fix/Rocksdb cache not set by @asdacap in #5578
• Rename WriteBytesPerSec to BytesPerSec and add direct io flag. by @asdacap in #5641
• Compressing db by @Scooletz in #5498
• Reduce SyncStatus lock contention by @LukaszRozmej in #5647
• Add option to disable low priority writes for full pruning by @asdacap in #5663
• Receipt key with block number by @asdacap in #5651
• Feature/readahead flag by @asdacap in #5645
• Allow configuration for db compression by @asdacap in #5669
• One that touches keccak.bytes by @asdacap in #5675
• Update pivot basing on message from CL by @marcindsobczak in #5256

Other changes

• Remove Nethermind.Native project by @rubo in #5615
• Add CODEOWNERS by @rubo in #5653
• Update submodules by @benaadams in #5611
• Add NUnit Analyzers by @rubo in #5638
• Fix Overseer path for .NET version update and config state by @benaadams in #561
• Removed never called private methods by @benaadams in #5603
• Replace Nethermind.Secp256k1 with a NuGet package by @rubo in #5610
• Refactor and remove obsolete stuff by @rubo in #5614
• Shanghai/Merge Ethereum test fix by @deffrian in #5318
• Boost and fix tests by @rubo in #5656
• Hide from doc by @asdacap in #5654
• temp fix flaky tests by @smartprogrammer93 in #564
• Replace Nethermind.Secp256k1 with a NuGet package by @rubo in #5610
• Replace System.IO.Abstractions with TestableIO.System.IO.Abstractions.Wrappers package by @rubo in #5612
• Replace Environment.Exit with ProcessExitSource and cancelation token by @LukaszRozmej in #5624
• Enable nullability in Api by @deffrian in #5631
• Adding more sepolia bootnodes by @parithosh in #5567

New Contributors

• @parithosh made their first contribution in #5567

Full Changelog: 1.18.0...1.19.0-rc

v1.18.2

Release notes

⚠️
This release revises the timestamp and fork-specific feature activations of the Chiado Shapella hard-fork and is a must-have replacement for any Chiado network operator using the previous v1.18.1.

For the Chiado network operators, this is a high-priority release because of the upcoming Shapella hard-fork. We urge you to update promptly to ensure seamless network functioning post-fork.

For other chains, this is a low-priority update recommended for those syncing from scratch as it significantly accelerates the sync process.

The Chiado Shapella hard-fork is scheduled at 13:17:00 PM UTC on May 24, 2023.

Full changelog: 1.18.0...1.18.2