Leap v4.0.4

Leap v4.0.4 is a critical patch which eliminates a security vulnerability, and also includes bug fixes aimed at enhancing the stability and performance of Leap.

All Antelope nodes should upgrade to a version of leap containing the security patch (v3.1.5, v3.2.4, v4.0.4).

Leap v4.0.4 Release Notes

Security Patch

Eliminate denial of service vulnerability

PRs

• (1397) [3.2 -> 4.0] Merge memory issue fix from release/3.2 to release/4.0

Leap v4.0.4 contains a security patch eliminating a denial of service vulnerability present in all prior versions of Leap.

Bug Fixes

Report transaction failed if trx was exhausted in non-producing mode

PRs

• (1329) [3.2 -> 4.0] Report transaction failed if trx was exhausted in non-producing mode

Summary:

Improved reporting of transaction failures in non-producing mode by immediately retrying them using a speculative block, resulting in faster resolution and reduced waiting time.

Problem:

Transaction failures in non-producing mode were not being reported efficiently, leading to delays in resolving the issues.

Impact:

The previous approach caused delays in addressing transaction failures, resulting in potential disruptions and slower processing.

Changes:

• Added functionality to report transaction failures if exhausted in non-producing mode
• Implemented a mechanism to restart a speculative block for immediate retry instead of waiting for a new block
• Updated terminology and added new members (in_producing_mode() and in_speculating_mode()) for improved clarity
• Removed unnecessary comments based on PR review feedback

Resolution:

With the implemented changes, transactionfailures in non-producing mode are now reported promptly, and the system initiates immediate retries using a speculative block. This reduces waiting time and improves the efficiency of resolving transaction failures.

Fixed incorrect serialization scenario

PRs

• (1364) [3.2] Fix incorrect serializing of std::optional when value is not provided
• (1373) [3.2 -> 4.0] Fix incorrect serializing of std::optional when value is not provided

Summary:

Fixed incorrect serialization of std::optional fields in the AntelopeIO/leap repository. The issue occurred when a value was not provided for a std::optional field during serialization, resulting in missing flags. This PR addresses the problem by adding the necessary flags to properly serialize missing values.

Problem:

Incorrect serialization of std::optional fields when a value is not provided.

Impact:

The missing flags during serialization of std::optional fields led to incorrect representation of missing values.

Changes:

• Added flags to the serialization process to indicate missing values in std::optional fields.
• Updated the ABI serializer and test cases.
• Made changes to the libraries/libfc/include/fc/time.hpp and unittests/abi_tests.cpp files.

Resolution:

Missing values in std::optional fields are correctly serialized, addressing the issue and preventing incorrect representation.

Close connection on aysnc_read with a closed socket

PRs

• (1366) [4.0] Close connection on aysnc_read with a closed socket

Summary:

Improved socket shutdown and cleanup logic in version 4.0 of AntelopeIO/leap to address the issue of closing a connection on async_read with a closed socket. The changes resolve frequent p2p connection drops observed in version 4.0.3.

Problem:

Closing a connection on async_read with a closed socket resulted in frequent p2p connection drops.

Impact:

Frequent p2p connection drops impacted the stability and reliability of the system.

Changes:

• Implemented a more paranoid socket shutdown when closing and handling async_read with a closed socket.
• Improved logic for connection duplicate check cleanup.
• Reset the organization on close to ensure correct sending of time messages in new connections.

Resolution:

The changes ensure proper closure of connections and improve the stability and reliability of the system, resolving the issue of frequent p2p connection drops.

Support snapshot start with full deltas

PRs

• (1375) [4.0] SHiP: Support snapshot start with full deltas

Summary:

Support state_history_plugin clients when starting from a snapshot. Clients can now connect and receive the full delta of a snapshot without consuming additional blocks.

Problem:

Starting Chronicle from a ship endpoint after a snapshot resulted in errors. The Chronicle did not receive the whole state first, which caused issues with populating its ABI database.

Impact:

Clients connecting to state_history_plugin when starting from a snapshot experienced difficulties in receiving the complete snapshot delta.

Changes:

• Fixed the issue with ship initial block in Leap 4.0.3.
• Support for state_history_plugin clients starting from a snapshot.
• Clients now receive the full delta of a snapshot without consuming additional blocks.
• Improved handling of requested blocks that are not available.

Resolution:

Clients can now successfully connect to state_history_plugin and receive the full delta of a snapshot without any additional block consumption.

Other Changes

Emit Correct Trace Id for Deferred Transactions Before the On-Chain ACtivation of Replaced-Defered.

PRs

• (1381) Emit correct trace id for deferred trx before replace_deferred protocol feature activation

Summary

Before protocol feature replace_deferred is activated the transaction id of the scheduled transaction can differ from the packed_transaction that is executed. This PR restores the behavior of tracking and reporting in transaction traces the scheduled transaction id instead of the packed_transaction id. This was not a consensus error but rather the wrong trx id was being reported in transaction traces for scheduled transactions before the on-chain activation of replaced_deferred. This manifested itself by confusing SHiP in recording the scheduled transactions reported in a block.

Improve startup when large number of SHiP logs in retain directory

PRs

• (1384) [4.0] SHiP: Improve startup when large number of SHiP logs in retain directory

Summary

Startup of state_history_plugin was very slow when a large number of SHiP logs were in the retain directory. This PR includes other performance improvements. Taken together these improvements resulted in a one order of magnitude improvement in benchmarked performance.

Changes to Logging

PRs

• (1345) [4.0] P2P Fix head_num reporting

Summary:

In this release of leap (v4.0.4), we have introduced better reporting for peer to peer connections.

Changes:

In some cases, head_num was incorrectly reported as 0, even though the unlying value was correct.

Benefits:

With this change block producers can see the correct state of their peers.

Changes to Pinned Builds

PRs

• (1338) [3.2 -> 4.0] Pinned Builds manual dispatch workflow in CI

Summary:

In this release of leap (v4.0.4), we have introduced a new manual dispatch workflow for Pinned Builds in CI. This workflow allows for more control and flexibility in running Pinned Builds using GitHub Actions.

Changes:

To enable the manual dispatch workflow for Pinned Builds, the following actions were taken in leap v4.0:

1. Build-script changes to enable manual dispatch.
2. Pinned builds now run seamlessly in GitHub Actions, ensuring reliable and consistent execution.
3. The artifact name for Pinned Builds has been to improve clarity and organization.

Benefits:

The introduction of the manual dispatch workflow for Pinned Builds brings the following benefits:

1. Pinned builds can now be executed in a controlled manner.
2. Improved integration with GitHub Actions for build automation.
3. Enhanced traceability and visibility of Pinned Build execution.

Changes to Testing

PRs

• (1387) [4.0] Test: Reduce load on CI/CD

Summary

Reduced the number of generated and expected transactions in Continuous Integration testing to reduce testing load, and increase reapability of tests.

Changes to Documentation

PRs

• (1350) [4.0] Documentation, additional nodeos help information
• (1368) [4.0] Removed "deprecated" from help for speculative read-mode

Summary:

Enhanced documentation for nodeos --help by adding additional information for the transaction-retry-interval-sec and transaction-retry-max-expiration-sec options....

Leap v3.2.4

Leap v3.2.4 is a critical patch which eliminates a security vulnerability, and also includes bug fixes aimed at enhancing the stability and performance of Leap.

All Antelope nodes should upgrade to a version of leap containing the security patch (v3.1.5, v3.2.4, v4.0.4).

Leap v3.2.4 Release Notes

Security Patch

Eliminate denial of service vulnerability

PRs

• (1347) [3.1 -> 3.2] Merge memory issue fix from release/3.1 to release/3.2

Leap v3.2.4 contains a security patch eliminating a denial of service vulnerability present in all prior versions of Leap.

Bug Fixes

Don't check signatures on dry run transactions

PRs

• (1089) [3.2] fix: not fail on signature checks for dry-run txns

Summary:

Signature checks for dry-run transactions were failing.

Impact:

The previous code failed to handle signature checks correctly for dry-run transactions, leading to unnecessary authorization checks. This issue could potentially result in erroneous behavior and wasted computational resources.

Resolution:

The following actions have been taken to resolve the issue:

• Modified the code to ensure that signature checks do not fail for dry-run transactions.
• Refactored the authorization manager to properly handle dry-run transactions.
• Backported the fixes to the 3.2 release of the software.
• Assigned appropriate milestones and labels to track the progress of the issue.

These improvements ensure that signature checks for dry-run transactions are correctly handled.

Fix i256 key type conversion by initialize buffer before usage

PRs

• (1264) [3.2] fix i256 key type conversion by initialize buffer before usage

Summary:

There was an issue related to the i256 key type conversion.

Impact:

The previous code had a bug where the export_bits function did not fill every byte in the buffer. This issue could potentially cause unpredictable behavior and incorrect results in code that used the i256 key type conversion.

Resolution:

The following actions have been taken to resolve the issue:

• Modified the code to initialize the buffer before usage, ensuring that every byte is properly filled during the export_bits operation.
• Backported the fix from pull request #1252 to the release/3.2 branch.

Report transaction failed if trx was exhausted in non-producing mode

PRs

• (1319) [3.2] Report transaction failed if trx was exhausted in non-producing mode

Summary:

Improved handling of exhausted transactions in non-producing mode.

Impact:

In previous versions, when running in non-default --read-mode speculative, exhausted transactions were not retried until a new block was received. This resulted in delays and potential inefficiencies in transaction processing.

Resolution:

To address this issue, the following changes were implemented:

• Modified the code to restart a speculative block immediately when a transaction is exhausted in non-producing mode, instead of waiting for a new block.
• Updated the logic in the process_incoming_transaction_async function to correctly handle exhausted blocks based on the pending block mode.
• Introduced the in_producing_mode() and in_speculating_mode() members to enhance transaction handling.
• Merged the changes into the release/3.2 branch.

These enhancements ensure that exhausted transactions in non-producing mode are promptly retried, improving the responsiveness and efficiency of transaction processing in Leap v3.2.

leap-util: Require output-dir for extract-blocks subcommand

PRs

• (1262) [3.2] Output-dir should be required for extract-blocks subcommand of leap-util

Summary:

Leap v3.2 requires the output-dir parameter for the extract-blocks subcommand of leap-util block-log.

Impact:

Previously, the output-dir parameter was not enforced for the extract-blocks subcommand, leading to potential filesystem errors and cryptic error messages. Users were not prompted to provide the output directory for the block log extracted from the specified blocks directory.

Resolution:

To address this issue, the following actions have been taken:

• Modified the blocklog_actions::setup function in the programs/leap-util/actions/blocklog.cpp file to make the --output-dir option required for the extract-blocks subcommand.
• Updated the code to enforce the requirement of the output-dir parameter, ensuring that users must provide the output directory.
• Merged the changes into the release/3.2 branch of the AntelopeIO/leap repository.

These improvements ensure that the output-dir parameter is mandatory for the extract-blocks subcommand of leap-util block-log, preventing filesystem errors and providing a clear requirement for users.

Respond to requests with disallowed HTTP HOST header

PRs

• (996) [3.2] response to disallowed host

Summary:

The behavior of the HTTP server in Leap has been improved to provide more informative error responses when the Host header value is not allowed.

Impact:

In Leap version 3.1, when the Host header value was not allowed by the http_plugin, an empty 400 response was returned to the client. However, in version 3.2 and onwards, the connection was simply closed without sending any headers to the client. This behavior change caused confusion and lacked proper error reporting.

Resolution:

The resolution for the issue involved modifications to the http_plugin:

• If the Host header is not allowed, a bad request error response is sent to the client.
• The response includes the HTTP status code and a descriptive message with details of the error.

These improvements ensure that when the Host header value is not allowed, an error response is sent to the client, providing valuable information about the error. The changes enhance the usability and clarity of the HTTP server behavior in Leap.

Fixed incorrect serialization scenario

PRs

• (1364) [3.2] Fix incorrect serializing of std::optional when value is not provided

Summary:

Fixed incorrect serialization of std::optional fields in the AntelopeIO/leap repository. The issue occurred when a value was not provided for a std::optional field during serialization, resulting in missing flags. This PR addresses the problem by adding the necessary flags to properly serialize missing values.

Problem:

Incorrect serialization of std::optional fields when a value is not provided.

Impact:

The missing flags during serialization of std::optional fields led to incorrect representation of missing values.

Changes:

• Added flags to the serialization process to indicate missing values in std::optional fields.
• Updated the ABI serializer and test cases.
• Made changes to the libraries/libfc/include/fc/time.hpp and unittests/abi_tests.cpp files.

Resolution:

Missing values in std::optional fields are correctly serialized, addressing the issue and preventing incorrect representation.

Other Changes

Changes to Logging

PRs

• (1000) [3.2] Log all http request/response at debug log level
• (1010) [3.2] Reduce producer_plugin debug level log spam

Log all HTTP request/response at debug log level

Summary

This pull request introduces the feature of logging all HTTP request and response information at the debug log level. The log output is limited to 1024 characters to prevent excessive logging.

Impact

Previously, there was no comprehensive logging of HTTP request and response data, making it difficult to debug and analyze issues related to HTTP communication. The lack of detailed logging hindered troubleshooting efforts and affected the overall visibility of the system.

Resolution

The following changes have been made to improve logging of HTTP request and response data:

• Modified the logging mechanism to include HTTP request and response information at the debug log level.
• Implemented a limit of 1024 characters for the logged output to prevent excessive logging and avoid overwhelming the log files.

These improvements provide detailed logging of HTTP request and response data, enhancing the system's visibility and aiding in troubleshooting efforts.

Reduce producer_plugin debug level log spam

Summary

This pull request addresses the issue of excessive debug level log messages generated by the producer_plugin. The modification reduces the log spam and improves the log output efficiency.

Impact:

Previously, the producer_plugin was generating an excessive amount of debug level log messages, leading to log spam. This log spam could negatively impact the system's performance and make it difficult to analyze important log information.

Resolution:

Modified the logging condition in the producer_plugin_impl class to log blocks only every 1000 during synchronization or when the time difference between the current time an...

Leap v3.1.5

Leap v3.1.5 is a critical patch which eliminates a security vulnerability, and also includes bug fixes aimed at enhancing the stability and performance of Leap.

All Antelope nodes should upgrade to a version of leap containing the security patch (v3.1.5, v3.2.4, v4.0.4).

Leap v3.1.5 Release Notes

Security Patch

Eliminate denial of service vulnerability

PRs

• (1393) [3.1] Memory issue fix

Leap v3.1.5 contains a security patch eliminating a denial of service vulnerability present in all prior versions of Leap.

Bug Fixes

net_plugin improve block latency calculation

PRs

• (676) [3.1] net_plugin improve block latency calculation
• (754) [3.1] keosd - Exit program if keosd fails to aquire wallet lock (GH #513)

Summary:

Changed net latency calculation by removing default lower bound of 1. Correctly logs 0 when there is no latency.

Fixed bug causing keosd to hang in some cases

Details:

Remove the +1 to the block latency calculation otherwise even 0ms latency indicates 1 block latency instead of 0. The +1 makes the comparisons of head and msg.head in the if-else off by 1 when there is no latency.

Added additional logging so that issues like #588 are easier to debug.

Before:

debug 2022-12-21T21:54:29.938 net-3     net_plugin.cpp:1790           recv_handshake       ] ["localhost:9878 - f9bdaf6" - 43 127.0.0.1:52370] Network latency is 0ms, 0 blocks discrepancy by network latency, 1 blocks discrepancy expected once message received

After:

debug 2023-01-31T00:27:27.386 net-0     net_plugin.cpp:1795           recv_handshake       ] ["localhost:9878 - 978ab39" - 1 127.0.0.1:54362] Network latency is 0ms, 0 blocks discrepancy by network latency, 0 blocks discrepancy expected once message received

Correct --terminate-at-block

PRs

• (684) [3.1] Fix --terminate-at-block to stop at correct block

Summary:

Fixed an issue where the --terminate-at-block option was terminating the node at the next block instead of the requested block in head mode. The test functionality was also reworked to improve its effectiveness and reduce test run time.

Problem:

--terminate-at-block was terminating the node at +1 requested block in head mode.

Impact:

The incorrect termination behavior of --terminate-at-block in head mode affected the accuracy of block synchronization and potentially led to unexpected block processing.

Changes:

• Fixed the code to ensure that the node terminates at the correct block when --terminate-at-block is set.
• Reworked the test functionality to better test the terminate-at-block functionality and improve test run time.

Resolution:

The issue has been resolved, and the --terminate-at-block option now correctly terminates the node at the specified block in head mode, ensuring accurate block synchronization.

Subjective Account Max Failure Window Size

PRs

• (777) [3.1] subjective-account-max-failures-window-size

Summary:

Added configuration option for setting the window size in blocks for subjective account max failures in Leap v3.1.

Problem:

Prior to this update, there was no configuration option to customize the window size for subjective account max failures.

Impact:

Users were unable to adjust the time frame for subjective account blocks, limiting their flexibility in managing failures.

Changes:

Introduces a new configuration option, "--subjective-account-max-failures-window-size," to set the window size in number of blocks.

Resolution:

This update empowers users to define the time window for subjective account blocks according to their specific requirements, providing greater control over the handling of failures in Leap v3.1.

Test Fixes

PRs

• (667) [3.1] Test fix: nodeos_forked_chain_lr_test

Summary:

Fixed an issue in the nodeos_forked_chain_lr_test test in version 3.1 of the AntelopeIO/leap software. The problem was related to the incorrect updating of preKillBlockNum and preKillBlockProducer, which affected the validation calculation. The issue has been resolved, and the test now runs without errors.

Problem:

Incorrect updating of preKillBlockNum and preKillBlockProducer in nodeos_forked_chain_lr_test.

Impact:

The problem affected the validation calculation in the nodeos_forked_chain_lr_test, potentially leading to incorrect results.

Changes:

• Updated the assignment of preKillBlockNum and preKillBlockProducer in the nodeos_forked_chain_test.py file.

Resolution:

The issue has been fixed, and the nodeos_forked_chain_lr_test now runs without errors, ensuring accurate validation calculations.

Changes to Pinned Builds

PRs

• (1310) [3.1] CMAKE_INSTALL_PREFIX=/usr for pinned builds in CI
• (1291) [3.1] Pinned Builds manual dispatch workflow in CI

Summary:

This release introduces a new manual dispatch workflow for the [3.1] Pinned Builds feature in AntelopeIO's leap project. The workflow runs Pinned Builds in GitHub Actions and addresses an issue related to the pinned_build.sh script and artifact uploads. It also includes improvements in the selection of build platforms and artifact naming.

Problem:

The previous workflow for Pinned Builds in leap had issues with the execution of the pinned_build.sh script and the naming of artifacts.

Impact:

The problems with the Pinned Builds workflow affected the execution and naming of artifacts, potentially causing confusion and hindering the use of Pinned Builds.

Changes:

• Created a shell for the initial pinned_build.yaml workflow, enabling the merge of the [3.1] Pinned Builds manual dispatch workflow.
• Modified the workflow matrix to include different build platforms (ubuntu18, ubuntu20, ubuntu22).
• Adjusted the container selection logic in the workflow matrix.
• Removed the redundant execution of cpack in the pinned_build.sh script.
• Fixed the syntax in the workflow and addressed a forgotten run command.
• Modified the artifact upload step to exclude the leap-dev.deb artifact.
• Removed the unused job "Discover Platforms" from the workflow.
• Simplified the artifact naming and path matching.
• Used a self-hosted runner for the Pinned Builds workflow.
• Excluded the resmon plugin test in ubuntu20 due to its flakiness.

Resolution:

After these changes, the Pinned Builds workflow in leap now runs smoothly in GitHub Actions, resolving the issues with script execution and artifact naming. The workflow ensures consistent and reliable Pinned Builds, improving the development process in leap.

Changes to Logging

PRs

• (700) [3.1] Fix reporting of dirty db

Summary:

Fixed issue with dirty database being reported as "genesis not found" instead of "dirty database".

Problem:

Dirty database reported as "genesis not found" instead of "dirty database."

Impact:

Misleading error message and incorrect reporting of the database state.

Changes:

• Propagated all exceptions except system_error.
• Added a comment explaining the non-propagation of system_error.
• Updated reporting of dirty database.

Resolution:

Dirty database is now correctly reported, improving error handling and providing accurate information about the database state.

Changes to Documentation

PRs

• (747) [3.1] Backport to 3.1 of bootstrap new protocol features activation
• (773) [3.1] Move subjective bill log to all level of producer_plugin logger.

Summary:

Fixed log messages in transaction_success_tracing and transaction_failure_tracing to improve relevancy and reduce CPU load and network bandwidth during transaction tracing.

Problem:

Log messages in transaction_success_tracing and transaction_failure_tracing were not entirely relevant to transaction tracing, leading to higher CPU load and network bandwidth.

Impact:

The unwanted log messages caused increased CPU load and network bandwidth, affecting the performance of the transaction tracing user interface.

Changes:

• Moved the log messages from transaction_success_tracing and transaction_failure_tracing to the producer_plugin logger.
• Adjusted the log level of the messages to the all (trace) level for better traceability.

Resolution:

Log messages in transaction_success_tracing and transaction_failure_tracing have been moved to the producer_plugin logger, improving relevancy and reducing CPU load and network bandwidth during transaction tracing.

Further details on changes since last release

Contributors

Special thanks to the contributors that submitted patches for this release:

• @heifner
• @jolly-fellow
• @greg7mdp
• @huangminghuang
• @spoonincode
• @oschwaldp-oci
• @linh2931
• @vladtr
• @mikelik
• @dimas1185

Full list of chan...

Leap v4.0.3

Leap v4.0.3 is a patch release, Leap v4.0.3 resolves an issue that could cause premature closure of idle connections in some scenarios (details provided below).

... and of course it builds on the great optimizations introduced in v4.0.0: https://github.com/AntelopeIO/leap/releases/tag/v4.0.0

... and also builds on the stability improvements in prior v4.x.x patch release:

• Leap v4.0.1: https://github.com/AntelopeIO/leap/releases/tag/v4.0.1
• Leap v4.0.2: https://github.com/AntelopeIO/leap/releases/tag/v4.0.2

Leap v4.0.3 Release Notes

Bug Fixes

There is only one significant change in this release:

Use magnitude safe time types

PR

• (1321) [4.0 -> main] P2P: Use magnitude safe time types

Issue:
Some node operators running leap 4.0.2 reported an issue where p2p connections were constantly disconnecting. The culprit was inaccurate timing of heartbeat calculations which could cause premature closure of idle connections.

Impact:
Constant p2p connection interruptions creates the potential for network instability. The issue was observed and addressed before any issues could propogate to any Antelope main net.

Due to variations in clock tick time precision across different hardware/OS environments, this issue only affected some nodes. Notably, this issue did NOT affect nodes running on Ubuntu 20.

Resolution:
PR #1321 addressed the issue by introducing magnitude safe time types. The problematic "tstamp" type was completely removed and replaced with std::chrono types to ensure accurate and consistent time calculations. The use of magnitude safe time types helps maintain the correct timing for heartbeat calculations and prevents premature closure of idle connections.

The following changes were made:

1. Use magnitude safe time types
2. Reset latest_msg_time and latest_blk_time
3. Completely remove problematic tstamp type

Further details

Contributors

Special thanks to the contributors that submitted patches for this release:

• @spoonincode
• @heifner
• @Lin2931

Full list of changes since last release

PR

• (1321) [4.0 -> main] P2P: Use magnitude safe time types
• (1327) [4.0 -> main] Merge release/4.0.3 version bump

Full Changelog: v4.0.2...v4.0.3

Leap v4.0.2

Leap v4.0.2 is a patch release, fixing defects discovered in features added in the recent v4.0.1 release. Changes include several bug fixes aimed at enhancing the stability and performance of the AntelopeIO/leap C++ implementation.

The key updates in this release include:

• Addressed the problem of excessive info logs generated by Nodeos, causing syslog size increase.
• Resolved an issue with startup segfault after clean shutdown.
• Resolved the issue of starting a node from a snapshot without blocks.
• Fixed intermittent missing blocks in the state history reader.

Leap v4.0.2 Release Notes

Key Changes

Start node from snapshot without blocks

PRs

• (1285) [4.0] Fix for loading snapshot with empty block log
• (1299) [4.0] Fix for loading a snapshot with an empty block log but an existing fork database

Issue:
In Leap v4.0.1, an issue was identified where attempting to start a node from a snapshot that had no blocks in the blocks folder resulted in an error. Despite trying different snapshots and various configurations, the error persisted.

Impact:
The inability to start a node from a snapshot without blocks required manual workarounds to ensure the node could start properly and could prevent some node operators from upgrading to Leap v4.x.x.

Resolution:
The issue has been resolved in Leap v4.0.2 with the following fixes:

1. Added a test for reading a snapshot with no block log option.
2. Improved assertion message for better diagnostics.
3. Fixed the empty block log to support loading from a snapshot.
4. Implemented additional tests for empty block log scenarios.
5. Handled the loading of a snapshot without a block log but with an existing fork database.
6. Ensured correct handling of forks during snapshot loading.

Intermittent Missing Blocks in State History Reader

PRs

• (1276) [4.0] SHiP: fix intermittent empty get_block_result

Issue:
In Leap v4.0.1, a state history reader was tested with a fork case and encountered a problem where it would get backed up and miss blocks after a certain point. Analysis revealed a null value for "this_block" in the ship plugin log, indicating an issue with block retrieval.

Impact:
The intermittent missing blocks had the following implications:

1. Incomplete state history data.
2. Disruption in data analysis and processing.
3. Potential inconsistencies in system behavior.

Resolution:
To address this issue, the following actions were taken in leap v4.0.2:

1. Fixed the intermittent empty get_block_result
2. Merged the fix into main

Prevent rapid increase in syslog size caused by excessive info logs

PRs

• (1267) [4.0] excessive logging caused by message spamming

Summary:
In Leap (v4.0.1), Nodeos was observed to generate an excessive number of info logs, resulting in a substantial increase in the size of the syslog.

Impact:
The syslog size could grow up to 5GB within an hour, significantly affecting system performance and disk space utilization. The excessive info logs generated by nodeos had the following potential impacts on the system:

1. Potential infinite loop in the net_plugin.
2. Consumption of disk space.
3. Degraded system performance.
4. Increased log noise.

Resolution:
To address this issue, the following actions have been taken in leap v4.0.2:

1. Resolved an infinite loop in the net_plugin by preventing the sending back of a handshake message under specific conditions
2. Improved the close process handling when not in sync to mitigate excessive logging
3. Implemented enhanced timestamp handling and round-trip time calculations to reduce log entries and avoid message spamming

Fix startup segfault after clean shutdown

PRs

• (1221) [4.0] allow ROtrx threads unlimited time to start

Issue:
In the previous version of Nodeos (v4.0.1), an issue was identified where the application would segfault during startup after being cleanly stopped for backup.

Impact:
Symptoms of the issue included the generation of multiple info logs related to net_api_plugin and http_plugin.

Resolution:

1. Code change in v4.0.2: allow ROtrx threads unlimited time to start.
2. Suggested workaround: set read-only-threads = 0 if chain_api_plugin is enabled on a non-public api node.

state-history-log-retain-blocks option handling

PRs

• (1241) [4.0] SHiP: Fix state-history-log-retain-blocks option handling

Issue:
In Nodeos version 4.0.1, an error occurs during the plugin initialization of the state_history_plugin: "Plugin exception: state history log prune configuration requires at least one block."

Impact:
The startup error hinders the proper initialization of the state_history_plugin.

Resolution:
The issue was identified as a bug in handling the state-history-log-retain-blocks option. To resolve this issue, the following actions were taken in leap v4.0.2 through pull requests:

1. Fix state-history-log-retain-blocks option handling
2. Merge changes into the main branch

Fix parsing Error in get_transaction_id Endpoint for CURL Requests

PRs

• (1280) [4.0] if actions.data & actions.hex_data provided, use the hex_data in API call get_transaction_id

Issue:
In the previous version of leap (v4.0.1), an issue was identified in the get_transaction_id endpoint when making CURL requests. Specifically, when sending a transaction JSON similar to the one accepted by cleos, a parsing error occurred.

Impact:
The parsing error resulted in the following impacts:

1. Failed CURL requests with a parsing error message: "Unable to parse valid input from POST body."
2. Inability to retrieve valid transaction IDs through CURL requests, leading to disruptions in transaction processing and related operations.

Resolution:
To resolve this issue, the following actions were taken in leap v4.0.2:

1. Accept both data and hex-data, aligning with the behavior of cleos.

Please note that these improvements are included in the Leap 4.0.2 release, providing users with a reliable and consistent experience when utilizing CURL requests for transaction processing.

Occurrences of "peer sent a handshake with a timestamp skewed by at least -2ms" persist

PRs

• (1303) [4.0] P2P: Do not log small clock skew

Issue:
Despite applying the patch for issue #1240 on top of Leap 4.0.1, the occurrence of log messages indicating timestamp skew persisted.

Impact:
The continuous log messages caused confusion and added noise to the system logs. The actual clocks were not skewed, as evidenced by similar block arrival times reported by nodes. However, the misleading log messages raised concerns and created unnecessary alarm.

Resolution:
To address this issue, the following actions were taken in Leap 4.0.2:

1. Refactored the logging behavior to reduce warning messages for insignificant clock skew that does not impact system operations.

Further details

Contributors

Special thanks to the contributors that submitted patches for this release:

• @iamveritas
• @spoonincode
• @heifner
• @766C6164
• @heifner
• @Lin2931

Full list of changes since last release

PRs

Bug Fixes:

• (1207) [4.0] remove innocuous stray } in log statement
• (1226) [4.0] fix NP & LR CI workflow for forked/mirrored repos that are not named leap
• (1221) [4.0] allow ROtrx threads unlimited time to start
• (1241) [4.0] SHiP: Fix state-history-log-retain-blocks option handling
• (1266) [3.2 -> 4.0] fix i256 key type conversion by initialize buffer before usage
• (1267) [4.0] P2P excessive logging caused by message spamming
• (1276) [4.0] SHiP: fix intermittent empty get_block_result
• (1285) [4.0] Fix for loading snapshot with empty block log
• (1280) [4.0] if actions.data & actions.hex_data provided, use the hex_data in api call get_transaction_id
• (1299) [4.0] Fix for loading a snapshot with an empty block log but an existing fork database
• (1303) [4.0] P2P: Do not log small clock skew

Documentation:

• (1204) [4.0] add docs/yaml content for missing chain api endpoints

Version Update:

• (1307) [4.0] Bump Leap version to 4.0.2

Full Changelog: v4.0.1...v4.0.2

Leap v4.0.1

Leap v4.0.1 Release Notes

Leap v4.0.1 is a patch release, fixing defects discovered in features added in the recent v4.0.0 release. This release includes fixes for:

• Failing signature checks for dry run transactions
• Failing replay when using a partitioned block log
• Excessive memory consumption during replay
• Other minor updates

Leap v4.0.1 Release Notes

Bug fixes

Fix for failing signature checks on dry run transactions

PRs

• (1089) [3.2] fix: not fail on signature checks for dry-run txns
• (1090) [3.2 -> 4.0] fix: not fail on signature checks for dry-run txns

Signature checks were erroneously being performed for dry run transactions. These unnecessary checks always failed since dry run transactions are never signed.

Dry run transactions are now excluded from signature checks, avoiding failures. To validate the correction, we have included a new test case called dry_run_auth_test. The test simulates dry run transactions, ensuring that they are not subject to unnecessary signature verification.

This change makes dry run transactions more reliable.

Fix replay of partitioned block log

PRs

• (1164) [4.0] fix replay for partitioned block log

The replay of partitioned block logs was failing.

To resolve the issue, several fixes were implemented including addressing the non-existent retained directory, fixing file permissions, ensuring the correct genesis was supplied, refactoring the extract_chain_context function, fixing the leap util, and removing unused includes. These changes collectively resolved the block replay problem with partitioned block logs.

This change improves the reliability block replay for a partitioned block log.

Fix for excessive memory consumption during blocks log replay

PRs

• (1191) [4.0] Fix for nodeos large memory consumption during a blocks log replay

During block log replay, nodeos consumed excessive memory, leading to memory exhaustion and causing potential performance degradation and instability.

This change significantly reduces memory consumption and improves the stability and performance of nodeos during block log replay.

Handle close while syncing node

PRs

• (1171) [4.0] Correctly handle close while syncing

When generating snapshots during syncing, nodeos would timeout a sync and close the connection. This would cause all queued blocks on that connection to be dropped and sync_next_expected_num to not be reset.

nodeos now correctly handles the close event during syncing.

This change improves the stability and reliability of the syncing process.

Fix shutdown caused by remote endpoint http errors

PRs

• (1175) [4.0] Ignore http error on remote_endpoint() causing shutdown

The remote_endpoint() call was generating an error "Transport endpoint is not connected," which caused the http thread pool to exit due to an uncaught exception, resulting in a shutdown.

To address this issue, the error generated by the remote_endpoint() call has been ignored, and "unknown" is logged for the remote endpoint instead. This prevents the http thread pool from exiting unexpectedly.

Other changes

Block log changes

PRs

• (1154) [4.0] Added block log version and num of blocks to output of leap-util block-log smoke-test
• (1141) [4.0] change max-retained-history-files default

In previous leap versions eosio-blocklog reported block log version and number of blocks when invoking smoke-test. The newer leap-util did not report block log version or number of blocks.

leap-util block-log smoke-test now returns block log version number and number of blocks similar to prior versions of eosio-blocklog.

Better read only option handling

PRs

• (1129) [4.0] Check read-only trx options only when read-only-thread enabled, and present better error message

Read only transaction options are now validated only when the number of read only threads is configured greater than 0. The error message for an invalid -read-only-read-window-time-us has also been improved.

Documentation Updates

PRs

• (849) Fix README.md Formatting
• (1105) 4.0: Documentation fixed Error path in producer plugin swagger
• (1087) [4.0] Update swagger files to include missing doc

Some minor changes were made to documentation and swagger output formatting.

Further details on changes since last release

Contributors

Special thanks to the contributors that submitted patches for this release:

• @766C6164
• @arhag
• @ericpassmore
• @heifner
• @huangminghuang
• @iamveritas
• @kj4ezj
• @linh2931

Full list of changes since last release

PRs

• (849) Fix README.md Formatting

• (1105) 4.0: Documentation fixed Error path in producer plugin swagger

• (1087) [4.0] Update swagger files to include missing doc

• (1089) [3.2] fix: not fail on signature checks for dry-run txns

• (1090) [3.2 -> 4.0] fix: not fail on signature checks for dry-run txns

• (1129) [4.0] Check read-only trx options only when read-only-thread enabled, and present better error message

• (1154) [4.0] Added block log version and num of blocks to output of leap-util block-log smoke-test

• (1141) [4.0] change max-retained-history-files default

• (1164) [4.0] fix replay for partitioned block log

• (1191) [4.0] Fix for nodeos large memory consumption during a blocks log replay

• (1171) [4.0] Correctly handle close while syncing

• (1175) [4.0] Ignore http error on remote_endpoint() causing shutdown

• (857) Simplify Pinned Build Scripts

• (1193) [4.0] Revert "Simplify Pinned Build Scripts"

• (1201) [4.0] Bump Leap to 4.0.1

Full Changelog: v4.0.0...v4.0.1

Leap v4.0.0

Antelope Leap v4.0.0 release focuses on optimizing node performance, improving block propagation, scheduling future snapshots, adding read-only transactions, and improving performance of EVM related functions.

Leap v4.0.0 Release Notes

New features

Read-Only Transactions

PRs

• (558) Introduce new read-only transaction RPC endpoint

We have introduced a RPC endpoint for sending read-only transactions. A read-only transaction does not change the state and is not added into the blockchain. It is useful for users to retrieve complex chain information.

The format is the same as regular transaction, except it does not contain authorization section. Mechanisms are in place to ensure actions (including inline actions) in a read-only transaction do not modify the state and deepmind logging is not performed.

The RPC URL is v1/chain/send_read_only_transaction

cleos is updated to send read-only transactions:

• cleos push transaction --read <transaction>
• cleos push action --read <action>

New producer_plugin logging categories are added for transient transactions (dry-run (by compute_transaction) and read-only transactions). They are transient_trx_success_tracing and transient_trx_failure_tracing . Existing transaction_success_tracing and transaction_failure_tracing do not include the logging for transient transactions.

max_action_return_value_size is not enforced for read-only transactions.

Auto-peering with schedule proximal Block Producer nodes

PRs

• (744) Add auto bp peering functionality

We've added a config option to automatically manage connections based on the Block Producer schedule. Node order and peering can impact a node's latency to receive last block of the preceding BP, and to send their last block to the next BP. Ideally, Block Producers:

• Receive the last block of the previous round as soon as possible
• Send their last block to the next Block Producer as soon as possible
• Avoid getting a late start on their block production

To achieve these goals, the following changes were made in v4.0.0:

Correlate BP account names and connection details

--p2p-auto-bp-peer has been added to nodeos to allow a node to correlate BP account names to p2p connection details. When a block producer node with this setting detects a pending schedule change, it will attempt to auto-peer with BPs that have come into schedule proximity, and disconnect from any auto-peered BPs that are no longer in schedule proximity. These auto-peered connections are not subjected to the limit specified by the --max-clients option.

BP Peer flag in net api

is_bp_peer has been added to every connection object in the v1/net/connections api endpoint response.

Snapshot Scheduling API

PRs

• (131) Proposal for a snapshot scheduling API
• (759) Programmable snapshots API

The Snapshot Scheduling API added in Leap v4.0.0 allows users to schedule and manage snapshot requests in advance, increasing reliability and reducing the potential for errors in the snapshot creation process. It also allows for greater flexibility in creating and managing snapshots, as users can specify parameters such as start and end blocks and block spacing.

The API provides endpoints for several features:

• /producer/schedule_snapshot: Add a task to perform snapshots
  • Parameters: start_block, end_block, block_spacing, snapshot_description
• /producer/get_snapshot_requests: Return all scheduled snapshot requests
• /producer/unschedule_snapshot: Remove a previously scheduled snapshot task
  • Parameters: snapshot_request_id

For more details, check out the Snapshot Scheduling API Design Specs.

Prometheus Exporter Plugin

Prometheus Instrumentation & Metrics

PRs

• (537) Instrument producer_plugin for Prometheus

Instrumentation of nodeos plugins to expose data with Prometheus. An initial phase of instrumentation will allow for the evaluation of the usability of the instrumentation when viewed from Prometheus. The initial instrumentation will not include any per account or per contract metrics. Additionally, the ability to expire metrics will not be included in the initial phase. To support dynamic metrics, the metric object for each plugin will include an indicator of when the set of metrics has last changed. The following metrics will be collected for the initial phase of instrumentation:

Net Plugin
gauge - number of clients
gauge - number of peers
gauge - number of dropped blocks

Producer Plugin (Producer plugin metrics will be gathered at the start of each block)
gauge - unapplied transaction queue sizes (number of transactions)
counter - blacklisted transactions count (total)
counter - blocks produced
counter - transactions produced
gauge - last irreversible
gauge - current block num
gauge - subjective billing number of accounts
gauge - subjective billing number of blocks
gauge - scheduled transactions

See the Prometheus Plugin Architecture for more details.

Prometheus on a dedicated IP/port

PRs

• (939) [4.0] prometheus on different port

The prometheus_plugin can run on a dedicated IP and port by setting prometheus-exporter-address and specifying the IP and port to listen for incoming prometheus metrics http request. A simple embedded server handles HTTP requests and exports Prometheus metrics.

NOTE: Future versions will generalize this functionality to allow custom IP/port configuration for categories of HTTP endpoints.

Log Splitting

PRs

• (532) block log partition functionality
• (628) SHiP stability fixes, log splitting, and memory usage improvements

Leap v4.0.0 includes two similar features aimed at giving node operators greater control over how important logs for blocks and state are managed: blocks log splitting and state history log splitting. The key benefit of these features is that it allows users to specify how many split log files to retain, and automates aging out older logs by deleting them or moving them another directory.

Blocks Log Splitting

To make it easy to use block log splitting, we've added some new options for the chain plugin:

• blocks-log-stride allows you to split the block log file when the head block number is the multiple of the stride. When the stride is reached, the current block log and index will be renamed 'blocks--.log/index', and a new current block log and index will be created with the most recent block. All files following this format will be used to construct an extended block log.
• max-retained-block-files alows you to specifiy the maximum number of block files to retain so that the blocks in those files can be queried. When the number is reached, the oldest block file would be moved to an archive directory or deleted if the archive directory is empty. It's important to note that the retained block log files should not be manipulated by users.
• Two new options allow you to configure the locations of blocks log directories, we've added two new options:
  • blocks-archive-dir allows you to specify the location of the blocks archive directory (absolute or relative path). If the value is empty, blocks files beyond the retained limit will be deleted. Files in the archive directory won't be accessed by nodeos anymore.
  • blocks-retained-dir allows you to specify the location of the blocks retained directory (absolute or relative path). If the value is empty, it is set to the value of the blocks dir.

Please note that all of the newly added options cannot be specified together with the --block-log-retain-blocks option. Overall, we believe that the Blocklog splitting feature and its associated options will help you better manage your block logs and improve the efficiency of your nodeos instance.

SHiP Log Splitting

To make it easy to use SHiP Log Splitting, we've added some new options for the State History Plugin:

• state-history-stride allows you to split the state history log file when the block number is the multiple of the stride. When the stride is reached, the current history log and index will be renamed *-history--.log/index, and a new current history log and index will be created with the most recent blocks. All files following this format will be used to construct an extended history log.
• max-retained-history-files allows you to specify the maximum number of state history log file groups to retain so that the blocks in those files can be queried. When the number is reached, the oldest log files would be moved to an archive directory or deleted if the archive directory is empty. The retained log files should not be manipulated by users.
• state-history-retained-dir allows you to specify the location of the state history retained directory (absolute path or relative to state-history dir). If the value is empty, it i...

Leap v4.0.0-rc3

The 4.0 release of Antelope Leap focuses on optimizing node performance, improving block propagation, scheduling future snapshots, adding read-only transactions, and improving performance of EVM related functions.

Antelope Leap v4.0.0-rc3 includes bug fixes, numerous updates, and several improvements. Key features include:

• The ability to specify a dedicated port for Prometheus metrics
• Support for -k / --kill
• Restoration of read-mode=speculative

Leap v4.0.0-rc3 Release Notes

New Features

Prometheus on a dedicated port

PRs

• (939) [4.0] prometheus on different port

The prometheus_plugin added a new URL for prometheus metrics, but it used the same port as the producer plugin, which needs to be restricted. The prometheus export needs to run on a separate port. In v4.0.0-rc3, we added a new option, prometheus-exporter-address, to specify the IP and port to listen for incoming prometheus metrics http request. A simple embedded server handles HTTP requests and exports Prometheus metrics. Now the prometheus exporter will run on a dedicated port.

Parallelize Read-only Opreations

PRs

• (901) [4.0] Execute read only safe tasks on read-only thread pool

Majority of Chain RPC get_* requests are safe to run in parallel with themselves and read-only transactions. To further improve API nodes throughputs, they are now executed on the read-only thread pool during the read-only-read-window-time-us window.

/v1/net/* calls were also moved to http thread pool, off main app thread.

--read-only-threads defaults to 3 on non-producer configured nodeos that enable the eosio::chain_api_plugin.
Block relay nodes or canary nodes should consider --read-only-threads=0. Since eosio::chain_api_plugin may be enabled so /v1/chain/get_info is available. Block relay nodes or canary nodes should consider --read-only-threads=0 if they have enabled eosio::chain_api_plugin as the read-only-thread pool is only used for read-only transactions and /v1/chain/get* calls. If read-only-threads are enabled, but not utilized, there is a very small overhead of switching back and forth between the read/write windows.

Note read and write windows are interrupted when new blocks are received.

Restore deprecated read-mode=speculative

PRs

• (986) [4.0] Restore read-mode=speculative

An improved version of read-mode=speculative has been added back to nodeos. The read-mode=head remains the default. Read on for more details.

A feature called read-mode=speculative was removed because it caused confusion. When using this feature, the state of a speculative transaction was shown instead of the actual state of execution. This produced a different state than any other node and caused confusion to users.

To fix simplify things, the read-mode=speculative feature was removed and only two modes remain: HEAD and IRREVERSIBLE. This simplified things for users and reflected the actual state of execution. However, some community members raised potential use cases for the read-mode=speculative feature.

The feature has now been added back as an alternative non-default read-mode. This new restored speculative mode will not have the state of API trxs auto applied at the start of each speculative block. Users will have to re-apply any transactions at each speculative block in order to observe their side-effects. Also, the restored read-mode=speculative will not apply schedule changes, protocol activations, or onblock trxs for a speculative mode into the speculative block which will be aborted when a new block is received from the network.

Support for -k --kill in launcher

PRs

• (921) [4.0] Implement support for -k --kill in launcher

Launcher now honors -k --kill <signal_number> argument. Cluster was updated for the Python launcher but support in the launcher itself was missed. Was causing a Test failure: resource_monitor_plugin_test

Add time summary logging for transient trx processing and read-only threads

PRs

• (885) Add time summary logging for transient trx processing and read-only threads

Before v4.0.0-rc3, it was difficult to keep track of time spent on dry-run, read-only, and other transactions. It was especially challenging to know how long it took to execute read-only transactions when using multiple threads. A new log summary report was introduced that provides details on the time spent executing dry-run and read-only transactions. When the option --read-only-threads is enabled, the report also includes time spent on read-only transactions across multiple threads. It is now easier to monitor the time spent on dry-run and read-only transactions, including execution time of read-only transactions across multiple threads. This will help identify any bottlenecks to the performance of the system.

Updates

Read-only Transactions and Operations Parallelization Hardening

PRs

• (885) Add time summary logging for transient trx processing and read-only threads
• (892) [4.0] Cleanup and prevent thread starvation while in the read window
• (933) Small improvements for parallelizing read-only transactions and tasks
• (947) [4.0] Correct time summary calculation for transient trxs due to change of read-only trxs windows
• (960) [4.0] Use is_write_window in eos-vm-oc main to determine if it is safe to update code cache
• (905) Use a single timer for read and write windows
• (964) [4.0] Added missing calls to wasmifs on read-only threads
• (975) [4.0] Remove global WASM::check_limits and initial memory from instantiate_module
• (982) [4.0] Set an upper limit on read-only-threads of 8
• (984) [4.0] Fix ROtrx dispatched to baseline runtime when compiled code not ready
• (993) [4.0] make eosio_exit impl more direct

A number of improvements and bug fixes are included in this RC release. Main ones are

• Before v4.0.0-rc3, it was difficult to keep track of time spent on dry-run, read-only, and other transactions. It was especially challenging to know how long it took to execute read-only transactions when using multiple threads. A new timing summary report was introduced that provides details on the time spent executing dry-run and read-only transactions. When the option --read-only-threads is enabled, the report also includes time spent on read-only transactions across multiple threads. It is now easier to monitor the time spent on dry-run and read-only transactions, including execution time of read-only transactions across multiple threads. This will help identify any bottlenecks to the performance of the system.
• Previously, certain functions were not being called by wasmifs on read-only threads, which could cause memory leaks. These missing calls were added.
• It was identified a global variable called check_limits could cause issues when accessed by multiple threads at the same time. It was removed.
• An upper limit of 8 read-only threads is enforced, regardless of whether EOS VM OC is enabled or not. This limit was chosen to prevent excessive memory usage for WebAssembly instantiation, and should be a conservative strategy to protect a typical EOS Mainnet nodeos from running out of RAM.
• Read window and write window run exclusively with each other; they do not run at the same time. A single timer is sufficient to control them and makes code simpler.
• When executing a read-only transaction with EOS VM OC is enabled, if VM OC has not yet finished compiling, the code falls back to the baseline runtime, which is not thread-safe. The code has been updated to make sure that when VM OC compile fails, read-only transactions will retry or fail permanently based on failure reasons.
• In previous implementation, a read-only thread would exit if no transactions are available, potentially leaving only one thread to complete all remaining tasks. To prevent this thread starvation, the tasks were modified to wait until other threads had finished their tasks before moving exiting. This change is expected to improve the efficiency of parallel processing of read-only transactions.
• Make it more explicit how eosio_exit works by eliminating some needless abstraction which is not straightford in the code where multiple wasm instances executing at the same time
• Update eos-vm-oc threads limit calculation based on the design.

Clean tmp and TestLogs after test runs

PR

• (915) [4.0] Clean tmp and TestLogs after test runs

Previously when tests failed, they left temporary files behind, which could cause problems if the tests were run on multiple devices with /tmp mounted on a different device. These files were not being cleaned up, and some te...

Leap v3.2.3

This patch release contains fixes for handling forks in state_history_plugin. All users of state_history_plugin running 3.2.1 or 3.2.2 are highly recommended to update to 3.2.3. Downstream consumers of state history data may have been affected by this defect.

state_history_plugin log files remain correct (not corrupt) for affected nodeos versions. However, in 3.2.1 and 3.2.2 there is a possibility when nodeos switches to a new fork, blocks from this new fork will not be sent to existing state_history_plugin connections. The side effects of this will be dependent on the downstream application that is consuming from state_history_plugin, but most likely will result in the downstream application consuming incorrect or missing data for a block because the downstream application will be unaware that a fork has occurred. Users that have populated downstream applications from 3.2.1 or 3.2.2 should repopulate downstream applications with 3.2.3 starting from a block prior to their upgrade to 3.2.1 or 3.2.2.

Leap v3.2.3 Release Notes

Bug fixes

SHiP forking behavior fix

PRs

• (943) [3.2] SHiP Fix forking behavior

The state_history_plugin was not sending updated blocks when nodeos forked. There has been an enhancement made to the ship_streamer_test to generate a fork and verify SHiP sends new blocks on forks.

Contributors

Special thanks to the contributors that submitted patches for this release:

• @heifner
• @linh2931
• @spoonincode
• @stephenpdeos

Full list of changes since last release

PRs

• (893) [3.2] Use block_num for interrupt of start_block
• (928) [3.2] SHiP flush logs on write
• (936) [3.2] forkdb reset in replay since blocks are signaled
• (943) [3.2] SHiP Fix forking behavior
• (953)[3.2] Bump Leap version to 3.2.3

Full Changelog: v3.2.2...v3.2.3

Leap v4.0.0-rc2

Antelope Leap v4.0.0 release focuses on optimizing node performance, improving block propagation, scheduling future snapshots, adding read-only transactions, and improving performance of EVM related functions.

📝	v4.0.0-rc2 is the first release candidate of 4.0.0 due to an issue discovered in a tagged but not released 4.0.0-rc1.

Leap v4.0.0-rc2 Release Notes

New features

Read-Only Transactions

PRs

• (558) Introduce new read-only transaction RPC endpoint

We have introduced a RPC endpoint for sending read-only transactions. A read-only transaction does not change the state and is not added into the blockchain (chainbase). It is useful for users to retrieve more complex chain information.

The format is the same as regular transaction, except it does not contain authorization section. Mechanisms are in place to ensure actions (including inline actions) in a read-only transaction do not modify the state and deepmind logging is not performed.

The RPC URL is v1/chain/send_readonly_transaction

Cleos is updated to send read-only transactions:

• cleos push transaction --read <transaction>
• cleos push action --read <action>

New producer_plugin logging categories are added for transient transactions (dry-run (by compute_transaction) and read-only transactions). They are transient_trx_success_tracing and transient_trx_failure_tracing . Existing transaction_success_tracing and transaction_failure_tracing do not include the logging for transient transactions.

max_action_return_value_size is not enforced for read-only transactions.

Auto-peering with schedule proximal BP nodes

PRs

• (744) Add auto bp peering functionality

We've added a config option to automatically manage connections based on the BP schedule. Node order and peering can impact a node's latency to receive last block of the preceding BP, and to send their last block to the next BP. Ideally Block Producers:

• receive the last block of the previous round as soon as possible
• send their last block to the next Block Producer as soon as possible
• avoid getting a late start on their block production

To achieve these goals, the following changes were made in v4.0.0:

Correlate BP account names and connection details

--p2p-auto-bp-peer has been added to nodeos to allow a node to correlate BP account names to p2p connection details. When a block producer node with this setting detects a pending schedule change, it will attempt to auto-peer with BPs that have come into schedule proximity, and disconnect from any auto-peered BPs that are no longer in schedule proximity. These auto-peered connections are not subjected to the limit specified by the --max-clients option.

BP Peer flag in net api

is_bp_peer has been added to every connection object in the v1/net/connections api endpoint response.

Snapshot Scheduling API

PRs

• (131) Proposal for a snapshot scheduling API
• (759) Programmable snapshots API

The Snapshot Scheduling API added in Leap v4.0.0 allows users to schedule and manage snapshot requests in advance, increasing reliability and reducing the potential for errors in the snapshot creation process. It also allows for greater flexibility in creating and managing snapshots, as users can specify parameters such as start and end blocks and block spacing.

The API provides endpoints for several features:

• /producer/schedule_snapshot: Add a task to perform snapshots
  • Parameters: start_block, end_block, block_spacing, snapshot_description
• /producer/get_snapshot_requests: Return all scheduled snapshot requests
• /producer/unschedule_snapshot: Remove a previously scheduled snapshot task
  • Parameters: snapshot_request_id

For more details, check out the Snapshot Scheduling API Design Specs.

Prometheus Exporter Plugin

PRs

• (537) Instrument producer_plugin for Prometheus

Instrumentation of nodeos plugins to expose data with Prometheus. An initial phase of instrumentation will allow for the evaluation of the usability of the instrumentation when viewed from Prometheus. The initial instrumentation will not include any per account or per contract metrics. Additionally, the ability to expire metrics will not be included in the initial phase. To support dynamic metrics, the metric object for each plugin will include an indicator of when the set of metrics has last changed. The following metrics will be collected for the initial phase of instrumentation:

Net Plugin
gauge - number of clients
gauge - number of peers
gauge - number of dropped blocks

Producer Plugin (Producer plugin metrics will be gathered at the start of each block)
gauge - unapplied transaction queue sizes (number of transactions)
counter - blacklisted transactions count (total)
counter - blocks produced
counter - transactions produced
gauge - last irreversible
gauge - current block num
gauge - subjective billing number of accounts
gauge - subjective billing number of blocks
gauge - scheduled transactions

See the Prometheus Plugin Architecture for more details.

Log Splitting

PRs

• (532) block log partition functionality
• (628) SHiP stability fixes, log splitting, and memory usage improvements

Leap v4.0.0 includes two similar features aimed at giving node operators greater control over how important logs for blocks and state are managed: blocks log splitting and state history log splitting. The key benefit of these features is that it allows users to specify how many split log files to retain, and automates aging out older logs by deleting them or moving them another directory.

Blocks Log Splitting

To make it easy to use block log splitting, we've added some new options for the chain plugin:

• blocks-log-stride allows you to split the block log file when the head block number is the multiple of the stride. When the stride is reached, the current block log and index will be renamed 'blocks--.log/index', and a new current block log and index will be created with the most recent block. All files following this format will be used to construct an extended block log.
• max-retained-block-files alows you to specifiy the maximum number of block files to retain so that the blocks in those files can be queried. When the number is reached, the oldest block file would be moved to an archive directory or deleted if the archive directory is empty. It's important to note that the retained block log files should not be manipulated by users.
• Two new options allow you to configure the locations of blocks log directories, we've added two new options:
  • blocks-archive-dir allows you to specify the location of the blocks archive directory (absolute or relative path). If the value is empty, blocks files beyond the retained limit will be deleted. Files in the archive directory won't be accessed by nodeos anymore.
  • blocks-retained-dir allows you to specify the location of the blocks retained directory (absolute or relative path). If the value is empty, it is set to the value of the blocks dir.

Please note that all of the newly added options cannot be specified together with the --block-log-retain-blocks option. Overall, we believe that the Blocklog splitting feature and its associated options will help you better manage your block logs and improve the efficiency of your nodeos instance.

SHiP Log Splitting

To make it easy to use SHiP Log Splitting, we've added some new options for the state history plugin:

• state-history-stride allows you to split the state history log file when the block number is the multiple of the stride. When the stride is reached, the current history log and index will be renamed '*-history--.log/index', and a new current history log and index will be created with the most recent blocks. All files following this format will be used to construct an extended history log.
• max-retained-history-files allows you to specify the maximum number of state history log file groups to retain so that the blocks in those files can be queried. When the number is reached, the oldest log files would be moved to an archive directory or deleted if the archive directory is empty. The retained log files should not be manipulated by users.
• state-history-retained-dir allows you to specify the location of the state history retained directory (absolute path or relative to state-history dir). If the value is empty, it is set to the value of the state history dir.
• state-history-archive-dir allows you to specify the location of the state history archive directory (absolute path or relative to state-history dir). If the value is empty string, log files beyond the retained limit will be deleted. All files in the archive directory are completely under user's control, i.e. they won't be accessed by nodeos anymore.

_Please note that all of the newly added options canno...