Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Import target block body during warp sync #12300

Merged
merged 9 commits into from
Sep 20, 2022

Conversation

dmitry-markin
Copy link
Contributor

Resolves #12255 implementing the first approach suggested.

@dmitry-markin dmitry-markin added A0-please_review Pull request needs code review. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. labels Sep 19, 2022
Copy link
Contributor

@altonen altonen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Left some general comments but most of them were not strictly about this PR

client/network/sync/src/lib.rs Show resolved Hide resolved
@@ -160,6 +232,10 @@ where
phase: WarpSyncPhase::DownloadingWarpProofs,
total_bytes: self.total_proof_bytes,
},
Phase::TargetBlock(_) => WarpSyncProgress {
phase: WarpSyncPhase::DownloadingTargetBlock,
total_bytes: self.total_proof_bytes,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this include an estimate/size of the target block?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocks are downloaded atomically, so may be we should add the block size in bytes to Phase::State stage below when the block is already downloaded. Don't know if it makes sense, considering the block size is orders of magnitude smaller than the latest state (but this might be different in some blockchains).

Phase::WarpProof { .. } => None,
Phase::TargetBlock(header) => {
let request = BlockRequest::<B> {
id: 0,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder why the request ID is always zero and if that is ever going to cause issues. With a quick check the ID is only used here and it's not really used for checking anything. Maybe it can be removed since libp2p already gives unique request IDs so not sure why they're needed in the BlockRequest struct as well.

Not related to this PR since the target block is requested only once but maybe regular syncing code has an issue. Or not. I'll need to check.

Copy link
Member

@arkpar arkpar Sep 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IDs were used before this protocol was moved to libp2p request-response. Now ID matching is done in libp2p and the field was left in BlockRequest and BlockResponse for compatibility I guess. We should eventually remove it.

},
Phase::TargetBlock(header) =>
if let Some(block_header) = &block.header {
if block_header == header {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing that is very nice when writing pallet code is the ensure!() macro that allows to exit early and it makes the error handling code slightly nicer. Maybe we could introduce something like that for the networking code with the ability to adding logging message so something like this:

ensure!(
    block_header == header,
    TargetBlockImportResult::BadResponse,
    "sync",
    "Importing target block failed: different header."
);

Hmm it's quite verbose if the logging is included. But that's something we could think about.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also not relevant to this PR but a general comment

Copy link
Member

@arkpar arkpar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've pushed a small commit that adds a check for target block body to the existing test.

@dmitry-markin
Copy link
Contributor Author

I've pushed a small commit that adds a check for target block body to the existing test.

@arkpar thanks!

@arkpar
Copy link
Member

arkpar commented Sep 20, 2022

bot merge

@paritytech-processbot paritytech-processbot bot merged commit 97ad64b into master Sep 20, 2022
@paritytech-processbot paritytech-processbot bot deleted the dm-missing-block-after-warp-sync branch September 20, 2022 15:05
ordian added a commit that referenced this pull request Sep 23, 2022
* master:
  [Fix] parameter_types! dead code errors (#12340)
  [Feature] Sequential migration execution for try-runtime (#12319)
  bench: Use `_` instead  of `::` in auto-generated file names (#12332)
  Fast Unstake Pallet (#12129)
  Rename anonymous to pure proxy (#12283)
  Migrate remaining old decl_* macros to the new pallet attribute macros (#12271)
  pallet-utility: Disallow none origin (#12321)
  Make automatic storage deposits resistant against changing deposit prices (#12083)
  Format templates and fix `--steps` default value (#12286)
  Bump `wasmtime` to 1.0.0 (#12317)
  Introduce 'intermediate_insert' method to hide implementation details (#12215)
  Bound staking storage items (#12230)
  Use `array-bytes` for All Array/Bytes/Hex Operations (#12190)
  BREAKING: Rename Origin (#12258)
  Use temporary db for benchmarking (#12254)
  rpc: Implement `chainSpec` RPC API (#12261)
  Import target block body during warp sync (#12300)
  Proper naming wrt expectations (#12311)
  [ci] Revert cancel-pipeline job (#12309)
ark0f pushed a commit to gear-tech/substrate that referenced this pull request Feb 27, 2023
* Receive and import target block body

* Request target block

* minor: wording

* Check for block body in the test

* Import target block justifications

* Fix: do not fail block validation if no justifications received

* Fix: import target blocks without justifications

Co-authored-by: arkpar <arkady.paronyan@gmail.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A0-please_review Pull request needs code review. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Single block missing after warp sync.
3 participants