Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[suggestion] WSV snapshot #3651

Closed
mversic opened this issue Jun 26, 2023 · 6 comments
Closed

[suggestion] WSV snapshot #3651

mversic opened this issue Jun 26, 2023 · 6 comments
Assignees
Labels
Enhancement New feature or request iroha2-dev The re-implementation of a BFT hyperledger in RUST QA-confirmed This bug is reproduced and needs a fix

Comments

@mversic
Copy link
Contributor

mversic commented Jun 26, 2023

Feature request

It is in the scope of this ticket to add a way for Iroha to serialize and deserialize wsv.

We can do this:

  1. periodically with a configurable dumping period
  2. we can implement another endpoint in Torii to force dumping of wsv to the disk.

Once restarted, the node will read the wsv from the disk and reapply any excess transactions from the block store on top of it. The node will try reading a serialized wsv only if it is started with a specific flag, otherwise not

We can also make the network exchange the wsv when started similarly to what we do with broadcasting genesis block. IMO this would require more work and development time for too small gains

Motivation

We need a way to restart Iroha without it having to reapply all transactions from the block store to get to the current WSV. This is a highly requested feature because it takes 3hrs to restart Iroha for Orillion team

Who can help?

@appetrosyan @mversic

@mversic mversic added Enhancement New feature or request iroha2-dev The re-implementation of a BFT hyperledger in RUST labels Jun 26, 2023
@QuentinI
Copy link
Contributor

I vaguely remember this discussed as a possible Kura init mode, where we periodically store snapshots of WSV in blockstore and restart by applying only txns that happened after the snapshot.
I'd say that makes the most sense to me.

@mversic
Copy link
Contributor Author

mversic commented Jun 26, 2023

you are referring to this discussion

@mversic
Copy link
Contributor Author

mversic commented Jun 26, 2023

I didn't think taking a wsv snapshot should be mixed with Kura init modes

@appetrosyan
Copy link
Contributor

I would recommend adding a SIGUSR1 handler too, since this is what most programs would do anyway.

@QuentinI QuentinI self-assigned this Jun 26, 2023
@Erigara
Copy link
Contributor

Erigara commented Jun 30, 2023

We probably would need to dump finalized_wsv, not wsv because it's guaranteed not to change.
This mean that we will need apply latest block to it in order to obtain original wsv.

QuentinI pushed a commit to QuentinI/iroha that referenced this issue Jul 20, 2023
Signed-off-by: Artemii Gerasimovich <gerasimovich@soramitsu.co.jp>
QuentinI pushed a commit to QuentinI/iroha that referenced this issue Jul 20, 2023
Signed-off-by: Artemii Gerasimovich <gerasimovich@soramitsu.co.jp>
appetrosyan pushed a commit to QuentinI/iroha that referenced this issue Jul 24, 2023
Signed-off-by: Artemii Gerasimovich <gerasimovich@soramitsu.co.jp>
QuentinI pushed a commit to QuentinI/iroha that referenced this issue Aug 11, 2023
Signed-off-by: Artemii Gerasimovich <gerasimovich@soramitsu.co.jp>
QuentinI pushed a commit to QuentinI/iroha that referenced this issue Aug 11, 2023
Signed-off-by: Artemii Gerasimovich <gerasimovich@soramitsu.co.jp>
appetrosyan pushed a commit to QuentinI/iroha that referenced this issue Aug 12, 2023
Signed-off-by: Artemii Gerasimovich <gerasimovich@soramitsu.co.jp>
appetrosyan pushed a commit that referenced this issue Aug 12, 2023
Signed-off-by: Artemii Gerasimovich <gerasimovich@soramitsu.co.jp>
appetrosyan pushed a commit that referenced this issue Aug 14, 2023
Signed-off-by: Artemii Gerasimovich <gerasimovich@soramitsu.co.jp>
@timofeevmd timofeevmd self-assigned this Aug 21, 2023
RealNeGate pushed a commit to RealNeGate/iroha that referenced this issue Aug 22, 2023
Signed-off-by: Artemii Gerasimovich <gerasimovich@soramitsu.co.jp>
@timofeevmd
Copy link
Contributor

Environment: mac m1
iroha2 commit: ee60fc8
setup: on system

step for reproduce:

  • ./scripts/test_env.sh setup
    -kill -2 ${pid}
  • ./scripts/test_env.sh setup`

console output
`

  • BLOCK_COUNT=
  • echo -1
    -1
  • '[' '' = 1 ']'
  • echo 'Sleeping 1 sec...'
    Sleeping 1 sec...
  • sleep 1
  • for i in $(seq 1 $1)
  • echo 'Waiting for genesis block to be created, attempt 16/20'
    Waiting for genesis block to be created, attempt 16/20
    ++ curl --fail -X GET http://127.0.0.1:8180/status/blocks
    % Total % Received % Xferd Average Speed Time Time Time Current
    Dload Upload Total Spent Left Speed
    0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
    curl: (7) Failed to connect to 127.0.0.1 port 8180 after 4 ms: Couldn't connect to server
    `

logs by peers was attached
WSV_SNAPSHOT_3651.txt

@timofeevmd timofeevmd added the QA-confirmed This bug is reproduced and needs a fix label Sep 7, 2023
mversic pushed a commit that referenced this issue Oct 17, 2023
Signed-off-by: Artemii Gerasimovich <gerasimovich@soramitsu.co.jp>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement New feature or request iroha2-dev The re-implementation of a BFT hyperledger in RUST QA-confirmed This bug is reproduced and needs a fix
Projects
None yet
Development

No branches or pull requests

5 participants