Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

added migrations for key fix #2226

Merged
merged 9 commits into from
Nov 22, 2024
Merged

added migrations for key fix #2226

merged 9 commits into from
Nov 22, 2024

Conversation

aramikm
Copy link
Collaborator

@aramikm aramikm commented Nov 20, 2024 •
edited
Loading

Goal

The goal of this PR is

Closes #2227

Details

  • The mainnet will be migrated as all other migrations since it is small enough that would easily fit into a block via
pallet_stateful_storage::migration::v1::MigrateToV1<Runtime>
  • For testnet we will use the same path as block just to mark the
Pallet::<T>::on_chain_storage_version();

but the actual multi-block migration is using on_initialize hook and a new storage item called MigrationPageIndex that would store the progress.

Verification

  • There are issues with try-runtime that doesn't allow me to run it against mainnet or testnet. Added some tests to verify the changes.

Checklist

  • Unit Tests added?
  • Spec version incremented?

@codecov Codecov
Copy link

codecov bot commented Nov 20, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 84.21053% with 18 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
pallets/stateful-storage/src/migration/v1.rs 83.49% 17 Missing ⚠️
pallets/stateful-storage/src/lib.rs 90.90% 1 Missing ⚠️
Files with missing lines Coverage Δ
...lets/capacity/src/migration/provider_boost_init.rs 0.00% <ø> (ø)
pallets/stateful-storage/src/types.rs 69.00% <ø> (+1.75%) ⬆️
pallets/stateful-storage/src/lib.rs 82.10% <90.90%> (+2.19%) ⬆️
pallets/stateful-storage/src/migration/v1.rs 83.49% <83.49%> (ø)

🚨 Try these New Features:

@aramikm aramikm marked this pull request as ready for review November 20, 2024 23:26
@github-actions github-actions bot added the metadata-changed Metadata has changed since the latest full release label Nov 21, 2024
saraswatpuneet
saraswatpuneet reviewed Nov 21, 2024
View reviewed changes
pallets/stateful-storage/src/migration/v1.rs
let weights = T::DbWeight::get().reads_writes(reads, writes).add_proof_size(bytes);
log::info!(target: LOG_TARGET, "migrate_msa_ids weights={:?}",weights);
weights
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic looks good
nit: reduce nesting but since its a one time migration

Copy link
Collaborator Author

@aramikm aramikm Nov 21, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I was on the fence about that but since I wanted to log all the failure cases the other alternative was to use a lot of continue in the loop for failure cases which is also not that great.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's cleared up once you get down to the alternate cases, but a couple of comments would be nice. I wonder if you have found any empty itemized pages stored.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't find any empty itemized pages but still had to handle it just in case.

saraswatpuneet
saraswatpuneet approved these changes Nov 21, 2024
View reviewed changes
Copy link
Collaborator

@saraswatpuneet saraswatpuneet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@github-actions github-actions bot removed the metadata-changed Metadata has changed since the latest full release label Nov 21, 2024
shannonwells
shannonwells approved these changes Nov 21, 2024
View reviewed changes
Copy link
Collaborator

@shannonwells shannonwells left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read through the code, looks good to go given the migration tests.

pallets/stateful-storage/src/migration/v1.rs
let weights = T::DbWeight::get().reads_writes(reads, writes).add_proof_size(bytes);
log::info!(target: LOG_TARGET, "migrate_msa_ids weights={:?}",weights);
weights
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's cleared up once you get down to the alternate cases, but a couple of comments would be nice. I wonder if you have found any empty itemized pages stored.

@github-actions github-actions bot added the metadata-changed Metadata has changed since the latest full release label Nov 21, 2024
@github-actions github-actions bot added metadata-changed Metadata has changed since the latest full release and removed metadata-changed Metadata has changed since the latest full release labels Nov 21, 2024
mattheworris
mattheworris reviewed Nov 22, 2024
View reviewed changes
.github/workflows/verify-pr-commit.yml
@@ -181,9 +181,10 @@ jobs:
steps:
- name: Check Out Repo
uses: actions/checkout@v4
# using older version of cargo deny since the new one requires rustc version >= 1.81
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to upgrade our base image container to a newer version?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should do it but outside the scope of this PR.

mattheworris
mattheworris approved these changes Nov 22, 2024
View reviewed changes
Copy link
Collaborator

@mattheworris mattheworris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Read through changes
  • Executed make try-runtime-upgrade-paseo-testnet
    • Did not see observed host function errors, but did get errors scraping keys, probably network related

🚢 it!

@github-actions github-actions bot added metadata-changed Metadata has changed since the latest full release and removed metadata-changed Metadata has changed since the latest full release labels Nov 22, 2024
@aramikm aramikm merged commit e95a823 into main Nov 22, 2024
30 checks passed
@aramikm aramikm deleted the key_fix_migrations branch November 22, 2024 23:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@saraswatpuneet saraswatpuneet saraswatpuneet approved these changes

@mattheworris mattheworris mattheworris approved these changes

@shannonwells shannonwells shannonwells approved these changes

@demisx demisx Awaiting requested review from demisx demisx is a code owner

@wilwade wilwade Awaiting requested review from wilwade wilwade is a code owner

@enddynayn enddynayn Awaiting requested review from enddynayn

@JoeCap08055 JoeCap08055 Awaiting requested review from JoeCap08055

Assignees
No one assigned
Labels
metadata-changed Metadata has changed since the latest full release
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Bug]: Fix the key issues
4 participants
@aramikm @shannonwells @mattheworris @saraswatpuneet