Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bugfix: schema name migration #2012

Merged
merged 4 commits into from
Jun 6, 2024
Merged

bugfix: schema name migration #2012

merged 4 commits into from
Jun 6, 2024

Conversation

aramikm
Copy link
Collaborator

@aramikm aramikm commented Jun 6, 2024

Goal

The goal of this PR is to make to fix the schema names on main-net that incorrectly got updated with test-net values.

Closes #2006

Changes

  • Added a new migration to fix the schema names
  • Checking the chain type by genesis hash
  • Removed extra migrations (kept the last one in each pallet to make creating the future migrations easier and know be able to track the latest migration)

Checklist

  • Chain spec updated
  • Tests added
  • try-runtime executed

Verification on Mainnet

[2024-06-05T23:49:26Z INFO  frequency_runtime] try-runtime::on_runtime_upgrade frequency.
[2024-06-05T23:49:26Z INFO  runtime::schemas] Running pre_upgrade...
[2024-06-05T23:49:26Z INFO  runtime::schemas] Found genesis... 0x4a587bf17a404e3572747add7aab7bbe56e805a5479c6c436f07f36fcc8d3ae1
[2024-06-05T23:49:26Z INFO  runtime::schemas] Detected Chain is FrequencyMain
[2024-06-05T23:49:26Z INFO  runtime::schemas] Running storage migration...
[2024-06-05T23:49:26Z INFO  runtime::schemas] onchain_version= StorageVersion(3), current_version=StorageVersion(4)
[2024-06-05T23:49:26Z INFO  runtime::schemas] Storage migrated to version 4  read=13, write=13, bytes=9544
[2024-06-05T23:49:26Z INFO  runtime::schemas] Migration Calculated weights=Weight { ref_time: 1625000000, proof_size: 9544 }
[2024-06-05T23:49:26Z INFO  runtime::schemas] Running post_upgrade...
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for System
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for ParachainSystem
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Timestamp
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for ParachainInfo
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Preimage
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Democracy
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Scheduler
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Utility
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Balances
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for TransactionPayment
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Council
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for TechnicalCommittee
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Treasury
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Authorship
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for CollatorSelection
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Session
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Aura
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for AuraExt
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Multisig
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for TimeRelease
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Proxy
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Msa
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Messages
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Schemas
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for StatefulStorage
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Capacity
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for FrequencyTxPayment
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Handles
[2024-06-05T23:49:26Z DEBUG wasm-heap] allocator dropped: AllocationStats { bytes_allocated: 56, bytes_allocated_peak: 792, bytes_allocated_sum: 19240, address_space_used: 1048 }
[2024-06-05T23:49:26Z DEBUG wasm-runtime] Allocated WASM instance 1/2
[2024-06-05T23:49:26Z INFO  try_runtime_core::commands::on_runtime_upgrade] -------------------------------------------------------------------
    
    
[2024-06-05T23:49:26Z INFO  try_runtime_core::commands::on_runtime_upgrade] 🔬 TryRuntime_on_runtime_upgrade succeeded! Running it again without checks for weight measurements.
    
    
[2024-06-05T23:49:26Z INFO  try_runtime_core::commands::on_runtime_upgrade] -------------------------------------------------------------------
[2024-06-05T23:49:26Z INFO  frequency_runtime] try-runtime::on_runtime_upgrade frequency.
[2024-06-05T23:49:26Z INFO  runtime::schemas] Running storage migration...
[2024-06-05T23:49:26Z INFO  runtime::schemas] onchain_version= StorageVersion(3), current_version=StorageVersion(4)
[2024-06-05T23:49:26Z INFO  runtime::schemas] Storage migrated to version 4  read=13, write=13, bytes=9544
[2024-06-05T23:49:26Z INFO  runtime::schemas] Migration Calculated weights=Weight { ref_time: 1625000000, proof_size: 9544 }
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for System
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for ParachainSystem
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Timestamp
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for ParachainInfo
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Preimage
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Democracy
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Scheduler
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Utility
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Balances
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for TransactionPayment
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Council
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for TechnicalCommittee
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Treasury
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Authorship
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for CollatorSelection
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Session
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Aura
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for AuraExt
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Multisig
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for TimeRelease
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Proxy
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Msa
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Messages
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Schemas
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for StatefulStorage
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Capacity
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for FrequencyTxPayment
[2024-06-05T23:49:26Z DEBUG runtime::frame-support] ✅ no migration for Handles
[2024-06-05T23:49:26Z DEBUG wasm-heap] allocator dropped: AllocationStats { bytes_allocated: 56, bytes_allocated_peak: 792, bytes_allocated_sum: 11664, address_space_used: 1048 }
[2024-06-05T23:49:26Z INFO  try_runtime_core::commands::on_runtime_upgrade] -------------------------------------------------------------------
    
    
[2024-06-05T23:49:26Z INFO  try_runtime_core::commands::on_runtime_upgrade] 🔬 Running TryRuntime_on_runtime_upgrade again to check idempotency: PreAndPost
    
    
[2024-06-05T23:49:26Z INFO  try_runtime_core::commands::on_runtime_upgrade] -------------------------------------------------------------------
[2024-06-05T23:49:26Z INFO  frequency_runtime] try-runtime::on_runtime_upgrade frequency.
[2024-06-05T23:49:26Z INFO  runtime::schemas] Running pre_upgrade...
[2024-06-05T23:49:26Z INFO  runtime::schemas] Running storage migration...
[2024-06-05T23:49:26Z INFO  runtime::schemas] onchain_version= StorageVersion(4), current_version=StorageVersion(4)
[2024-06-05T23:49:26Z INFO  runtime::schemas] Migration did not execute. This probably should be removed onchain:StorageVersion(4), current:StorageVersion(4)
[2024-06-05T23:49:26Z INFO  runtime::schemas] Running post_upgrade...

Verification on Paseo

[2024-06-05T23:56:01Z INFO  frequency_runtime] try-runtime::on_runtime_upgrade frequency.
[2024-06-05T23:56:01Z INFO  runtime::schemas] Running pre_upgrade...
[2024-06-05T23:56:01Z INFO  runtime::schemas] Found genesis... 0x203c6838fc78ea3660a2f298a58d859519c72a5efdc0f194abd6f0d5ce1838e0
[2024-06-05T23:56:01Z INFO  runtime::schemas] Detected Chain is FrequencyPaseo
[2024-06-05T23:56:01Z INFO  runtime::schemas] Running storage migration...
[2024-06-05T23:56:01Z INFO  runtime::schemas] onchain_version= StorageVersion(3), current_version=StorageVersion(4)
[2024-06-05T23:56:01Z INFO  runtime::schemas] Storage migrated to version 4  read=1, write=1, bytes=0
[2024-06-05T23:56:01Z INFO  runtime::schemas] Migration Calculated weights=Weight { ref_time: 125000000, proof_size: 0 }
[2024-06-05T23:56:01Z INFO  runtime::schemas] Running post_upgrade...
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for System
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for ParachainSystem
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Timestamp
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for ParachainInfo
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Sudo
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Preimage
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Democracy
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Scheduler
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Utility
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Balances
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for TransactionPayment
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Council
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for TechnicalCommittee
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Treasury
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Authorship
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for CollatorSelection
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Session
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Aura
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for AuraExt
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Multisig
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for TimeRelease
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Proxy
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Msa
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Messages
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Schemas
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for StatefulStorage
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Capacity
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for FrequencyTxPayment
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Handles
[2024-06-05T23:56:01Z DEBUG wasm-heap] allocator dropped: AllocationStats { bytes_allocated: 40, bytes_allocated_peak: 456, bytes_allocated_sum: 12592, address_space_used: 720 }
[2024-06-05T23:56:01Z DEBUG wasm-runtime] Allocated WASM instance 1/2
[2024-06-05T23:56:01Z INFO  try_runtime_core::commands::on_runtime_upgrade] -------------------------------------------------------------------
    
    
[2024-06-05T23:56:01Z INFO  try_runtime_core::commands::on_runtime_upgrade] 🔬 TryRuntime_on_runtime_upgrade succeeded! Running it again without checks for weight measurements.
    
    
[2024-06-05T23:56:01Z INFO  try_runtime_core::commands::on_runtime_upgrade] -------------------------------------------------------------------
[2024-06-05T23:56:01Z INFO  frequency_runtime] try-runtime::on_runtime_upgrade frequency.
[2024-06-05T23:56:01Z INFO  runtime::schemas] Running storage migration...
[2024-06-05T23:56:01Z INFO  runtime::schemas] onchain_version= StorageVersion(3), current_version=StorageVersion(4)
[2024-06-05T23:56:01Z INFO  runtime::schemas] Storage migrated to version 4  read=1, write=1, bytes=0
[2024-06-05T23:56:01Z INFO  runtime::schemas] Migration Calculated weights=Weight { ref_time: 125000000, proof_size: 0 }
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for System
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for ParachainSystem
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Timestamp
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for ParachainInfo
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Sudo
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Preimage
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Democracy
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Scheduler
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Utility
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Balances
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for TransactionPayment
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Council
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for TechnicalCommittee
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Treasury
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Authorship
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for CollatorSelection
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Session
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Aura
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for AuraExt
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Multisig
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for TimeRelease
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Proxy
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Msa
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Messages
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Schemas
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for StatefulStorage
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Capacity
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for FrequencyTxPayment
[2024-06-05T23:56:01Z DEBUG runtime::frame-support] ✅ no migration for Handles
[2024-06-05T23:56:01Z DEBUG wasm-heap] allocator dropped: AllocationStats { bytes_allocated: 40, bytes_allocated_peak: 416, bytes_allocated_sum: 4792, address_space_used: 680 }
[2024-06-05T23:56:01Z INFO  try_runtime_core::commands::on_runtime_upgrade] -------------------------------------------------------------------
    
    
[2024-06-05T23:56:01Z INFO  try_runtime_core::commands::on_runtime_upgrade] 🔬 Running TryRuntime_on_runtime_upgrade again to check idempotency: PreAndPost
    
    
[2024-06-05T23:56:01Z INFO  try_runtime_core::commands::on_runtime_upgrade] -------------------------------------------------------------------
[2024-06-05T23:56:01Z INFO  frequency_runtime] try-runtime::on_runtime_upgrade frequency.
[2024-06-05T23:56:01Z INFO  runtime::schemas] Running pre_upgrade...
[2024-06-05T23:56:01Z INFO  runtime::schemas] Running storage migration...
[2024-06-05T23:56:01Z INFO  runtime::schemas] onchain_version= StorageVersion(4), current_version=StorageVersion(4)
[2024-06-05T23:56:01Z INFO  runtime::schemas] Migration did not execute. This probably should be removed onchain:StorageVersion(4), current:StorageVersion(4)
[2024-06-05T23:56:01Z INFO  runtime::schemas] Running post_upgrade...

Copy link

codecov bot commented Jun 6, 2024

Codecov Report

Attention: Patch coverage is 71.79487% with 11 lines in your changes missing coverage. Please review.

Files Coverage Δ
common/primitives/src/utils.rs 87.50% <100.00%> (+1.78%) ⬆️
pallets/schemas/src/types.rs 92.30% <ø> (ø)
pallets/schemas/src/migration/v4.rs 41.53% <50.00%> (ø)

... and 3 files with indirect coverage changes

@github-actions github-actions bot added the metadata-changed Metadata has changed since the latest full release label Jun 6, 2024
Copy link
Collaborator

@JoeCap08055 JoeCap08055 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good--one question/discussion point: would it be worthwhile to add a script/program somewhere that could be run (manually, I guess) against mainnet after an upgrade to validate things from a client perspective? Might be a useful pattern to have in place for future migrations...

Otherwise, 🚢 it!

// check to ensure updates took place
let known_schemas = get_known_schemas::<T>();
for (schema_id, schema_name) in known_schemas.into_iter() {
// safe to use unwrap since only used in try_runtime
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious nit: why doesn't lint complain about unwrap here? I don't see it disabled at the top of the file?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be since it's under "try-runtime" feature flag

Copy link
Collaborator

@wilwade wilwade left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few non-blocking comments, but looks good. Still trying to run the try-runtime locally, but those issues are on me.

  • Reviewed Code
  • Trying to run locally

@@ -1,4 +1,2 @@
/// migrations to v2
pub mod v2;
/// migrations to v3
pub mod v3;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this one go away as well?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to keep at least the last migration for each pallet for easier access but I can remove this usage part so it does not get included in the code.

common/primitives/src/utils.rs Outdated Show resolved Hide resolved
Copy link
Collaborator

@mattheworris mattheworris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Reviewed code, looks good.
    🚢 it!

@enddynayn
Copy link
Collaborator

Why not submit transactions for these instead of writing a migration script?

@wilwade
Copy link
Collaborator

wilwade commented Jun 6, 2024

Why not submit transactions for these instead of writing a migration script?

Good question!

  1. They were migrations originally and we wanted to understand (and document in code) what was wrong and right.
  2. As a pair was setup wrong, the transactions to make that change would be hard to audit for governance voting on it (as it would be a direct state set which could be anything from a tx view)
  3. Clear trail per #2

@github-actions github-actions bot removed the metadata-changed Metadata has changed since the latest full release label Jun 6, 2024
@github-actions github-actions bot added the metadata-changed Metadata has changed since the latest full release label Jun 6, 2024
@aramikm aramikm merged commit 24c6859 into main Jun 6, 2024
28 checks passed
@aramikm aramikm deleted the fix_migration_bug branch June 6, 2024 22:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
metadata-changed Metadata has changed since the latest full release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: Schema Names Migration Failure
5 participants