Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(runtime/v2): store loader on simappv2 #21704

Merged
merged 57 commits into from
Sep 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
ebf5006
add state export
tac0turtle Jun 24, 2024
268c034
minor
tac0turtle Jun 24, 2024
babf339
Merge branch 'main' into marko/state_export
tac0turtle Jul 1, 2024
6780043
comment out export zero height
tac0turtle Jul 1, 2024
61c0343
Merge branch 'main' into marko/state_export
randygrok Jul 30, 2024
b8ac0e5
add logic for export app state and bootstrap simapp tests
randygrok Aug 6, 2024
a1b0a5f
fix bug with export genesis nil pointer
randygrok Aug 6, 2024
71d0027
some additions to continue the test
randygrok Aug 6, 2024
cadccd4
add context with value
randygrok Aug 6, 2024
7bc73f0
remove commented code
randygrok Aug 7, 2024
728313a
go mod tidy all
randygrok Aug 7, 2024
e97fda6
add version on export
randygrok Aug 7, 2024
57c9751
include export validators
randygrok Aug 8, 2024
a32d778
add validators into status
randygrok Aug 8, 2024
4bcf345
include new types mode
randygrok Aug 9, 2024
7006da5
go mod tidy all
randygrok Aug 13, 2024
91570fb
temp commit, include logic for zeroheight
randygrok Aug 13, 2024
de93866
remove zero height part
randygrok Aug 13, 2024
9fc2546
add export
randygrok Aug 13, 2024
7a1fa60
apply into simapp v2
randygrok Aug 13, 2024
6b32d53
Merge remote-tracking branch 'origin/main' into randy/export-genesis
randygrok Aug 13, 2024
db542cf
remove option to export certain modules only
randygrok Aug 13, 2024
70f1090
remove modules to export feature
randygrok Aug 13, 2024
ea0193d
remove modules to export feature part 2
randygrok Aug 13, 2024
c552beb
remove debug entry in manager
randygrok Aug 13, 2024
f67770e
delete commented code
randygrok Aug 14, 2024
2948e09
rename RunWithCtx
randygrok Aug 14, 2024
a0ffd0a
race condition not fixed yet
randygrok Aug 14, 2024
519d6b7
remove concurrent export
randygrok Aug 14, 2024
f199f06
include issue number
randygrok Aug 14, 2024
11ec8d2
fix lint
randygrok Aug 15, 2024
a634633
fix lint, last
randygrok Aug 15, 2024
e0093cf
move ExportedApp to genutil
randygrok Aug 15, 2024
5ab7c1a
Merge branch 'main' into randy/export-genesis
randygrok Aug 15, 2024
da07a4a
lint
randygrok Aug 15, 2024
264434e
Merge branch 'randy/export-genesis' of git.luolix.top-randy:cosmos/cosmos…
randygrok Aug 15, 2024
aedc7b0
remove duplicated package on import
randygrok Aug 15, 2024
8f97b21
lint
randygrok Aug 15, 2024
9156b86
linter
randygrok Aug 15, 2024
858af16
pass linter
randygrok Aug 15, 2024
a789413
remove unsafe message
randygrok Aug 15, 2024
dee4c3b
remove command
randygrok Aug 15, 2024
5051c67
review changes
randygrok Aug 15, 2024
73e839b
Revert "remove option to export certain modules only"
randygrok Aug 15, 2024
98b4397
remove comment
randygrok Aug 15, 2024
4324f42
Merge branch 'main' into randy/export-genesis
randygrok Aug 15, 2024
3ea6367
bring back viper sets
randygrok Aug 16, 2024
1874e0c
Merge branch 'randy/export-genesis' of git.luolix.top-randy:cosmos/cosmos…
randygrok Aug 16, 2024
64eb9a4
temp commit with only store changes
randygrok Sep 13, 2024
8035440
Merge remote-tracking branch 'origin/main' into feat/store-loader
randygrok Sep 13, 2024
1aeda3a
add upgrade handler
randygrok Sep 13, 2024
346e715
bring back version
randygrok Sep 13, 2024
7586ee2
move UpgradeStoreLoader to runtime mod
randygrok Sep 13, 2024
4ecf936
fix from review
randygrok Sep 13, 2024
c732286
Merge branch 'main' into feat/store-loader
randygrok Sep 13, 2024
b2082c8
add changelog
randygrok Sep 13, 2024
8c23aa5
lint upgrades
randygrok Sep 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ Every module contains its own CHANGELOG.md. Please refer to the module you are i
### Features

* (baseapp) [#20291](https://github.com/cosmos/cosmos-sdk/pull/20291) Simulate nested messages.
* (runtime) [#21704](https://github.com/cosmos/cosmos-sdk/pull/21704) Add StoreLoader in simappv2.

### Improvements

Expand Down
21 changes: 21 additions & 0 deletions runtime/store.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (
"cosmossdk.io/core/store"
storetypes "cosmossdk.io/store/types"

"github.com/cosmos/cosmos-sdk/baseapp"
sdk "github.com/cosmos/cosmos-sdk/types"
)

Expand Down Expand Up @@ -179,3 +180,23 @@ func (s kvStoreAdapter) ReverseIterator(start, end []byte) store.Iterator {
func KVStoreAdapter(store store.KVStore) storetypes.KVStore {
return &kvStoreAdapter{store}
}

// UpgradeStoreLoader is used to prepare baseapp with a fixed StoreLoader
// pattern. This is useful for custom upgrade loading logic.
func UpgradeStoreLoader(upgradeHeight int64, storeUpgrades *store.StoreUpgrades) baseapp.StoreLoader {
return func(ms storetypes.CommitMultiStore) error {
if upgradeHeight == ms.LastCommitID().Version+1 {
// Check if the current commit version and upgrade height matches
if len(storeUpgrades.Deleted) > 0 || len(storeUpgrades.Added) > 0 {
stup := &storetypes.StoreUpgrades{
Added: storeUpgrades.Added,
Deleted: storeUpgrades.Deleted,
}
return ms.LoadLatestVersionAndUpgrade(stup)
}
}

// Otherwise load default store loader
return baseapp.DefaultStoreLoader(ms)
}
}
9 changes: 8 additions & 1 deletion runtime/v2/app.go
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,8 @@ type App[T transaction.Tx] struct {
// GRPCMethodsToMessageMap maps gRPC method name to a function that decodes the request
// bytes into a gogoproto.Message, which then can be passed to appmanager.
GRPCMethodsToMessageMap map[string]func() gogoproto.Message

storeLoader StoreLoader
}

// Name returns the app name.
Expand All @@ -68,9 +70,14 @@ func (a *App[T]) DefaultGenesis() map[string]json.RawMessage {
return a.moduleManager.DefaultGenesis()
}

// SetStoreLoader sets the store loader.
func (a *App[T]) SetStoreLoader(loader StoreLoader) {
a.storeLoader = loader
}

// LoadLatest loads the latest version.
func (a *App[T]) LoadLatest() error {
return a.db.LoadLatestVersion()
return a.storeLoader(a.db)
}

// LoadHeight loads a particular height
Expand Down
1 change: 1 addition & 0 deletions runtime/v2/module.go
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,7 @@ func ProvideAppBuilder[T transaction.Tx](
msgRouterBuilder: msgRouterBuilder,
queryRouterBuilder: stf.NewMsgRouterBuilder(), // TODO dedicated query router
GRPCMethodsToMessageMap: map[string]func() proto.Message{},
storeLoader: DefaultStoreLoader,
}
appBuilder := &AppBuilder[T]{app: app}

Expand Down
33 changes: 33 additions & 0 deletions runtime/v2/store.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
package runtime

import (
"fmt"

"cosmossdk.io/core/store"
"cosmossdk.io/server/v2/stf"
storev2 "cosmossdk.io/store/v2"
Expand Down Expand Up @@ -55,3 +57,34 @@ type Store interface {
// LastCommitID returns the latest commit ID
LastCommitID() (proof.CommitID, error)
}

// StoreLoader allows for custom loading of the store, this is useful when upgrading the store from a previous version
type StoreLoader func(store Store) error

// DefaultStoreLoader just calls LoadLatestVersion on the store
func DefaultStoreLoader(store Store) error {
return store.LoadLatestVersion()
}

// UpgradeStoreLoader upgrades the store if the upgrade height matches the current version, it is used as a replacement
// for the DefaultStoreLoader when there are store upgrades
func UpgradeStoreLoader(upgradeHeight int64, storeUpgrades *store.StoreUpgrades) StoreLoader {
return func(store Store) error {
latestVersion, err := store.GetLatestVersion()
if err != nil {
return err
}

if uint64(upgradeHeight) == latestVersion+1 {
if len(storeUpgrades.Deleted) > 0 || len(storeUpgrades.Added) > 0 {
if upgrader, ok := store.(storev2.UpgradeableStore); ok {
return upgrader.LoadVersionAndUpgrade(latestVersion, storeUpgrades)
}

return fmt.Errorf("store does not support upgrades")
}
}

return DefaultStoreLoader(store)
}
}
5 changes: 2 additions & 3 deletions simapp/v2/upgrades.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import (

"cosmossdk.io/core/appmodule"
"cosmossdk.io/core/store"
"cosmossdk.io/runtime/v2"
"cosmossdk.io/x/accounts"
bankv2types "cosmossdk.io/x/bank/v2/types"
epochstypes "cosmossdk.io/x/epochs/types"
Expand Down Expand Up @@ -44,8 +45,6 @@ func (app *SimApp[T]) RegisterUpgradeHandlers() {
Deleted: []string{"crisis"}, // The SDK discontinued the crisis module in v0.52.0
}

// configure store loader that checks if version == upgradeHeight and applies store upgrades
_ = storeUpgrades
// app.SetStoreLoader(upgradetypes.UpgradeStoreLoader(upgradeInfo.Height, &storeUpgrades))
app.SetStoreLoader(runtime.UpgradeStoreLoader(upgradeInfo.Height, &storeUpgrades))
}
}
3 changes: 3 additions & 0 deletions x/upgrade/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,9 @@ expected upgrade. It eliminiates the chances for the new binary to execute `Stor
times every time on restart. Also if there are multiple upgrades planned on same height, the `Name`
will ensure these `StoreUpgrades` takes place only in planned upgrade handler.

**Note:** The `StoreLoader` helper function for StoreUpgrades in v2 is not part of the `x/upgrade` module;
instead, you can find it in the runtime v2 module.

### Proposal

Typically, a `Plan` is proposed and submitted through governance via a proposal
Expand Down
28 changes: 3 additions & 25 deletions x/upgrade/types/storeloader.go
Original file line number Diff line number Diff line change
@@ -1,28 +1,6 @@
package types

import (
corestore "cosmossdk.io/core/store"
storetypes "cosmossdk.io/store/types"
import "github.com/cosmos/cosmos-sdk/runtime"

"github.com/cosmos/cosmos-sdk/baseapp"
)

// UpgradeStoreLoader is used to prepare baseapp with a fixed StoreLoader
// pattern. This is useful for custom upgrade loading logic.
func UpgradeStoreLoader(upgradeHeight int64, storeUpgrades *corestore.StoreUpgrades) baseapp.StoreLoader {
return func(ms storetypes.CommitMultiStore) error {
if upgradeHeight == ms.LastCommitID().Version+1 {
// Check if the current commit version and upgrade height matches
if len(storeUpgrades.Deleted) > 0 || len(storeUpgrades.Added) > 0 {
stup := &storetypes.StoreUpgrades{
Added: storeUpgrades.Added,
Deleted: storeUpgrades.Deleted,
}
return ms.LoadLatestVersionAndUpgrade(stup)
}
}

// Otherwise load default store loader
return baseapp.DefaultStoreLoader(ms)
}
}
// UpgradeStoreLoader moved to runtime package, keeping this for backwards compatibility
var UpgradeStoreLoader = runtime.UpgradeStoreLoader
Loading