documentation for in-place migrations with x/upgrade module

docs/building-modules/README.md

@@ -20,3 +20,4 @@ This repository contains documentation on concepts developers need to know in or
 10. [Module Interfaces](./module-interfaces.md)
 11. [Standard Module Structure](./structure.md)
 12. [Errors](./errors.md)
+13. [In-Place Store Migrations](./upgrade.md)

docs/building-modules/upgrade.md

@@ -0,0 +1,55 @@
+<!--
+order: 13
+-->
+
+# In-Place Store Migrations
+
+In-place store migrations allow your modules to upgrade to new versions that include breaking changes. This document outlines how to build modules to take advantage of this functionality. {synopsis}
+## Prerequisite Readings
+
+- [In-Place Store Migration](../core-concepts/upgrade.md) {prereq}
+## Consensus Version
+
+Successful upgrades of existing modules require your `AppModule` to implement the function `ConsensusVersion() uint64`. 
+
+- The versions must be hard-coded by the module developer. 
+- The initial version **must** be set to 1. 
+
+Consensus versions serve as state-breaking versions of app modules and are incremented when the module is upgraded. 
+
+## Registering Migrations
+
+To register the functionality that takes place during a module upgrade, you must register which migrations we want to take place.
+
+Migration registration takes place in the `Configurator` using the `RegisterMigration` method. The `AppModule` reference to the configurator is in the `RegisterServices` method. 
+
+You can register one or more migrations. If you register more than one migration script, list the migrations in increasing order and ensure there are enough migrations that lead to the desired consensus version. For example, to migrate to version 3 of a module, register separate migrations for version 1 and version 2 as shown in the following example:
+
+```golang
+func (am AppModule) RegisterServices(cfg module.Configurator) {
+    // --snip--
+    cfg.RegisterMigration(types.ModuleName, 1, func(ctx sdk.Context) error {
+        // Perform in-place store migrations from ConsensusVersion 1 to 2.
+    })
+     cfg.RegisterMigration(types.ModuleName, 2, func(ctx sdk.Context) error {
+        // Perform in-place store migrations from ConsensusVersion 2 to 3.
+    })
+}
+```
+
+Since these migrations are functions that need access to a Keeper's store, use a wrapper around the keepers called `Migrator` as shown in this example:
+
++++ https://github.com/cosmos/cosmos-sdk/blob/6ac8898fec9bd7ea2c1e5c79e0ed0c3f827beb55/x/bank/keeper/migrations.go#L8-L21
+
+## Writing Migration Scripts
+
+To define the functionality that takes place during an upgrade, write a migration script. Since migration scripts manipulate legacy code, place these functions in a `legacy/` directory. For example, to write migration scripts for the bank module, place the functions in `x/bank/legacy/`. Use the recommended naming convention for these functions. For example, `v043bank` is the script that migrates this legacy package `x/bank/legacy/v043:
+
+```golang
+// Migrating bank module from version 1 to 2
+func (m Migrator) Migrate1to2(ctx sdk.Context) error {
+	return v043bank.MigrateStore(ctx, m.keeper.storeKey) // v043bank is package `x/bank/legacy/v043`.
+}
+```
+
+To see example code of changes that were implemented in a migration of balance keys, check out the [func migrateBalanceKeys](https://github.com/cosmos/cosmos-sdk/blob/36f68eb9e041e20a5bb47e216ac5eb8b91f95471/x/bank/legacy/v043/store.go#L41-L62) code. For context, this code introduced migrations of the bank store that updated addresses to be prefixed by their length in bytes as outlined in [ADR-028](../architecture/adr-028-public-key-addresses.md).

docs/core/README.md

@@ -9,19 +9,20 @@ parent:
 This repository contains reference documentation on the core concepts of the Cosmos SDK.
 
 1. [`BaseApp`](./baseapp.md)
-1. [Transaction](./transactions.md)
-1. [Context](./context.md)
-1. [Node Client](./node.md)
-1. [Store](./store.md)
-1. [Encoding](./encoding.md)
-1. [gRPC, REST and Tendermint Endpoints](./grpc_rest.md)
-1. [Command-Line Interface](./cli.md)
-1. [Events](./events.md)
-1. [Telemetry](./telemetry.md)
-1. [Object-Capabilities](./ocap.md)
-1. [RunTx recovery middleware](./runtx_middleware.md)
-1. [Simulation](./simulation.md)
-1. [Protobuf documentation](./proto-docs.md)
+2. [Transaction](./transactions.md)
+3. [Context](./context.md)
+4. [Node Client](./node.md)
+5. [Store](./store.md)
+6. [Encoding](./encoding.md)
+7. [gRPC, REST and Tendermint Endpoints](./grpc_rest.md)
+8. [Command-Line Interface](./cli.md)
+9. [Events](./events.md)
+10. [Telemetry](./telemetry.md)
+11. [Object-Capabilities](./ocap.md)
+12. [RunTx recovery middleware](./runtx_middleware.md)
+13. [Simulation](./simulation.md)
+14. [Protobuf documentation](./proto-docs.md)
+15. [In-Place Store Migrations](./upgrade.md)
 
 After reading about the core concepts, check the [IBC documentation](../ibc/README.md) to learn more
-about the IBC core concepts and how to integrate it to you application.
+about the IBC core concepts and how to integrate IBC in your application.

docs/core/upgrade.md

@@ -0,0 +1,133 @@
+<!--
+order: 15
+-->
+
+# In-Place Store Migrations
+
+::: warning
+Read and understand all of the in-place store migration documentation before you run a migration on a live chain.
+:::
+
+Upgrade your app modules smoothly with custom in-place migration logic. {synopsis}
+
+The Cosmos SDK uses two methods to perform upgrades. 
+
+- Exporting the entire application state to a JSON file using the `export` CLI command, making changes, and then starting a new binary with the changed JSON file as the genesis file. See the [Chain Upgrade Guide](../migrations/chain-upgrade-guide-040.md#upgrade-procedure). 
+
+- Version v0.43 and later can perform upgrades in place to significantly decrease the upgrade time for chains with a larger state. Use the [Migration Upgrade Guide](../building-modules/upgrade.md) guide to set up your application modules to take advantage of in-place upgrades.
+
+
+This document outlines the second method described above. 
+
+## Tracking Module Versions
+
+Each module gets assigned a consensus version by the module developer. The consensus version serves as the breaking change version of the module. The SDK keeps track of all module consensus versions in the x/upgrade `VersionMap` store. During an upgrade, the difference between the old `VersionMap` stored in state and the new `VersionMap` is calculated by the Cosmos SDK. For each identified difference, the module-specific migrations are run and the respective consensus version of each upgraded module is incremented.
+
+
+## Genesis State
+
+When starting a new chain, the consensus version of each module must be saved to state during the application's genesis. To save the consensus version, add the following line to the `InitChainer` method in `app.go`:
+
+```diff
+func (app *MyApp) InitChainer(ctx sdk.Context, req abci.RequestInitChain) abci.ResponseInitChain {
+  ...
++ app.UpgradeKeeper.SetModuleVersionMap(ctx, app.mm.GetVersionMap())
+  ...
+}
+```
+
+This information is used by the Cosmos SDK to detect when modules with newer versions are introduced to the app. 
+
+### Consensus Version
+The consensus version is defined on each app module by the module developer and serves as the breaking change version of the module. The consensus version informs the SDK on which modules need to be upgraded. For example, if the bank module was version 2 and an upgrade introduces bank module 3, the SDK upgrades the bank module and runs the "version 2 to 3" migration script.
+
+### Version Map
+The version map is a mapping of module names to consensus versions. The map is persisted to x/upgrade's state for use during in-place migrations. When migrations finish, the updated version map is persisted to state. 
+
+## Upgrade Handlers
+
+Upgrades use an `UpgradeHandler` to facilitate migrations. The `UpgradeHandler` functions implemented by the app developer must conform to the following function signature. These functions retrieve the `VersionMap` from x/upgrade's state and return the new `VersionMap` to be stored in x/upgrade after the upgrade. The diff between the two `VersionMap`s determines which modules need upgrading.
+
+```golang
+type UpgradeHandler func(ctx sdk.Context, plan Plan, fromVM VersionMap) (VersionMap, error)
+```
+
+Inside these functions, you must perform any upgrade logic to include in the provided `plan`. All upgrade handler functions must end with the following line of code:
+
+```golang
+  return app.mm.RunMigrations(ctx, cfg, fromVM)
+```
+
+## Running Migrations
+
+Migrations are run inside of an `UpgradeHandler` using  `app.mm.RunMigrations(ctx, cfg, vm)`. The `UpgradeHandler` functions describe the functionality to occur during an upgrade. The `RunMigration` function loops through the `VersionMap` argument and runs the migration scripts for all versions that are less than the versions of the new binary app module. After the migrations are finished, a new `VersionMap` is returned to persist the upgraded module versions to state.
+
+```golang
+cfg := module.NewConfigurator(...)
+app.UpgradeKeeper.SetUpgradeHandler("my-plan", func(ctx sdk.Context, plan upgradetypes.Plan, vm module.VersionMap) (module.VersionMap, error) {
+
+    // ...
+    // do upgrade logic
+    // ...
+
+    // RunMigrations returns the VersionMap
+    // with the updated module ConsensusVersions
+    return app.mm.RunMigrations(ctx, vm)
+})
+```
+
+To learn more about configuring migration scripts for your modules, see the [Migration Upgrade Guide](../building-modules/upgrade.md).
+
+## Adding New Modules During Upgrades
+
+You can introduce entirely new modules to the application during an upgrade. New modules are recognized because they have not yet been registered in `x/upgrade`'s `VersionMap` store. In this case, `RunMigrations` calls the `InitGenesis` function from the corresponding module to set up its initial state.
+
+## Overwriting Genesis Functions
+
+The Cosmos SDK offers modules that the application developer can import in their app. These modules often have an `InitGenesis` function already defined.
+
+You can write your own `InitGenesis` function for an imported module. To do this, manually trigger your custom genesis function in the upgrade handler.
+
+::: warning
+You MUST manually set the consensus version in the version map passed to the `UpgradeHandler` function. Without this, the SDK will run the Module's existing `InitGenesis` code even if you triggered your custom function in the `UpgradeHandler`. 
+:::
+
+```go
+import foo "github.com/my/module/foo"
+
+app.UpgradeKeeper.SetUpgradeHandler("my-plan", func(ctx sdk.Context, plan upgradetypes.Plan, vm module.VersionMap)  (module.VersionMap, error) {
+
+    // Register the consensus version in the version map
+    // to avoid the SDK from triggering the default 
+    // InitGenesis function. 
+    vm["foo"] = foo.AppModule{}.ConsensusVersion()
+
+    // Run custom InitGenesis for foo
+    app.mm["foo"].InitGenesis(ctx, app.appCodec, myCustomGenesisState)
+
+    return app.mm.RunMigrations(ctx, cfg, vm)
+})
+```
+
+If you do not have a custom genesis function and want to skip the module's default genesis function, you can simply register the module with the version map in the `UpgradeHandler` as shown in the example:
+
+```go
+import foo "github.com/my/module/foo"
+
+app.UpgradeKeeper.SetUpgradeHandler("my-plan", func(ctx sdk.Context, plan upgradetypes.Plan, vm module.VersionMap)  (module.VersionMap, error) {
+
+    // Set foo's version to the latest ConsensusVersion in the VersionMap.
+    // This will skip running InitGenesis on Foo
+    vm["foo"] = foo.AppModule{}.ConsensusVersion()
+
+    return app.mm.RunMigrations(ctx, cfg, vm)
+})
+```
+
+## Syncing a Full Node to an Upgraded Blockchain
+
+You can sync a full node to an existing blockchain which has been upgraded using Cosmovisor 
+
+In order to successfully sync, you must start with the initial binary that the blockchain started with at genesis. Cosmovisor will handle downloading and switching to the binaries associated with each sequential upgrade. 
+
+To learn more about Cosmovisor, see the [Cosmovisor Quick Start](../run-node/cosmovisor.md).