Document chain upgrading procedure

[adizere]

Summary

This issue documents how to upgrade a Gaia v4 chain, i.e., start and complete a software-upgrade proposal. These are post-meeting notes w/ @greg-szabo and @mircea-c .

Unclear where to put this, so we keep it here until we find the best place for it.
Originally, we drafted this guide to help with testing the client upgrade CLI (#357). We ended up with a different guide that details how to test client upgrades, however. This guide might move somewhere into gaia or a IBC developer knowledge base.

Prerequisites

• gaiad version 4.0.0
  • we will need to patch and make build this later on

Procedure

Start a vanilla gaia instance

killall gaiad
cd ibc-rs/
./scripts/one-chain gaiad ibc-0 ./data 26657 26656 6060 9090 100000000000

If all went well, we should have the home folder for this instance in ./data/ibc-0:

$ ls -l data/
total 392
drwxr-xr-x  7 adi  staff     224 Mar  4 18:57 ibc-0
-rw-r--r--  1 adi  staff  174147 Mar  4 18:57 ibc-0.log

Parametrize the voting period

By default, the voting period is 2 days. This is parametrized in the genesis file, gov section. Reconfigure that to something smaller, e.g., 200 seconds.

--- data/ibc-0/config/genesis.json
+++ data/ibc-0/config/genesis.json
        "max_deposit_period": "172800s"
      },
      "voting_params": {
-        "voting_period": "172800s"
+        "voting_period": "200s"
      },
      "tally_params": {
        "quorum": "0.334000000000000000",
        "threshold": "0.500000000000000000",
        "veto_threshold": "0.334000000000000000"
      }

Do a fresh start the ibc-0 chain

$ killall gaiad
$
$ gaiad unsafe-reset-all --home data/ibc-0/

7:11PM INF Removed existing address book file=data/ibc-0/config/addrbook.json
7:11PM INF Removed all blockchain history dir=data/ibc-0/data
7:11PM INF Reset private validator file to genesis state keyFile=data/ibc-0/config/priv_validator_key.json stateFile=data/ibc-0/data/priv_validator_state.json

In a separate terminal, start the chain

gaiad --home ./data/ibc-0/ start --pruning=nothing --grpc.address=0.0.0.0:9090 2>&1 >./data/ibc-0.log

The chain should be regularly creating blocks at this point (average block rate slightly above 1 second).

Submit the governance proposal for undergoing an upgrade

NOTE: It's important to do this step (submit) and the following step (vote) within 200 seconds from each other.

There are two important parameters when submitting the proposal:

• the proposal name: we'll use simply name-test (this will be relevant later on)
• the height: parameter --upgrade-height
  • this denotes the height where the chain should halt (assuming the proposal is accepted);
  • it's important that the height specified with --upgrade-height occurs after the voting period is exhausted, that is, sometime at least 200 seconds in the future;
  • as a rule of thumb, use a height that is around 300 blocks in the future (i.e., higher than the current height of the chain);

Use the command below, replacing 900 for the correct height:

gaiad tx gov submit-proposal software-upgrade name-test --title title-test --description desc-test \
--from user --upgrade-height 900 --deposit 10000000stake --home data/ibc-0/ \
--keyring-backend test --keyring-dir data/ibc-0/ --chain-id ibc-0

Once submitted, press y and Enter.

The output is rather dense, but we're mostly interested in the proposal identifier, which should be 1:

.... {"key":"proposal_id","value":"1"}...

Submit a 'yes' vote

We'll now vote for this proposal.

gaiad tx gov vote 1 yes --home data/ibc-0/data/ --keyring-backend test --keyring-dir data/ibc-0/ --chain-id ibc-0 --from validator

Query the proposal and wait until it passes

To read the state of the proposal, execute:

gaiad query gov proposal 1 --home data/ibc-0/

An example output follows:

content:
  '@type': /cosmos.upgrade.v1beta1.SoftwareUpgradeProposal
  description: desc-test
  plan:
    height: "900"
    info: ""
    name: name-test
    time: "0001-01-01T00:00:00Z"
    upgraded_client_state: null
  title: title-test
deposit_end_time: "2021-03-06T18:44:44.357347Z"
final_tally_result:
  abstain: "0"
  "no": "0"
  no_with_veto: "0"
  "yes": "0"
proposal_id: "1"
status: PROPOSAL_STATUS_VOTING_PERIOD
submit_time: "2021-03-04T18:44:44.357347Z"
total_deposit:
- amount: "10000000"
  denom: stake
voting_end_time: "2021-03-04T18:48:04.357347Z"
voting_start_time: "2021-03-04T18:44:44.357347Z"

Note the height (900) and the 200 seconds period of voting (from voting_start_time to voting_end_time).

When the chain ibc-0 attains the upgrade height, we should see the chain halt, as follows:

ERR UPGRADE "name-test" NEEDED at height: 900:
ERR CONSENSUS FAILURE!!! err="UPGRADE \"name-test\" NEEDED at height: 900:

Once this error appears, we can quit (or kill) the gaiad process that was running.

Patch the app in the gaia binary:

The name-test string below coincides with the proposal name we submitted earlier.

cd ~/go/src/github.com/cosmos/gaia

Make the following modifications:

$ git diff app/app.go
diff --git a/app/app.go b/app/app.go
index 6aeef44..730970d 100644
--- a/app/app.go
+++ b/app/app.go
@@ -265,6 +265,10 @@ func NewGaiaApp(
        )
        app.UpgradeKeeper = upgradekeeper.NewKeeper(skipUpgradeHeights, keys[upgradetypes.StoreKey], appCodec, homePath)

+       app.UpgradeKeeper.SetUpgradeHandler("name-test", func(ctx sdk.Context, plan upgradetypes.Plan) {
+        })
+
        // register the staking hooks
        // NOTE: stakingKeeper above is passed by reference, so that it will contain these hooks
        app.StakingKeeper = *stakingKeeper.SetHooks(

Now re-build the gaiad binary.

make build

Start the patched binary

The previous step should place the patched binary in ~/go/src/github.com/cosmos/gaia/build/gaiad. We'll start the ibc-0 chain using this upgraded binary:

 ~/go/src/github.com/cosmos/gaia/build/gaiad --home ./data/ibc-0/ start --pruning=nothing --grpc.address=0.0.0.0:9090 2>&1 >./data/ibc-0.log

The chain should pick up where it left from.

Further resources

• https://github.com/cosmos/cosmos-sdk/tree/master/x/gov/spec
• https://github.com/cosmos/gaia/blob/main/docs/gaia-tutorials/live-upgrade-tutorial.md
• https://github.com/cosmos/cosmos-sdk/blob/master/docs/ibc/upgrades/quick-guide.md

[adizere]

The upgrade procedure is capture in the Hermes guide here
https://hermes.informal.systems/commands/upgrade/test.html