feat: add array of messages to a proposal #9810

cmwaters · 2021-07-28T19:52:40Z

Description

Closes: #9438

This PR allows for an array of arbitrary sdk messages to be added to a proposal. It introduces a new proposal, Proposal2 which has a field messages that accepts an array of sdk.Msg{} instead of using the Content interface. Both proposal types: Proposal and Proposal2 co-exist with the former being deprecated in a later release. The introduction of a new proposal type touches a lot of the gov module (down to introducing a new key type to store the proposals). Although this approach allows for Clients to gradually adopt the new format it does introduce the problem that querying will need to be for both types.

Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

confirmed the correct type prefix in the PR title
confirmed ! in the type prefix if API or client breaking change
confirmed all author checklist items have been addressed
reviewed state machine logic
reviewed API design and naming
reviewed documentation is accurate
reviewed tests and test coverage
manually tested (if applicable)

proto/cosmos/gov/v1beta1/gov.proto

blushi

Thanks @cmwaters for this first smaller PR.
Some gov/keeper tests need to be fixed, which should be minor adjustments.

Regarding the migration tests failing, the options proposed by @aaronc in #9810 (comment) should get around it. I would tend to prefer bumping the proto version which seems cleaner and more aligned with our plan to bump everything to v1 for 0.44 (#9446 (comment)). If we go with the latter option, then this means some additional work related to migrations (something similar to what @AmauryM had done as part of #9492)

x/gov/keeper/proposal.go

x/gov/types/proposal.go

orijbot · 2021-07-29T13:19:02Z

Visit https://dashboard.github.orijtech.com?pr=9810&repo=cosmos%2Fcosmos-sdk to see benchmark details.

x/gov/types/msgs.go

cmwaters · 2021-07-30T07:21:57Z

When you say "bump everything to v1", does this just means to change:

package cosmos.gov.v1beta1;

to

package cosmos.gov.v1;

blushi · 2021-07-30T07:42:08Z

When you say "bump everything to v1", does this just means to change:
package cosmos.gov.v1beta1;
to
package cosmos.gov.v1;

Yes as part of a new folder proto/cosmos/gov/v1
If we go that route, we should also keep proto/cosmos/gov/v1beta1 but just generate the files in some legacy package so we can still use the previous type definitions for migrations as in #9492

But before that, let's make sure this is what we wanna do here, thoughts @aaronc @AmauryM?

amaury1093 · 2021-07-30T11:23:24Z

But before that, let's make sure this is what we wanna do here, thoughts @aaronc @AmauryM?

I'm not part of the WG, so I didn't follow how confident you guys are on this API, but I put some thoughts here: #9810 (comment)

If we go that route, we should also keep proto/cosmos/gov/v1beta1 but just generate the files in some legacy package so we can still use the previous type definitions for migrations as in #9492

We actually didn't finalize the decision on how to handle proto version bumps in #9613. Legacy package is a solution, but aaron also proposed for 1 node to handle 2 versions at the same time.

blushi · 2021-07-30T11:47:16Z

If we go that route, we should also keep proto/cosmos/gov/v1beta1 but just generate the files in some legacy package so we can still use the previous type definitions for migrations as in #9492

We actually didn't finalize the decision on how to handle proto version bumps in #9613. Legacy package is a solution, but aaron also proposed for 1 node to handle 2 versions at the same time.

Right, thanks for pointing that out!

@cmwaters FYI This is something that is on the agenda for today's architecture call: https://hackmd.io/vh9ck9qvSiaM1ivb50NWMg
Hopefully, we'll get more clarity on that soon.

cmwaters · 2021-07-30T12:55:53Z

Ok. I will wait out until a decision is made on this 👍

cmwaters · 2021-08-03T06:57:52Z

Hey, did you guys reach a decision on what path to take regarding breaking the proto structs?

amaury1093 · 2021-08-04T09:27:31Z

Hey, did you guys reach a decision on what path to take regarding breaking the proto structs?

I added an agenda item to our internal Regen meeting today to discuss if we're bumping all proto pkgs to v1 for 0.44 or not.

I'd like to add that if we're bumping to v1, then there's some groundwork to prepare (either the legacy migrations that Marie mentioned #9810 (comment), or for the SDK to support multiple proto versions at the same time). This groundwork should be ideally done separately, on a master PR, and might take 2 weeks.

So to unblock your work for this PR, maybe let's temporarily keep MsgSubmitProposal2 as proposed in #9810 (comment) ?

okwme · 2021-11-22T13:45:10Z

x/gov/client/testutil/deposits.go

+	govAcc, err := s.cfg.AccountRetriever.GetAccount(val.ClientCtx, authtypes.NewModuleAddress(types.ModuleName))
+	s.Require().NoError(err)
+
+	proposal := []sdk.Msg{types.NewMsgVote(govAcc.GetAddress(), 1, types.OptionYes)}


am i reading it correctly that the proposal msg that is getting voted on is itself a vote proposal?

yes lol. This is just a dummy proposal used in testing but essentially the governance account would vote on a proposal.

## Description Ref: #9810 Bumps gov proto files to v1beta2 and creates new `Proposal` type --- ### Author Checklist *All items are required. Please add a note to the item if the item is not applicable and please add links to any relevant follow up issues.* I have... - [ ] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] added `!` to the type prefix if API or client breaking change - [ ] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting)) - [ ] provided a link to the relevant issue or specification - [ ] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules) - [ ] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing) - [ ] added a changelog entry to `CHANGELOG.md` - [ ] included comments for [documenting Go code](https://blog.golang.org/godoc) - [ ] updated the relevant documentation or specification - [ ] reviewed "Files changed" and left comments if necessary - [ ] confirmed all CI checks have passed ### Reviewers Checklist *All items are required. Please add a note if the item is not applicable and please add your handle next to the items reviewed if you only reviewed selected items.* I have... - [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] confirmed `!` in the type prefix if API or client breaking change - [ ] confirmed all author checklist items have been addressed - [ ] reviewed state machine logic - [ ] reviewed API design and naming - [ ] reviewed documentation is accurate - [ ] reviewed tests and test coverage - [ ] manually tested (if applicable)

tac0turtle · 2021-11-23T10:04:45Z

putting a short merge date on this. In two days we will merge this PR with any open questions landing in a follow up PRs

aaronc · 2021-11-23T14:57:41Z

putting a short merge date on this. In two days we will merge this PR with any open questions landing in a follow up PRs

I'm all for trying to move this quickly. But we also need to make sure simulations aren't broken

tac0turtle · 2021-11-23T17:26:15Z

putting a short merge date on this. In two days we will merge this PR with any open questions landing in a follow up PRs

I'm all for trying to move this quickly. But we also need to make sure simulations aren't broken

didn't see the failing tests. @cmwaters once we fix these we will be able to merge.

cmwaters · 2021-11-24T10:13:00Z

How do I go about debugging the sims test?

aaronc · 2021-11-24T14:23:31Z

I'm maybe not the best resource here. @alexanderbez @AmauryM does either of you have advice?

cmwaters · 2021-11-25T11:52:53Z

For additional context, I've run this failing simulation:

go test ./simapp -run TestAppImportExport -Enabled=true -NumBlocks=50 -Genesis= -Verbose=true -Commit=true -Seed=89182391 -Period=5 -ExportParamsPath /tmp/sim-logs-1633807703/sim_params-89182391.json -ExportStatePath /tmp/sim-logs-1633807703/sim_state-89182391.json -v -timeout 24h

and get:

simulation halted due to panic on block 35
Logs to writing to /home/callum/.simapp/simulations/2021-11-25_12:50:24.log
--- FAIL: TestAppImportExport (25.18s)
panic: invariant broken: staking: delegator shares invariant
broken delegator shares invariance:
        validator.DelegatorShares: 34148342106.937500000000000000
        sum of Delegator.Shares: 34148342106.000000000000000000


        CRITICAL please submit the following transaction:
                 tx crisis invariant-broken staking delegator-shares [recovered]
        panic: invariant broken: staking: delegator shares invariant

x/gov/types/query.pb.go

## Description Ref: cosmos#9810 Bumps gov proto files to v1beta2 and creates new `Proposal` type --- ### Author Checklist *All items are required. Please add a note to the item if the item is not applicable and please add links to any relevant follow up issues.* I have... - [ ] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] added `!` to the type prefix if API or client breaking change - [ ] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting)) - [ ] provided a link to the relevant issue or specification - [ ] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules) - [ ] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing) - [ ] added a changelog entry to `CHANGELOG.md` - [ ] included comments for [documenting Go code](https://blog.golang.org/godoc) - [ ] updated the relevant documentation or specification - [ ] reviewed "Files changed" and left comments if necessary - [ ] confirmed all CI checks have passed ### Reviewers Checklist *All items are required. Please add a note if the item is not applicable and please add your handle next to the items reviewed if you only reviewed selected items.* I have... - [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] confirmed `!` in the type prefix if API or client breaking change - [ ] confirmed all author checklist items have been addressed - [ ] reviewed state machine logic - [ ] reviewed API design and naming - [ ] reviewed documentation is accurate - [ ] reviewed tests and test coverage - [ ] manually tested (if applicable)

amaury1093 · 2021-12-13T16:07:06Z

Should we close this PR?

Ref: #9810 Moves all legacy gov code to `v1beta1`. This preserves all existing behavior (i.e. everything still uses v1beta1). It's merely moving things around to get everything in the right place logistically (hence the large diff still) --- ### Author Checklist *All items are required. Please add a note to the item if the item is not applicable and please add links to any relevant follow up issues.* I have... - [ ] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] added `!` to the type prefix if API or client breaking change - [ ] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting)) - [ ] provided a link to the relevant issue or specification - [ ] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules) - [ ] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing) - [ ] added a changelog entry to `CHANGELOG.md` - [ ] included comments for [documenting Go code](https://blog.golang.org/godoc) - [ ] updated the relevant documentation or specification - [ ] reviewed "Files changed" and left comments if necessary - [ ] confirmed all CI checks have passed ### Reviewers Checklist *All items are required. Please add a note if the item is not applicable and please add your handle next to the items reviewed if you only reviewed selected items.* I have... - [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] confirmed `!` in the type prefix if API or client breaking change - [ ] confirmed all author checklist items have been addressed - [ ] reviewed state machine logic - [ ] reviewed API design and naming - [ ] reviewed documentation is accurate - [ ] reviewed tests and test coverage - [ ] manually tested (if applicable)

Ref: cosmos#9810 Moves all legacy gov code to `v1beta1`. This preserves all existing behavior (i.e. everything still uses v1beta1). It's merely moving things around to get everything in the right place logistically (hence the large diff still) --- ### Author Checklist *All items are required. Please add a note to the item if the item is not applicable and please add links to any relevant follow up issues.* I have... - [ ] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] added `!` to the type prefix if API or client breaking change - [ ] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting)) - [ ] provided a link to the relevant issue or specification - [ ] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules) - [ ] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing) - [ ] added a changelog entry to `CHANGELOG.md` - [ ] included comments for [documenting Go code](https://blog.golang.org/godoc) - [ ] updated the relevant documentation or specification - [ ] reviewed "Files changed" and left comments if necessary - [ ] confirmed all CI checks have passed ### Reviewers Checklist *All items are required. Please add a note if the item is not applicable and please add your handle next to the items reviewed if you only reviewed selected items.* I have... - [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title - [ ] confirmed `!` in the type prefix if API or client breaking change - [ ] confirmed all author checklist items have been addressed - [ ] reviewed state machine logic - [ ] reviewed API design and naming - [ ] reviewed documentation is accurate - [ ] reviewed tests and test coverage - [ ] manually tested (if applicable)

cmwaters requested review from aaronc and alexanderbez as code owners July 28, 2021 19:52

github-actions bot added C:CLI C:Simulations C:x/distribution distribution module related C:x/gov C:x/params C:x/upgrade labels Jul 28, 2021

cmwaters requested a review from blushi July 28, 2021 19:53

aaronc reviewed Jul 28, 2021

View reviewed changes

proto/cosmos/gov/v1beta1/gov.proto Outdated Show resolved Hide resolved

blushi reviewed Jul 29, 2021

View reviewed changes

x/gov/keeper/proposal.go Outdated Show resolved Hide resolved

x/gov/types/proposal.go Show resolved Hide resolved

cmwaters commented Jul 29, 2021

View reviewed changes

x/gov/types/msgs.go Outdated Show resolved Hide resolved

blushi self-assigned this Aug 10, 2021

cmwaters mentioned this pull request Aug 10, 2021

!feat: allow arbitrary messages in gov proposal #9809

Closed

27 tasks

github-actions bot added C:Cosmovisor Issues and PR related to Cosmovisor C:Keys Keybase, KMS and HSMs C:Rosetta Issues and PR related to Rosetta C:x/auth C:x/authz and removed C:x/params C:x/upgrade C:x/distribution distribution module related labels Aug 27, 2021

okwme reviewed Nov 22, 2021

View reviewed changes

Merge branch 'master' into callum/gov-proposal-messages

89dbdf5

cmwaters added 2 commits November 23, 2021 22:06

Merge branch 'master' into callum/gov-proposal-messages

17e3400

run go mod tidy

9a5d243

Merge branch 'master' into callum/gov-proposal-messages

f2ce991

github-actions bot added the C:x/nft label Nov 24, 2021

cmwaters and others added 8 commits November 25, 2021 12:53

minor update

c37019b

Merge branch 'master' into callum/gov-proposal-messages

ed4c196

Merge branch 'master' into callum/gov-proposal-messages

b9b4345

add back v1beta1 query and tx proto files

0c64c1a

generate new proto

a6d4edd

remove tx proto

ab39d38

rename weights

dbf7e9c

Merge branch 'master' into callum/gov-proposal-messages

7faf1f2

amaury1093 reviewed Dec 8, 2021

View reviewed changes

x/gov/types/query.pb.go Show resolved Hide resolved

cmwaters mentioned this pull request Dec 13, 2021

refactor: move legacy gov to v1beta1 #10748

Merged

19 tasks

aleem1314 mentioned this pull request Dec 13, 2021

simulations are failing on some PR's #10750

Closed

4 tasks

blushi closed this Dec 14, 2021

tac0turtle deleted the callum/gov-proposal-messages branch February 7, 2022 15:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add array of messages to a proposal #9810

feat: add array of messages to a proposal #9810

cmwaters commented Jul 28, 2021 •

edited

Loading

blushi left a comment

orijbot commented Jul 29, 2021

cmwaters commented Jul 30, 2021

blushi commented Jul 30, 2021

amaury1093 commented Jul 30, 2021 •

edited

Loading

blushi commented Jul 30, 2021 •

edited

Loading

cmwaters commented Jul 30, 2021

cmwaters commented Aug 3, 2021

amaury1093 commented Aug 4, 2021 •

edited

Loading

okwme Nov 22, 2021

cmwaters Nov 22, 2021

tac0turtle commented Nov 23, 2021

aaronc commented Nov 23, 2021

tac0turtle commented Nov 23, 2021

cmwaters commented Nov 24, 2021

aaronc commented Nov 24, 2021

cmwaters commented Nov 25, 2021

amaury1093 commented Dec 13, 2021

feat: add array of messages to a proposal #9810

feat: add array of messages to a proposal #9810

Conversation

cmwaters commented Jul 28, 2021 • edited Loading

Description

Author Checklist

Reviewers Checklist

blushi left a comment

Choose a reason for hiding this comment

orijbot commented Jul 29, 2021

cmwaters commented Jul 30, 2021

blushi commented Jul 30, 2021

amaury1093 commented Jul 30, 2021 • edited Loading

blushi commented Jul 30, 2021 • edited Loading

cmwaters commented Jul 30, 2021

cmwaters commented Aug 3, 2021

amaury1093 commented Aug 4, 2021 • edited Loading

okwme Nov 22, 2021

Choose a reason for hiding this comment

cmwaters Nov 22, 2021

Choose a reason for hiding this comment

tac0turtle commented Nov 23, 2021

aaronc commented Nov 23, 2021

tac0turtle commented Nov 23, 2021

cmwaters commented Nov 24, 2021

aaronc commented Nov 24, 2021

cmwaters commented Nov 25, 2021

amaury1093 commented Dec 13, 2021

cmwaters commented Jul 28, 2021 •

edited

Loading

amaury1093 commented Jul 30, 2021 •

edited

Loading

blushi commented Jul 30, 2021 •

edited

Loading

amaury1093 commented Aug 4, 2021 •

edited

Loading