Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: flaky simulation test: unique constraint violation #15741

Closed
mark-rushakoff opened this issue Apr 7, 2023 · 2 comments · Fixed by #15943
Closed

[Bug]: flaky simulation test: unique constraint violation #15741

mark-rushakoff opened this issue Apr 7, 2023 · 2 comments · Fixed by #15943
Labels

Comments

@mark-rushakoff
Copy link
Member

Summary of Bug

https://github.com/cosmos/cosmos-sdk/actions/runs/4638318575/jobs/8208005708?pr=15727 failed with stack:

--- FAIL: TestAppStateDeterminism (0.27s)
panic: group policies: unique constraint violation [recovered]
	panic: group policies: unique constraint violation

goroutine 230 [running]:
testing.tRunner.func1.2({0x25139e0, 0xc0002e4b60})
	/opt/hostedtoolcache/go/1.20.3/x64/src/testing/testing.go:1526 +0x24e
testing.tRunner.func1()
	/opt/hostedtoolcache/go/1.20.3/x64/src/testing/testing.go:1529 +0x39f
panic({0x25139e0, 0xc0002e4b60})
	/opt/hostedtoolcache/go/1.20.3/x64/src/runtime/panic.go:884 +0x213
github.com/cosmos/cosmos-sdk/x/group/keeper.Keeper.InitGenesis({{0x3353b40, 0xc000c9bda0}, {0x7fcdd4386590, 0xc000d21560}, {0xc001827500, {0x1}}, {0x3374660, 0xc000981380}, {0xc001827560}, {0x3374660, ...}, ...}, ...)
	/home/runner/work/cosmos-sdk/cosmos-sdk/x/group/keeper/genesis.go:29 +0x645
github.com/cosmos/cosmos-sdk/x/group/module.AppModule.InitGenesis(...)
	/home/runner/work/cosmos-sdk/cosmos-sdk/x/group/module/module.go:133
github.com/cosmos/cosmos-sdk/types/module.(*Manager).InitGenesis(_, {{0x336f7d8, 0xc0000dc000}, {0x3382c10, 0xc0023fe640}, {{0x0, 0x0}, {0x27839ae, 0xe}, 0x0, ...}, ...}, ...)
	/home/runner/work/cosmos-sdk/cosmos-sdk/types/module/module.go:419 +0x3b3
github.com/cosmos/cosmos-sdk/runtime.(*App).InitChainer(_, {{0x336f7d8, 0xc0000dc000}, {0x3382c10, 0xc0023fe640}, {{0x0, 0x0}, {0x27839ae, 0xe}, 0x0, ...}, ...}, ...)
	/home/runner/work/cosmos-sdk/cosmos-sdk/runtime/app.go:134 +0x12e
github.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).InitChain(0xc0003d4e00, {{0x0, 0xf3e97aec7, 0x4876d80}, {0x27839ae, 0xe}, 0xc000149e00, {0x0, 0x0, 0x0}, ...})
	/home/runner/work/cosmos-sdk/cosmos-sdk/baseapp/abci.go:83 +0x635
github.com/cosmos/cosmos-sdk/x/simulation.initChain(0x0?, {0x3fda9e03ce8b6356, 0x3, _, {_, _, _}, {_, _}, {_, ...}}, ...)
	/home/runner/work/cosmos-sdk/cosmos-sdk/x/simulation/simulate.go:42 +0x265
github.com/cosmos/cosmos-sdk/x/simulation.SimulateFromSeed({0x3398660?, 0xc000728340?}, {0x334fb60?, _}, _, _, _, {0xc003085500, 0x2e, 0x38}, ...)
	/home/runner/work/cosmos-sdk/cosmos-sdk/x/simulation/simulate.go:76 +0x4b5
cosmossdk.io/simapp.TestAppStateDeterminism(0xc000728340)
	/home/runner/work/cosmos-sdk/cosmos-sdk/simapp/sim_test.go:371 +0xcc8
testing.tRunner(0xc000728340, 0x2efc168)
	/opt/hostedtoolcache/go/1.20.3/x64/src/testing/testing.go:1576 +0x10b
created by testing.(*T).Run
	/opt/hostedtoolcache/go/1.20.3/x64/src/testing/testing.go:1629 +0x3ea
exit status 2
FAIL	cosmossdk.io/simapp	0.357s

Version

main branch at commit df161c2

Steps to Reproduce

Apply this patch to use the fixed seed from the failure:

diff --git a/simapp/sim_test.go b/simapp/sim_test.go
index b30f1771c..d89088a13 100644
--- a/simapp/sim_test.go
+++ b/simapp/sim_test.go
@@ -351,6 +351,7 @@ func TestAppStateDeterminism(t *testing.T) {

        for i := 0; i < numSeeds; i++ {
                config.Seed = rand.Int63()
+               config.Seed = 7748983398051311403

                for j := 0; j < numTimesToRunPerSeed; j++ {
                        var logger log.Logger

(the -Seed flag isn't respected in this test)

The failure reproduces immediately.

Upon investigating the state, the group policies look like:

[
  {
    "address": "cosmos138ya066vjg82r7ldrhnw9z99auvk24ksqu4ze9",
    "group_id": "1",
    "admin": "cosmos138ya066vjg82r7ldrhnw9z99auvk24ksqu4ze9",
    "metadata": "MGZkMAdSAQ",
    "version": "1",
    "decision_policy": {
      "@type": "/cosmos.group.v1.ThresholdDecisionPolicy",
      "threshold": "10",
      "windows": {
        "voting_period": "1s",
        "min_execution_period": "0s"
      }
    },
    "created_at": "0001-01-01T00:00:00Z"
  },
  {
    "address": "cosmos1h85jr2t5n0ma8cl0s5v3zg9xdufkjm6a9h2sv7",
    "group_id": "2",
    "admin": "cosmos1h85jr2t5n0ma8cl0s5v3zg9xdufkjm6a9h2sv7",
    "metadata": "tSdwuNguAr",
    "version": "1",
    "decision_policy": {
      "@type": "/cosmos.group.v1.ThresholdDecisionPolicy",
      "threshold": "10",
      "windows": {
        "voting_period": "1s",
        "min_execution_period": "0s"
      }
    },
    "created_at": "0001-01-01T00:00:00Z"
  },
  {
    "address": "cosmos138ya066vjg82r7ldrhnw9z99auvk24ksqu4ze9",
    "group_id": "3",
    "admin": "cosmos138ya066vjg82r7ldrhnw9z99auvk24ksqu4ze9",
    "metadata": "nbQqkXjdmH",
    "version": "1",
    "decision_policy": {
      "@type": "/cosmos.group.v1.ThresholdDecisionPolicy",
      "threshold": "10",
      "windows": {
        "voting_period": "1s",
        "min_execution_period": "0s"
      }
    },
    "created_at": "0001-01-01T00:00:00Z"
  }
]

The address and admin are identical on the first and last entry. Since the failure mentions an invalid unique constraint, maybe we need to prevent generating that state?

@mark-rushakoff mark-rushakoff changed the title [Bug]: flaky simulation test [Bug]: flaky simulation test: unique constraint violation Apr 7, 2023
@mark-rushakoff
Copy link
Member Author

This happened again at https://github.com/cosmos/cosmos-sdk/actions/runs/4679650341/jobs/8289964158#step:5:4980 for #15819 (changing unrelated CI scripts), with stack trace:

--- FAIL: TestAppStateDeterminism (1430.48s)
panic: group policies: unique constraint violation [recovered]
	panic: group policies: unique constraint violation

goroutine 227 [running]:
testing.tRunner.func1.2({0x254b3e0, 0xc00576d4c0})
	/opt/hostedtoolcache/go/1.20.3/x64/src/testing/testing.go:1526 +0x24e
testing.tRunner.func1()
	/opt/hostedtoolcache/go/1.20.3/x64/src/testing/testing.go:1529 +0x39f
panic({0x254b3e0, 0xc00576d4c0})
	/opt/hostedtoolcache/go/1.20.3/x64/src/runtime/panic.go:884 +0x213
github.com/cosmos/cosmos-sdk/x/group/keeper.Keeper.InitGenesis({{0x3391fc8, 0xc041c68c00}, {0x7f4f6c356dd8, 0xc05af708f0}, {0xc057d997a0, {0x1}}, {0x33b29e0, 0xc05bbfb940}, {0xc057d99800}, {0x33b29e0, ...}, ...}, ...)
	/home/runner/work/cosmos-sdk/cosmos-sdk/x/group/keeper/genesis.go:29 +0x645
github.com/cosmos/cosmos-sdk/x/group/module.AppModule.InitGenesis(...)
	/home/runner/work/cosmos-sdk/cosmos-sdk/x/group/module/module.go:133
github.com/cosmos/cosmos-sdk/types/module.(*Manager).InitGenesis(_, {{0x33add58, 0xc000058128}, {0x33c1530, 0xc006b6af40}, {{0x0, 0x0}, {0x27bfc3a, 0xe}, 0x0, ...}, ...}, ...)
	/home/runner/work/cosmos-sdk/cosmos-sdk/types/module/module.go:419 +0x3b3
github.com/cosmos/cosmos-sdk/runtime.(*App).InitChainer(_, {{0x33add58, 0xc000058128}, {0x33c1530, 0xc006b6af40}, {{0x0, 0x0}, {0x27bfc3a, 0xe}, 0x0, ...}, ...}, ...)
	/home/runner/work/cosmos-sdk/cosmos-sdk/runtime/app.go:134 +0x12e
github.com/cosmos/cosmos-sdk/baseapp.(*BaseApp).InitChain(0xc05bb28400, {{0x0, 0x1054c3d9b2, 0x48cef60}, {0x27bfc3a, 0xe}, 0xc0036a62a0, {0x0, 0x0, 0x0}, ...})
	/home/runner/work/cosmos-sdk/cosmos-sdk/baseapp/abci.go:84 +0x5b5
github.com/cosmos/cosmos-sdk/x/simulation.initChain(0x0?, {0x3f92f1707cee6af8, 0x19, _, {_, _, _}, {_, _}, {_, ...}}, ...)
	/home/runner/work/cosmos-sdk/cosmos-sdk/x/simulation/simulate.go:42 +0x265
github.com/cosmos/cosmos-sdk/x/simulation.SimulateFromSeed({0x33d70c0?, 0xc000647d40?}, {0x338dd80?, _}, _, _, _, {0xc00d05ae00, 0x2e, 0x38}, ...)
	/home/runner/work/cosmos-sdk/cosmos-sdk/x/simulation/simulate.go:76 +0x4b5
cosmossdk.io/simapp.TestAppStateDeterminism(0xc000647d40)
	/home/runner/work/cosmos-sdk/cosmos-sdk/simapp/sim_test.go:371 +0xcc8
testing.tRunner(0xc000647d40, 0x2f38500)
	/opt/hostedtoolcache/go/1.20.3/x64/src/testing/testing.go:1576 +0x10b
created by testing.(*T).Run
	/opt/hostedtoolcache/go/1.20.3/x64/src/testing/testing.go:1629 +0x3ea
exit status 2
FAIL	cosmossdk.io/simapp	1430.635s
make: *** [Makefile:258: test-sim-nondeterminism] Error 1

and with seed 8979509998758717859.

@gjermundgaraba
Copy link
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants