Skip to content
This repository has been archived by the owner on Nov 30, 2021. It is now read-only.

SDK modules simulations #269

Merged
merged 3 commits into from
Apr 29, 2020
Merged

SDK modules simulations #269

merged 3 commits into from
Apr 29, 2020

Conversation

fedekunze
Copy link
Contributor

@fedekunze fedekunze commented Apr 24, 2020

Closes #187

Description

Adds support for simulations


For contributor use:

  • Targeted PR against correct branch (see CONTRIBUTING.md)
  • Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
  • Code follows the module structure standards.
  • Wrote unit and integration tests
  • Updated relevant documentation (docs/) or specification (x/<module>/spec/)
  • Added relevant godoc comments.
  • Added a relevant changelog entry to the Unreleased section in CHANGELOG.md
  • Re-reviewed Files changed in the Github PR explorer

For admin use:

  • Added appropriate labels to PR (ex. WIP, R4R, docs, etc)
  • Reviewers assigned
  • Squashed all commits, uses message "Merge pull request #XYZ: [title]" (coding standards)

@fedekunze fedekunze marked this pull request as ready for review April 24, 2020 20:16
@thomasmodeneis
Copy link

Are you planning to add this tests to the CI ?

@fedekunze
Copy link
Contributor Author

not until travis is merged. Feel free to test it

@noot
Copy link
Contributor

noot commented Apr 28, 2020

@fedekunze I got an error when running the second test:

$ make test-sim-custom-genesis-fast
Running custom genesis simulation...
By default, /home/elizabeth/.emintd/config/genesis.json will be used.
=== RUN   TestFullAppSimulation
Starting SimulateFromSeed with randomness created with seed 99
Randomized simulation params: 
{}
Starting the simulation from time Tue Apr 28 14:43:50 UTC 2020 (unixtime 1588085030)
Simulating... block 1/100, operation 0/0.Logs to writing to /home/elizabeth/.simapp/simulations/2020-04-28_15:48:21.log
--- FAIL: TestFullAppSimulation (0.14s)
    simulate.go:284: error on block  1/100, operation (0/268) from x/staking:
        pubKey does not match signer address cosmos19mnmuhxrj9l07fgy5zhpl9tu30928cux24dm09 with signer index: 0: invalid pubkey [cosmos/cosmos-sdk@v0.34.4-0.20200403200637-7f78e61b93a5/x/auth/ante/sigverify.go:80]
        Comment: 
FAIL
FAIL	github.com/cosmos/ethermint/app	0.231s
FAIL
Makefile:266: recipe for target 'test-sim-custom-genesis-fast' failed
make: *** [test-sim-custom-genesis-fast] Error 1

let me know if I'm running it incorrectly

@fedekunze
Copy link
Contributor Author

@noot test-sim-custom-genesis-fast runs the simulation with a genesis state provided. Probably the simulation account generates a tendermint public key which is different from the one we're using on ethermint.

I just run make test-sim-multi-seed-short which worked fine:

Running multi-seed application simulation. This may take awhile!
Allocating 4 workers...
[W3] Worker is up and running
[W2] Worker is up and running
[W1] Worker is up and running
[W0] Worker is up and running
[W3] Spawned simulation with pid 52602 [seed=1 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-1-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-1-date-04-29-2020_092052.stderr]
[W2] Spawned simulation with pid 52605 [seed=2 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-2-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-2-date-04-29-2020_092052.stderr]
[W1] Spawned simulation with pid 52603 [seed=4 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-4-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-4-date-04-29-2020_092052.stderr]
[W0] Spawned simulation with pid 52604 [seed=7 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-7-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-7-date-04-29-2020_092052.stderr]
[W1] Spawned simulation with pid 52714 [seed=32 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-32-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-32-date-04-29-2020_092052.stderr]
[W0] Spawned simulation with pid 52743 [seed=123 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-123-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-123-date-04-29-2020_092052.stderr]
[W1] Spawned simulation with pid 52767 [seed=124 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-124-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-124-date-04-29-2020_092052.stderr]
.
[W2] Spawned simulation with pid 52795 [seed=582 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-582-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-582-date-04-29-2020_092052.stderr]
[W0] Spawned simulation with pid 52822 [seed=1893 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-1893-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-1893-date-04-29-2020_092052.stderr]
.
[W1] Spawned simulation with pid 52850 [seed=2989 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-2989-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-2989-date-04-29-2020_092052.stderr]
[W0] Spawned simulation with pid 52879 [seed=3012 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-3012-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-3012-date-04-29-2020_092052.stderr]
[W0] Spawned simulation with pid 52907 [seed=4728 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-4728-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-4728-date-04-29-2020_092052.stderr]
[W1] Spawned simulation with pid 52935 [seed=37827 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-37827-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-37827-date-04-29-2020_092052.stderr]
[W3] Spawned simulation with pid 52964 [seed=981928 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-981928-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-981928-date-04-29-2020_092052.stderr]
.
[W1] Spawned simulation with pid 52991 [seed=87821 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-87821-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-87821-date-04-29-2020_092052.stderr]
[W0] Spawned simulation with pid 53019 [seed=891823782 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-891823782-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-891823782-date-04-29-2020_092052.stderr]
[W0] Spawned simulation with pid 53047 [seed=989182 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-989182-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-989182-date-04-29-2020_092052.stderr]
[W1] Spawned simulation with pid 53074 [seed=89182391 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-89182391-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-89182391-date-04-29-2020_092052.stderr]
.
[W2] Spawned simulation with pid 53104 [seed=11 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-11-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-11-date-04-29-2020_092052.stderr]
[W0] Spawned simulation with pid 53138 [seed=22 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-22-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-22-date-04-29-2020_092052.stderr]
[W1] Spawned simulation with pid 53164 [seed=44 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-44-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-44-date-04-29-2020_092052.stderr]
.
[W2] Spawned simulation with pid 53194 [seed=77 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-77-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-77-date-04-29-2020_092052.stderr]
[W0] Spawned simulation with pid 53220 [seed=99 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-99-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-99-date-04-29-2020_092052.stderr]
[W3] Spawned simulation with pid 53245 [seed=2020 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-2020-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-2020-date-04-29-2020_092052.stderr]
.
[W1] Spawned simulation with pid 53276 [seed=3232 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-3232-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-3232-date-04-29-2020_092052.stderr]
[W2] Spawned simulation with pid 53302 [seed=123123 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-123123-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-123123-date-04-29-2020_092052.stderr]
.
[W1] Spawned simulation with pid 53334 [seed=124124 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-124124-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-124124-date-04-29-2020_092052.stderr]
[W3] Spawned simulation with pid 53361 [seed=582582 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-582582-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-582582-date-04-29-2020_092052.stderr]
[W0] Spawned simulation with pid 53393 [seed=18931893 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-18931893-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-18931893-date-04-29-2020_092052.stderr]
[W1] Spawned simulation with pid 53414 [seed=29892989 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-29892989-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-29892989-date-04-29-2020_092052.stderr]
[W3] Spawned simulation with pid 53450 [seed=30123012 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-30123012-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-30123012-date-04-29-2020_092052.stderr]
[W1] Spawned simulation with pid 53479 [seed=47284728 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-47284728-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-47284728-date-04-29-2020_092052.stderr]
.
[W1] Spawned simulation with pid 53505 [seed=7601778 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-7601778-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-7601778-date-04-29-2020_092052.stderr]
[W0] Spawned simulation with pid 53535 [seed=8090485 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-8090485-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-8090485-date-04-29-2020_092052.stderr]
[W0] Spawned simulation with pid 53561 [seed=977367484 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-977367484-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-977367484-date-04-29-2020_092052.stderr]
.
[W2] Spawned simulation with pid 53594 [seed=491163361 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-491163361-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-491163361-date-04-29-2020_092052.stderr]
[W3] Spawned simulation with pid 53625 [seed=424254581 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-424254581-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-424254581-date-04-29-2020_092052.stderr]
.
[W1] Spawned simulation with pid 53657 [seed=673398983 stdout=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-673398983-date-04-29-2020_092052.stdout stderr=/var/folders/48/dv7qpgqd7wqfxkqx9vnhgc2w0000gn/T/sim-logs-470857436/app-simulation-seed-673398983-date-04-29-2020_092052.stderr]
[W3] no seeds left, shutting down
[W0] no seeds left, shutting down
[W1] no seeds left, shutting down
.
[W2] no seeds left, shutting down

I propose to merge this and I'll investigate the simulation issue on the SDK

@fedekunze fedekunze merged commit 7c5e0af into development Apr 29, 2020
@fedekunze fedekunze deleted the fedekunze/simulations branch April 29, 2020 15:50
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Simulator for EthermintApp
3 participants