Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem: sim tests are removed #304

Merged
merged 5 commits into from
Sep 7, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 81 additions & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -129,4 +129,84 @@ jobs:
name: ethermint
signingKey: "${{ secrets.CACHIX_SIGNING_KEY }}"
- name: 'instantiate integration test env'
run: nix-store -r "$(nix-instantiate tests/integration_tests/shell.nix)"
run: nix-store -r $(nix-instantiate tests/integration_tests/shell.nix)

test-sim-nondeterminism:
runs-on: ubuntu-latest
timeout-minutes: 120
steps:
- uses: actions/setup-go@v3
with:
go-version: 1.19
check-latest: true
- uses: actions/checkout@v3
- uses: technote-space/get-diff-action@v6.1.1

Check warning

Code scanning / Semgrep

Semgrep Finding: yaml.github-actions.security.third-party-action-not-pinned-to-commit-sha.third-party-action-not-pinned-to-commit-sha

An action sourced from a third-party repository on GitHub is not pinned to a full length commit SHA. Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload.
with:
PATTERNS: |
**/**.go
go.mod
go.sum
- name: Test simulation nondeterminism
run: |
make test-sim-nondeterminism
if: env.GIT_DIFF

test-sim-random-genesis-fast:
runs-on: ubuntu-latest
timeout-minutes: 120
steps:
- uses: actions/setup-go@v3
with:
go-version: 1.19
check-latest: true
- uses: actions/checkout@v3
- uses: technote-space/get-diff-action@v6.1.1

Check warning

Code scanning / Semgrep

Semgrep Finding: yaml.github-actions.security.third-party-action-not-pinned-to-commit-sha.third-party-action-not-pinned-to-commit-sha

An action sourced from a third-party repository on GitHub is not pinned to a full length commit SHA. Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload.
with:
PATTERNS: |
**/**.go
go.mod
go.sum
- name: Test simulation with random genesis
run: |
make test-sim-random-genesis-fast
if: env.GIT_DIFF

test-sim-import-export:
runs-on: ubuntu-latest
timeout-minutes: 120
steps:
- uses: actions/setup-go@v3
with:
go-version: 1.19
check-latest: true
- uses: actions/checkout@v3
- uses: technote-space/get-diff-action@v6.1.1

Check warning

Code scanning / Semgrep

Semgrep Finding: yaml.github-actions.security.third-party-action-not-pinned-to-commit-sha.third-party-action-not-pinned-to-commit-sha

An action sourced from a third-party repository on GitHub is not pinned to a full length commit SHA. Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload.
with:
PATTERNS: |
**/**.go
go.mod
go.sum
- name: Simulation of import and export genesis
run: |
make test-sim-import-export
if: env.GIT_DIFF

test-sim-after-import:
runs-on: ubuntu-latest
timeout-minutes: 120
steps:
- uses: actions/setup-go@v3
with:
go-version: 1.19
check-latest: true
- uses: actions/checkout@v3
- uses: technote-space/get-diff-action@v6.1.1

Check warning

Code scanning / Semgrep

Semgrep Finding: yaml.github-actions.security.third-party-action-not-pinned-to-commit-sha.third-party-action-not-pinned-to-commit-sha

An action sourced from a third-party repository on GitHub is not pinned to a full length commit SHA. Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA helps mitigate the risk of a bad actor adding a backdoor to the action's repository, as they would need to generate a SHA-1 collision for a valid Git object payload.
with:
PATTERNS: |
**/**.go
go.mod
go.sum
- name: Test simulation after import
run: |
make test-sim-after-import
if: env.GIT_DIFF
45 changes: 45 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -359,6 +359,51 @@ test-solidity:

.PHONY: run-tests test test-all test-import test-rpc test-contract test-solidity $(TEST_TARGETS)

test-sim-nondeterminism:
@echo "Running non-determinism test..."
@go test -mod=readonly $(SIMAPP) -run TestAppStateDeterminism -Enabled=true \
-NumBlocks=100 -BlockSize=200 -Commit=true -Period=0 -v -timeout 24h

test-sim-random-genesis-fast:
@echo "Running random genesis simulation..."
@go test -mod=readonly $(SIMAPP) -run TestFullAppSimulation \
-Enabled=true -NumBlocks=100 -BlockSize=200 -Commit=true -Seed=99 -Period=5 -v -timeout 24h

test-sim-import-export: runsim
@echo "Running application import/export simulation. This may take several minutes..."
@$(BINDIR)/runsim -Jobs=4 -SimAppPkg=$(SIMAPP) -ExitOnFail 50 5 TestAppImportExport

test-sim-after-import: runsim
@echo "Running application simulation-after-import. This may take several minutes..."
@$(BINDIR)/runsim -Jobs=4 -SimAppPkg=$(SIMAPP) -ExitOnFail 50 5 TestAppSimulationAfterImport

test-sim-random-genesis-multi-seed: runsim
@echo "Running multi-seed custom genesis simulation..."
@$(BINDIR)/runsim -SimAppPkg=$(SIMAPP) -ExitOnFail 400 5 TestFullAppSimulation

test-sim-multi-seed-long: runsim
@echo "Running long multi-seed application simulation. This may take awhile!"
@$(BINDIR)/runsim -Jobs=4 -SimAppPkg=$(SIMAPP) -ExitOnFail 500 50 TestFullAppSimulation

test-sim-multi-seed-short: runsim
@echo "Running short multi-seed application simulation. This may take awhile!"
@$(BINDIR)/runsim -Jobs=4 -SimAppPkg=$(SIMAPP) -ExitOnFail 50 10 TestFullAppSimulation

test-sim-benchmark-invariants:
@echo "Running simulation invariant benchmarks..."
@go test -mod=readonly $(SIMAPP) -benchmem -bench=BenchmarkInvariants -run=^$ \
-Enabled=true -NumBlocks=1000 -BlockSize=200 \
-Period=1 -Commit=true -Seed=57 -v -timeout 24h

.PHONY: \
test-sim-nondeterminism \
test-sim-custom-genesis-fast \
test-sim-import-export \
test-sim-after-import \
test-sim-custom-genesis-multi-seed \
test-sim-multi-seed-short \
test-sim-multi-seed-long \
test-sim-benchmark-invariants

benchmark:
@go test -mod=readonly -bench=. $(PACKAGES_NOSIMULATION)
Expand Down
1 change: 1 addition & 0 deletions app/ante/signverify_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import (
evmtypes "github.com/evmos/ethermint/x/evm/types"
)


func (suite AnteTestSuite) TestEthSigVerificationDecorator() {
addr, privKey := tests.NewAddrKey()

Expand Down
21 changes: 21 additions & 0 deletions app/app.go
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,8 @@ var (
allowedReceivingModAcc = map[string]bool{}
)

var _ simapp.App = (*EthermintApp)(nil)

// var _ server.Application (*EthermintApp)(nil)

// EthermintApp implements an extended ABCI application. It is an application
Expand Down Expand Up @@ -245,6 +247,9 @@ type EthermintApp struct {
// the module manager
mm *module.Manager

// simulation manager
sm *module.SimulationManager

// the configurator
configurator module.Configurator
}
Expand Down Expand Up @@ -615,6 +620,17 @@ func NewEthermintApp(
// add test gRPC service for testing gRPC queries in isolation
// testdata.RegisterTestServiceServer(app.GRPCQueryRouter(), testdata.TestServiceImpl{})

// create the simulation manager and define the order of the modules for deterministic simulations
//
// NOTE: this is not required apps that don't use the simulator for fuzz testing
// transactions
overrideModules := map[string]module.AppModuleSimulation{
authtypes.ModuleName: auth.NewAppModule(app.appCodec, app.AccountKeeper, authsims.RandomGenesisAccounts),
}
app.sm = module.NewSimulationManagerFromAppModules(app.mm.Modules, overrideModules)

app.sm.RegisterStoreDecoders()

// initialize stores
app.MountKVStores(keys)
app.MountTransientStores(tkeys)
Expand Down Expand Up @@ -788,6 +804,11 @@ func (app *EthermintApp) GetSubspace(moduleName string) paramstypes.Subspace {
return subspace
}

// SimulationManager implements the SimulationApp interface
func (app *EthermintApp) SimulationManager() *module.SimulationManager {
return app.sm
}

// RegisterAPIRoutes registers all application module routes with the provided
// API server.
func (app *EthermintApp) RegisterAPIRoutes(apiSvr *api.Server, apiConfig config.APIConfig) {
Expand Down
Loading