Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Makefile, bash scripts to test multiple nodes #20

Merged
merged 40 commits into from
May 22, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
018a959
Add multinode test scripts, makefile
asalzmann May 17, 2022
35576db
merge in TEST-40 (remove ibc v2 deps)
asalzmann May 17, 2022
a7f3d28
update gitignore, fix Stride-labs ref
asalzmann May 17, 2022
28f5ec7
Add Makefile that builds + tests chain
asalzmann May 17, 2022
23d2ba6
ignore strided
asalzmann May 17, 2022
6b52cc0
TEST-35 tweak markdown explanation; was able to run multinode local s…
riley-stride May 18, 2022
1bb3d8b
TEST-35 add test_helpers from osmosis repo, contains required testing…
riley-stride May 18, 2022
ee3e86e
TEST-35 add keeper test from osmosis
riley-stride May 18, 2022
4db9c1f
TEST-35: make test working without unit tests
riley-stride May 18, 2022
5023b2b
TEST-35 (chore): add .DS_Store to .gitignore
riley-stride May 18, 2022
b2704b7
Working docker build
shellvish May 20, 2022
337247a
updated to proper genesis file
shellvish May 20, 2022
631fe13
add test commands to multinode-local-testnet
asalzmann May 20, 2022
eb344fa
move vars, add peers
asalzmann May 20, 2022
c9625e9
update gitignore
asalzmann May 20, 2022
08e7bf0
update git ignore, rm comment
asalzmann May 20, 2022
ca7de8b
all nodes are validators and talk to each other
shellvish May 20, 2022
cddcad4
TEST-35 add readme for running stride validators
May 21, 2022
39f2f66
update node network ip addresses, map ports for local debugging
asalzmann May 21, 2022
dd4d86a
run gaia
asalzmann May 21, 2022
a7a2dd5
run gaia
asalzmann May 21, 2022
54d4f4a
delete stored scripts/state
shellvish May 21, 2022
c9674cc
updated init to be run with `make init`
shellvish May 21, 2022
909acb5
INTERMEDIATE-COMMIT GAIA
shellvish May 21, 2022
0cb2d61
INTERMEDIATE COMMIT - DOES NOT RUN
shellvish May 21, 2022
f83eaaa
STRIDE configs
May 21, 2022
b151823
WORKING MAKE INIT
shellvish May 21, 2022
0647fd9
relayer running
asalzmann May 21, 2022
94e8902
bump hermes to v0.14.0
asalzmann May 22, 2022
c7ca652
add rly keys
asalzmann May 22, 2022
c20e5c2
riley temp changes hermes debugging
May 22, 2022
bc6717c
merging hermes changes
May 22, 2022
ca80de3
trying to resolves cosmos addrs vs stride addrs
May 22, 2022
e6d5ca5
prefix stride accounts with strd
asalzmann May 22, 2022
98722d3
working relayer
asalzmann May 22, 2022
3f46028
Fix makefile and add documentation
shellvish May 22, 2022
d2ee5f1
Merge branch 'TEST-35' of https://github.com/Stride-Labs/stride into …
shellvish May 22, 2022
6d9172f
ibc transfer comment
asalzmann May 22, 2022
187fcbc
updated init instructions
shellvish May 22, 2022
a6b0025
fix conflicts
asalzmann May 22, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
281 changes: 281 additions & 0 deletions stride/Makefile
Original file line number Diff line number Diff line change
@@ -1,2 +1,283 @@
#!/usr/bin/make -f

PACKAGES_SIMTEST=$(shell go list ./... | grep '/simulation')
VERSION := $(shell echo $(shell git describe --tags) | sed 's/^v//')
COMMIT := $(shell git log -1 --format='%H')
LEDGER_ENABLED ?= true
SDK_PACK := $(shell go list -m github.com/cosmos/cosmos-sdk | sed 's/ /\@/g')
DOCKER := $(shell which docker)
BUILDDIR ?= $(CURDIR)/build
TEST_DOCKER_REPO=Stride-Labs/stridednode

export GO111MODULE = on

# process build tags

# set GCCEXE, UNAME_S, GCC, build_tags
build_tags = netgo
ifeq ($(LEDGER_ENABLED),true)
ifeq ($(OS),Windows_NT)
GCCEXE = $(shell where gcc.exe 2> NUL)
ifeq ($(GCCEXE),)
$(error gcc.exe not installed for ledger support, please install or set LEDGER_ENABLED=false)
else
build_tags += ledger
endif
else
UNAME_S = $(shell uname -s)
ifeq ($(UNAME_S),OpenBSD)
$(warning OpenBSD detected, disabling ledger support (https://github.com/cosmos/cosmos-sdk/issues/1988))
else
GCC = $(shell command -v gcc 2> /dev/null)
ifeq ($(GCC),)
$(error gcc not installed for ledger support, please install or set LEDGER_ENABLED=false)
else
build_tags += ledger
endif
endif
endif
endif

ifeq (cleveldb,$(findstring cleveldb,$(STRIDE_BUILD_OPTIONS)))
build_tags += gcc
else ifeq (rocksdb,$(findstring rocksdb,$(STRIDE_BUILD_OPTIONS)))
build_tags += gcc
endif
build_tags += $(BUILD_TAGS)
build_tags := $(strip $(build_tags))

whitespace :=
whitespace += $(whitespace)
comma := ,
build_tags_comma_sep := $(subst $(whitespace),$(comma),$(build_tags))

# process linker flags

ldflags = -X github.com/cosmos/cosmos-sdk/version.Name=stride \
-X github.com/cosmos/cosmos-sdk/version.AppName=strided \
-X github.com/cosmos/cosmos-sdk/version.Version=$(VERSION) \
-X github.com/cosmos/cosmos-sdk/version.Commit=$(COMMIT) \
-X "github.com/cosmos/cosmos-sdk/version.BuildTags=$(build_tags_comma_sep)"

ifeq (cleveldb,$(findstring cleveldb,$(STRIDE_BUILD_OPTIONS)))
ldflags += -X github.com/cosmos/cosmos-sdk/types.DBBackend=cleveldb
else ifeq (rocksdb,$(findstring rocksdb,$(STRIDE_BUILD_OPTIONS)))
ldflags += -X github.com/cosmos/cosmos-sdk/types.DBBackend=rocksdb
endif
ifeq (,$(findstring nostrip,$(STRIDE_BUILD_OPTIONS)))
ldflags += -w -s
endif
ldflags += $(LDFLAGS)
ldflags := $(strip $(ldflags))

BUILD_FLAGS := -tags "$(build_tags)" -ldflags '$(ldflags)'
# check for nostrip option
ifeq (,$(findstring nostrip,$(STRIDE_BUILD_OPTIONS)))
BUILD_FLAGS += -trimpath
endif

# The below include contains the tools target.
include tools.mk

###############################################################################
### Build ###
###############################################################################

all: install lint test

BUILD_TARGETS := build install

build: BUILD_ARGS=-o $(BUILDDIR)/

$(BUILD_TARGETS): go.sum $(BUILDDIR)/
go $@ -mod=readonly $(BUILD_FLAGS) $(BUILD_ARGS) ./...

$(BUILDDIR)/:
mkdir -p $(BUILDDIR)/

build-reproducible: go.sum
$(DOCKER) rm latest-build || true
$(DOCKER) run --volume=$(CURDIR):/sources:ro \
--env TARGET_PLATFORMS='linux/amd64 darwin/amd64 linux/arm64 windows/amd64' \
--env APP=strided \
--env VERSION=$(VERSION) \
--env COMMIT=$(COMMIT) \
--env LEDGER_ENABLED=$(LEDGER_ENABLED) \
--name latest-build cosmossdk/rbuilder:latest
$(DOCKER) cp -a latest-build:/home/builder/artifacts/ $(CURDIR)/

build-linux: go.sum
LEDGER_ENABLED=false GOOS=linux GOARCH=amd64 $(MAKE) build

build-contract-tests-hooks:
mkdir -p $(BUILDDIR)
go build -mod=readonly $(BUILD_FLAGS) -o $(BUILDDIR)/ ./cmd/contract_tests

go-mod-cache: go.sum
@echo "--> Download go modules to local cache"
@go mod download

go.sum: go.mod
@echo "--> Ensure dependencies have not been modified"
@go mod verify

draw-deps:
@# requires brew install graphviz or apt-get install graphviz
go get github.com/RobotsAndPencils/goviz
@goviz -i ./cmd/osmosisd -d 2 | dot -Tpng -o dependency-graph.png

clean:
rm -rf $(BUILDDIR)/ artifacts/

distclean: clean
rm -rf vendor/

###############################################################################
### Proto ###
###############################################################################

proto-all: proto-format proto-gen

proto:
@echo
@echo "=========== Generate Message ============"
@echo
./scripts/protocgen.sh
@echo
@echo "=========== Generate Complete ============"
@echo

test:
@go test -v ./x/...

docs:
@echo
@echo "=========== Generate Message ============"
@echo
./scripts/generate-docs.sh

statik -src=client/docs/static -dest=client/docs -f -m
@if [ -n "$(git status --porcelain)" ]; then \
echo "\033[91mSwagger docs are out of sync!!!\033[0m";\
exit 1;\
else \
echo "\033[92mSwagger docs are in sync\033[0m";\
fi
@echo
@echo "=========== Generate Complete ============"
@echo

protoVer=v0.2
protoImageName=tendermintdev/sdk-proto-gen:$(protoVer)
containerProtoGen=osmosis-proto-gen-$(protoVer)
containerProtoFmt=osmosis-proto-fmt-$(protoVer)

proto-gen:
@echo "Generating Protobuf files"
@if docker ps -a --format '{{.Names}}' | grep -Eq "^${containerProtoGen}$$"; then docker start -a $(containerProtoGen); else docker run --name $(containerProtoGen) -v $(CURDIR):/workspace --workdir /workspace $(protoImageName) \
sh ./scripts/protocgen.sh; fi

proto-format:
@echo "Formatting Protobuf files"
@if docker ps -a --format '{{.Names}}' | grep -Eq "^${containerProtoFmt}$$"; then docker start -a $(containerProtoFmt); else docker run --name $(containerProtoFmt) -v $(CURDIR):/workspace --workdir /workspace tendermintdev/docker-build-proto \
find ./ -not -path "./third_party/*" -name "*.proto" -exec clang-format -i {} \; ; fi

###############################################################################
### Devdoc ###
###############################################################################

build-docs:
@cd docs && \
while read p; do \
(git checkout $${p} && npm install && VUEPRESS_BASE="/$${p}/" npm run build) ; \
mkdir -p ~/output/$${p} ; \
cp -r .vuepress/dist/* ~/output/$${p}/ ; \
cp ~/output/$${p}/index.html ~/output ; \
done < versions ;

sync-docs:
cd ~/output && \
echo "role_arn = ${DEPLOYMENT_ROLE_ARN}" >> /root/.aws/config ; \
echo "CI job = ${CIRCLE_BUILD_URL}" >> version.html ; \
aws s3 sync . s3://${WEBSITE_BUCKET} --profile terraform --delete ; \
aws cloudfront create-invalidation --distribution-id ${CF_DISTRIBUTION_ID} --profile terraform --path "/*" ;
.PHONY: sync-docs


###############################################################################
### Tests & Simulation ###
###############################################################################

include sims.mk

test: test-unit test-build

test-all: check test-race test-cover

test-unit:
@VERSION=$(VERSION) go test -mod=readonly -tags='ledger test_ledger_mock norace' ./...

test-race:
@VERSION=$(VERSION) go test -mod=readonly -race -tags='ledger test_ledger_mock' ./...

test-cover:
@go test -mod=readonly -timeout 30m -coverprofile=coverage.txt -covermode=atomic -tags='ledger test_ledger_mock' ./...

benchmark:
@go test -mod=readonly -bench=. ./...


###############################################################################
### Linting ###
###############################################################################

lint:
golangci-lint run --disable-all -E errcheck
find . -name '*.go' -type f -not -path "./vendor*" -not -path "*.git*" | xargs gofmt -d -s

format:
find . -name '*.go' -type f -not -path "./vendor*" -not -path "*.git*" -not -path "./client/lcd/statik/statik.go" | xargs gofmt -w -s
find . -name '*.go' -type f -not -path "./vendor*" -not -path "*.git*" -not -path "./client/lcd/statik/statik.go" | xargs misspell -w
find . -name '*.go' -type f -not -path "./vendor*" -not -path "*.git*" -not -path "./client/lcd/statik/statik.go" | xargs goimports -w -local github.com/cosmos/cosmos-sdk

###############################################################################
### Localnet ###
###############################################################################

build-docker-osmosisdnode:
$(MAKE) -C contrib/localtestnet

# Run a 4-node testnet locally
localnet-start: build-linux build-docker-osmosisdnode # localnet-stop
@if ! [ -f $(BUILDDIR)/node0/osmosisd/config/genesis.json ]; \
then docker run --rm -v $(BUILDDIR):/osmosisd:Z osmosis-labs/osmosisdnode testnet --v 4 -o . --starting-ip-address 192.168.10.2 --keyring-backend=test ; \
fi
docker-compose up -d

# Stop testnet
localnet-stop:
docker-compose down

clean-localnet: localnet-stop
rm -rf $(BUILDDIR)/node* $(BUILDDIR)/gentxs

test-docker:
@docker build -f contrib/Dockerfile.test -t ${TEST_DOCKER_REPO}:$(shell git rev-parse --short HEAD) .
@docker tag ${TEST_DOCKER_REPO}:$(shell git rev-parse --short HEAD) ${TEST_DOCKER_REPO}:$(shell git rev-parse --abbrev-ref HEAD | sed 's#/#_#g')
@docker tag ${TEST_DOCKER_REPO}:$(shell git rev-parse --short HEAD) ${TEST_DOCKER_REPO}:latest

test-docker-push: test-docker
@docker push ${TEST_DOCKER_REPO}:$(shell git rev-parse --short HEAD)
@docker push ${TEST_DOCKER_REPO}:$(shell git rev-parse --abbrev-ref HEAD | sed 's#/#_#g')
@docker push ${TEST_DOCKER_REPO}:latest

.PHONY: all build-linux install format lint \
go-mod-cache draw-deps clean build \
setup-transactions setup-contract-tests-data start-osmosis run-lcd-contract-tests contract-tests \
test test-all test-build test-cover test-unit test-race \
benchmark \
build-docker-osmosisdnode localnet-start localnet-stop \
docker-single-node

test:
Copy link
Contributor

@riley-stride riley-stride May 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The makefile now has three "test" targets.

  • The first and second define recipes which run all the tests in the packages within the ./x/ folder and unit tests (many of which fail, as expected, since we haven't written these yet; I'm holding these tests are placeholders for now)
    • The second runs unit tests too through test-unit dependency. This seems to run the same tests as the first recipe, but it breaks on error. I'm commenting this out for now to get things running since we wouldn't expect these unit tests to work, so we would not want to fail on them until we've written them properly. Added TODO(TEST-49) to write the unit tests properly.
    • It also tries to run a test-build dependency which doesn't exist (?)
  • The third runs script/simple_test.sh, which works after I made a tweak to reset blockchain state (added the --reset-once flag

Is your intention to run the tests in each folder of the project or to run simple_test.sh? @asalzmann

./scripts/simple_test.sh
Binary file added stride/build/strided
Binary file not shown.
53 changes: 53 additions & 0 deletions stride/scripts/multinode-local-testnet.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Multi Node Local Testnet Script

This script creates a multi node local testnet with three validator nodes on a single machine. Note: The default weights of these validators is 5:5:4 respectively. That means in order to keep the chain running, at a minimum Validator1 and Validator2 must be running in order to keep greater than 66% power online.

## Instructions

Clone the stride repo

Checkout the branch you are looking to test

Make install / reload profile

Give the script permission with `chmod +x multinode-local-testnet.sh`

Run with `./multinode-local-testnet.sh` (allow ~45 seconds to run, required sleep commands due to multiple transactions)

## Logs

Validator1: `tmux a -t validator1`

Validator2: `tmux a -t validator2`

Validator3: `tmux a -t validator3`

## Directories

Validator1: `$HOME/.strided/validator1`

Validator2: `$HOME/.strided/validator2`

Validator3: `$HOME/.strided/validator3`

## Ports

Validator1: `1317, 9090, 9091, 26658, 26657, 26656, 6060`

Validator2: `1316, 9088, 9089, 26655, 26654, 26653, 6061`

Validator3: `1315, 9086, 9087, 26652, 26651, 26650, 6062`

Ensure to include the `--home` flag or `--node` flag when using a particular node.

## Examples

Validator2: `strided status --node "tcp://localhost:26654"`

Validator3: `strided status --node "tcp://localhost:26651"`

or

Validator1: `strided keys list --keyring-backend test --home $HOME/.strided/validator1`

Validator2: `strided keys list --keyring-backend test --home $HOME/.strided/validator2`
Loading