Sg integrationtest chainprotocol (#359)

* update integration test readme

* readme feedback incorporated

* remove Log.Fatal and improve logging on fail

* revert to Mumbai. change to Nairobi was a mistake - tests with chainID in them break there

* sg-integration-test adding chain protocol via env var

* sg-integrationtest fix workflow env var

* sg-integrationtest fix workflow path to env file

* sg-integrationtest more fixing of paths

* sg-integrationtest fix test step

* sg-integrationtest simplify workflow's env var

* sg-integrationtest remove use of --env-var option and update readme

* sg-integrationtest small readme update

* sg-integrationtest update readme with re-running tests instruction

.github/workflows/build.yaml

@@ -107,10 +107,9 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        octez-versions:
-          - v16.0-rc3
-    env:
-      IMAGE: ghcr.io/ecadlabs/signatory:${{ github.head_ref || github.ref_name }}-amd64
+        testenvs:
+          - env.current.amd64
+          - env.next.amd64
     steps:
       - uses: actions/checkout@v2
       - name: Login to DockerHub
@@ -120,16 +119,11 @@ jobs:
           registry: ghcr.io
           username: ${{ github.actor }}
           password: ${{ secrets.GITHUB_TOKEN }}
-      - name: Pull images
-        run: >
-         docker pull oxheadalpha/flextesa:latest;
-         docker pull tezos/tezos:${{ matrix.octez-versions }};
-         docker pull $IMAGE;        
-      - name: Start containers
-        run: IMAGE=$IMAGE OCTEZ_VERSION=${{ matrix.octez-versions }} docker compose -f integration_test/docker-compose.yml up -d --wait
-      - name: Modify octez
-        run: docker exec octez sudo chown -R tezos /home/tezos/.tezos-client
       - name: Run tests
-        run: IMAGE=$IMAGE OCTEZ_VERSION=${{ matrix.octez-versions }} go test $(go list ./... | grep integration_test)
-      - name: Kill containers
-        run: IMAGE=$IMAGE OCTEZ_VERSION=${{ matrix.octez-versions }} docker compose -f integration_test/docker-compose.yml kill
+        run: >
+         export $(xargs <integration_test/${{ matrix.testenvs }}); 
+         export IMAGE=ghcr.io/ecadlabs/signatory:${{ github.head_ref || github.ref_name }}-amd64;        
+         docker compose -f integration_test/docker-compose.yml up -d --wait --pull always;
+         docker exec octez sudo chown -R tezos /home/tezos/.tezos-client;
+         go test $(go list ./... | grep integration_test);
+         docker compose -f integration_test/docker-compose.yml kill;

integration_test/README.md

@@ -6,15 +6,19 @@ The version of Signatory that is run is defined by an environment variable named
 
 The `octez-client` that is run by the tests is provided by the `tezos` container, not the `octez-client` that is onboard the `flextesa` image, so that official `tezos` image releases can be used.  The version of `tezos` container is defined by an environment variable named `OCTEZ_VERSION`.
 
-Currently, it is always the `latest` version of the `flextesa` image that is run by the tests, which protocol the testnet runs is defined in the script `flextesa.sh`
+Currently, it is always the `latest` version of the `flextesa` image that is run by the tests.  The economic protocol run by flextesa is defined by an environment variable named `PROTOCOL`
 
 ## Pulling the images
 
 Pre-release Signatory images are available in [github container registry](https://github.com/ecadlabs/signatory/pkgs/container/signatory)
 Official image releases are available in [dockerhub](https://hub.docker.com/r/ecadlabs/signatory/tags)
-If you get a 404 from the github container registry, you can request access from an admin.
+If you get a 404 from the github container registry web console, you can request access from an admin.
 
-### Github container registry authentication setup
+[flextesa](https://hub.docker.com/r/oxheadalpha/flextesa/tags) image is used.
+
+[tezos](https://hub.docker.com/r/tezos/tezos/tags) image is used
+
+## Github container registry authentication setup
 
 If this is your first time pulling an image from github packages, then you'll need to configure a [Personal Access Token PAT (classic)](https://github.com/settings/tokens). The only access you should grant the PAT is `read:packages`.  With that token as the value of env var `$PAT`, you can now login:
 
@@ -24,43 +28,54 @@ echo $PAT |docker login ghcr.io -u <your_github_name> --password-stdin
 
 ## Running the tests
 
-Pull the images for the version and architecture that suit your needs from [flextesa](https://hub.docker.com/r/oxheadalpha/flextesa/tags), [tezos](https://hub.docker.com/r/tezos/tezos/tags), [signatory (pre-release)](https://github.com/ecadlabs/signatory/pkgs/container/signatory/versions), or [signatory (release)](https://hub.docker.com/r/ecadlabs/signatory/tags)
+```sh
+cd integration_test
+```
+
+Exporting the Environment Variables used by the test is required. Choose the set of env var to use from the files `env.current.arm64`, `env.next.arm64`, `env.current.amd64`, `env.next.amd64`.  Use `current` if you'd like the economic protocol run by flextesa to match mainnet, use `next` if you'd like the next protocol instead.  Use `arm64` or `amd64` depending on your host architecture. 
 
-Example:
+So, to set the env to use mainnet protocol, using a build of Signatory's `main` branch, on a macbook m1 host:
 
 ```sh
-export SIGY_IMAGE=ghcr.io/ecadlabs/signatory:main-arm64
-export OCTEZ_VERSION=arm64_v17.0-beta1
-docker pull oxheadalpha/flextesa:latest
-docker pull tezos/tezos:$OCTEZ_VERSION
-docker pull $SIGY_IMAGE
+export $(xargs <env.current.arm64)
+export IMAGE=ghcr.io/ecadlabs/signatory:main-arm64
+```
+
+Likewise, to set the env to use the next protocol, using a build of Signatory's `main` branch, on an x86_64 host:
+
+```sh
+export $(xargs <env.next.amd64)
+export IMAGE=ghcr.io/ecadlabs/signatory:main-amd64
 ```
 
 Next, start the stack:
 
 ```sh
-cd integration_test
-IMAGE=$SIGY_IMAGE OCTEZ_VERSION=$OCTEZ_VERSION docker compose up -d --wait
+docker compose up -d --wait --pull always
 ```
 
 Run all the tests:
 
 ```sh
-IMAGE=$SIGY_IMAGE OCTEZ_VERSION=$OCTEZ_VERSION go test ./...
+go clean -testcache && go test ./...
 ```
 
 Or, just run a single test:
 
 ```sh
-IMAGE=$SIGY_IMAGE OCTEZ_VERSION=$OCTEZ_VERSION go test -run ^TestOperationAllowPolicy
+go clean -testcache && go test -run ^TestOperationAllowPolicy
 ```
 
 Stop the stack when you are done:
 
 ```sh
-IMAGE=$SIGY_IMAGE OCTEZ_VERSION=$OCTEZ_VERSION docker compose down
+docker compose down
 ```
 
+## Re-Running Tests
+
+Most tests can be re-run successfully as detailed above.  Some tests (like the `reveal` operation) can only be run once on a chain.  So, when re-running all, stop the stack and bring it up again in between test runs. 
+
 ## Notes to the operator
 
 Some tests in this folder make edits to `signatory.yaml` configuration and restart the Signatory service. By design, tests that do this shall clean up after themselves by restoring the copy of the file that is in the code repository.  If `git status` after a test run shows you have modifications to the `signatory.yaml` file, then that would mean a test is failing to clean up after itself and should be corrected.

integration_test/docker-compose.yml

@@ -15,7 +15,7 @@ services:
       - ./flextesa.sh:/usr/bin/sigybox
     environment:
       - block_time=3
-    command: sigybox
+    command: sigybox $PROTOCOL
     healthcheck:
       test: "curl --fail http://localhost:20000/chains/main/blocks/head/header || exit 1"
       interval: 1s

integration_test/env.current.amd64

@@ -0,0 +1,2 @@
+OCTEZ_VERSION=amd64_v16.0-rc3
+PROTOCOL=Mumbai

integration_test/env.current.arm64

@@ -0,0 +1,2 @@
+OCTEZ_VERSION=arm64_v16.0-rc3
+PROTOCOL=Mumbai

integration_test/env.next.amd64

@@ -0,0 +1,2 @@
+OCTEZ_VERSION=amd64_v17.0-beta1
+PROTOCOL=Nairobi

integration_test/env.next.arm64

@@ -0,0 +1,2 @@
+OCTEZ_VERSION=arm64_v17.0-beta1
+PROTOCOL=Nairobi

integration_test/flextesa.sh

@@ -1,6 +1,6 @@
 #! /bin/sh
 
-protocol=Mumbai
+protocol=$1
 time_bb=${block_time:-1}
 
 
@@ -13,7 +13,7 @@ export user1="user1,edpkvNSVE2pL4eaCYrew1NEuLi4nnYmwQfe4tdM4NoVgNMEoVCNGoW,tz1Qg
 root_path=/tmp/mini-box
 
 flextesa mini-net \
-         --root "$root_path" --size 1 "$@" \
+         --root "$root_path" --size 1 \
          --set-history-mode N000:archive \
          --number-of-b 1 \
          --balance-of-bootstrap-accounts tez:100_000_000 \