Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Daily -> GHA #6386

Merged
merged 16 commits into from
Jul 25, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .azure-pipelines-templates/deploy_aci.yml
Original file line number Diff line number Diff line change
@@ -98,7 +98,7 @@ jobs:
set -ex
cd /CCF/build
npm config set cache /ccfci/workspace_$(Build.BuildNumber)/.npm
WORKSPACE=/ccfci/workspace_$(Build.BuildNumber) ELECTION_TIMEOUT_MS=10000 ./tests.sh -VV -T Test -LE "benchmark|perf|tlstest|vegeta|suite|snp_flaky" -E "lts_compatibility"
WORKSPACE=/ccfci/workspace_$(Build.BuildNumber) ELECTION_TIMEOUT_MS=10000 ./tests.sh -VV -T Test -LE "benchmark|perf|tlstest|suite|snp_flaky" -E "lts_compatibility"
# Remove irrelevant and bulky data from workspace before uploading
find /ccfci/workspace_$(Build.BuildNumber) -type f -name cchost -delete
find /ccfci/workspace_$(Build.BuildNumber) -type f -name "*.so" -delete
13 changes: 0 additions & 13 deletions .azure-pipelines-templates/stress-matrix.yml

This file was deleted.

2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -90,7 +90,7 @@ jobs:
# Unit tests
./tests.sh --output-on-failure -L unit -j$(nproc --all)
# All other acceptably fast tests, mostly end-to-end
./tests.sh --timeout 360 --output-on-failure -LE "benchmark|perf|protocolstest|vegeta|suite|unit"
./tests.sh --timeout 360 --output-on-failure -LE "benchmark|perf|protocolstest|suite|unit"
# Partitions tests
./tests.sh --timeout 240 --output-on-failure -L partitions -C partitions
shell: bash
102 changes: 102 additions & 0 deletions .github/workflows/long-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
name: Long Test

on:
schedule:
- cron: "0 0 * * 1-5"
workflow_dispatch:

jobs:
scan_build:
name: "Scan build"
runs-on: [self-hosted, 1ES.Pool=gha-virtual-ccf-sub]
container:
image: ghcr.io/microsoft/ccf/ci/default:build-26-06-2024

steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: "Run scan"
run: |
set -x
mkdir build
cd build
../scripts/scan-build.sh

long-asan:
name: ASAN
runs-on: [self-hosted, 1ES.Pool=gha-virtual-ccf-sub]
container:
image: ghcr.io/microsoft/ccf/ci/default:build-26-06-2024

steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: "Install deps"
run: |
sudo apt-get -y update
sudo apt install ansible -y
cd getting_started/setup_vm
ansible-playbook ccf-extended-testing.yml

- name: "Build"
run: |
git config --global --add safe.directory "$GITHUB_WORKSPACE"
mkdir build
cd build
cmake -GNinja -DCOMPILE_TARGET=virtual -DCMAKE_BUILD_TYPE=Debug -DLONG_TESTS=ON -DLVI_MITIGATIONS=OFF -DSAN=ON ..
ninja

- name: "Test"
run: |
set +x
cd build
./tests.sh --output-on-failure --timeout 1600 -LE "benchmark|perf"

- name: "Upload logs"
if: success() || failure()
uses: actions/upload-artifact@v4
with:
name: logs-asan
path: |
build/workspace/*/*.config.json
build/workspace/*/out
build/workspace/*/err
if-no-files-found: ignore

long-tsan:
name: TSAN
runs-on: [self-hosted, 1ES.Pool=gha-virtual-ccf-sub]
container:
image: ghcr.io/microsoft/ccf/ci/default:build-26-06-2024

steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: "Build"
run: |
git config --global --add safe.directory /__w/CCF/CCF
mkdir build
cd build
cmake -GNinja -DCOMPILE_TARGET=virtual -DCMAKE_BUILD_TYPE=Debug -DLONG_TESTS=ON -DLVI_MITIGATIONS=OFF -DTSAN=ON -DWORKER_THREADS=2 ..
ninja

- name: "Test"
run: |
set +x
cd build
./tests.sh --output-on-failure --timeout 1600 -LE "benchmark|perf"

- name: "Upload logs"
if: success() || failure()
uses: actions/upload-artifact@v4
with:
name: logs-tsan
path: |
build/workspace/*/*.config.json
build/workspace/*/out
build/workspace/*/err
if-no-files-found: ignore
33 changes: 0 additions & 33 deletions .stress.yml

This file was deleted.

15 changes: 0 additions & 15 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1509,21 +1509,6 @@ if(BUILD_TESTS)
PYTHON_SCRIPT ${CMAKE_SOURCE_DIR}/tests/acme_endorsement.py
LABEL ACME
)

add_e2e_test(
NAME vegeta_stress
PYTHON_SCRIPT ${CMAKE_SOURCE_DIR}/tests/vegeta_stress.py
LABEL vegeta
ADDITIONAL_ARGS -p "samples/apps/logging/liblogging"
)

add_e2e_test(
NAME vegeta_long_stress
PYTHON_SCRIPT ${CMAKE_SOURCE_DIR}/tests/vegeta_stress.py
LABEL vegeta
CONFIGURATIONS long_stress
ADDITIONAL_ARGS -p "samples/apps/logging/liblogging" --duration 45m
)
endif()

add_piccolo_test(
34 changes: 0 additions & 34 deletions doc/build_apps/run_app.rst
Original file line number Diff line number Diff line change
@@ -176,37 +176,3 @@ Integration Tests
The ``sandbox.sh`` script can be a helpful element of infrastructure to execute Integration Tests against a CCF test network running a particular application (see :ccf_repo:`test_install.sh </tests/test_install.sh>` script as example).

``test_install.sh`` illustrates how to wait for the sandbox to be :ccf_repo:`ready </tests/test_install.sh#L48>` before issuing application transactions, how to shut it down cleanly, and how to trigger a recovery. Recovering a test network can be a useful way to inspect post-test application test.

Performance Tests
-----------------

``sandbox.sh`` can be equally useful for performance testing, for example with a load testing tool such as `vegeta <https://github.com/tsenart/vegeta>`_:

.. code-block:: bash

$ /opt/ccf_virtual/bin/sandbox.sh --package ./liblogging.virtual.so
...
[16:14:10.011] Node [0] = https://127.0.0.1:8000
...
[16:14:10.011] Keys and certificates have been copied to the common folder: /data/src/CCF/build/workspace/sandbox_common
...

.. code-block:: bash

# Extracted from the output of sandbox.sh, above.
$ export SCDIR=/data/src/CCF/build/workspace/sandbox_common
$ export VEGETA=/opt/vegeta/vegeta
$ $VEGETA attack --targets sample_targets.json
--format json --duration 10s \
--cert $SCDIR/user0_cert.pem \
--key $SCDIR/user0_privk.pem \
--root-certs $SCDIR/service_cert.pem | /opt/vegeta/vegeta report

Where ``sample_targets.json`` is a file containing some sample requests to be sent as load testing, for example:

.. code-block:: json

{"method": "POST", "url": "https://127.0.0.1:8000/app/log/private", "header": {"Content-Type": ["application/json"]}, "body": "eyJpZCI6IDAsICJtc2ciOiAiUHJpdmF0ZSBtZXNzYWdlOiAwIn0="}
{"method": "GET", "url": "https://127.0.0.1:8000/app/log/private?id=0", "header": {"Content-Type": ["application/json"]}}

.. _curl: https://curl.haxx.se/
3 changes: 0 additions & 3 deletions getting_started/setup_vm/ccf-extended-testing.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,5 @@
- hosts: localhost
tasks:
- import_role:
name: vegeta
tasks_from: install.yml
- import_role:
name: pebble
tasks_from: install.yml
20 changes: 0 additions & 20 deletions getting_started/setup_vm/roles/vegeta/tasks/install.yml

This file was deleted.

3 changes: 0 additions & 3 deletions getting_started/setup_vm/roles/vegeta/vars/common.yml

This file was deleted.

123 changes: 0 additions & 123 deletions tests/generate_vegeta_targets.py

This file was deleted.

Loading
Loading