Skip to content

Commit

Permalink
chore(ci): fix scheduled system tests (#7053)
Browse files Browse the repository at this point in the history
System tests are scheduled, but the recent check that compute if the run
is needed is not working in that use case.

Also simplifying a little bit the yaml file.

## Reviewer Checklist

- [x] Title is accurate.
- [x] No unnecessary changes are introduced.
- [x] Description motivates each change.
- [x] Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes unless absolutely necessary.
- [x] Testing strategy adequately addresses listed risk(s).
- [x] Change is maintainable (easy to change, telemetry, documentation).
- [x] Release note makes sense to a user of the library.
- [x] Reviewer has explicitly acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment.
- [x] Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)
- [x] If this PR touches code that signs or publishes builds or
packages, or handles credentials of any kind, I've requested a review
from `@DataDog/security-design-and-guidance`.
- [x] This PR doesn't touch any of that.
  • Loading branch information
cbeauchesne authored Sep 26, 2023
1 parent 065784b commit b96f391
Showing 1 changed file with 2 additions and 30 deletions.
32 changes: 2 additions & 30 deletions .github/workflows/system-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ jobs:
system-tests:
runs-on: ubuntu-latest
needs: needs-run
if: needs.needs-run.outputs.outcome == 'success' || github.event_name == 'schedule'
strategy:
matrix:
include:
Expand All @@ -46,146 +47,117 @@ jobs:
CMAKE_BUILD_PARALLEL_LEVEL: 12
steps:
- name: Setup python 3.9
if: needs.needs-run.outputs.outcome == 'success'
uses: actions/setup-python@v4
with:
python-version: '3.9'

- name: Checkout system tests
if: needs.needs-run.outputs.outcome == 'success'
uses: actions/checkout@v3
with:
repository: 'DataDog/system-tests'

- name: Checkout dd-trace-py
if: needs.needs-run.outputs.outcome == 'success'
uses: actions/checkout@v3
with:
path: 'binaries/dd-trace-py'
fetch-depth: 0

- name: Build
if: needs.needs-run.outputs.outcome == 'success'
run: ./build.sh

- name: Run
if: needs.needs-run.outputs.outcome == 'success'
run: ./run.sh

- name: Run REMOTE_CONFIG_MOCKED_BACKEND_ASM_FEATURES
if: needs.needs-run.outputs.outcome == 'success'
run: ./run.sh REMOTE_CONFIG_MOCKED_BACKEND_ASM_FEATURES

- name: Run REMOTE_CONFIG_MOCKED_BACKEND_LIVE_DEBUGGING
if: needs.needs-run.outputs.outcome == 'success'
run: ./run.sh REMOTE_CONFIG_MOCKED_BACKEND_LIVE_DEBUGGING

- name: Run REMOTE_CONFIG_MOCKED_BACKEND_ASM_DD
if: needs.needs-run.outputs.outcome == 'success'
run: ./run.sh REMOTE_CONFIG_MOCKED_BACKEND_ASM_DD

- name: Run APPSEC_MISSING_RULES
if: needs.needs-run.outputs.outcome == 'success'
run: ./run.sh APPSEC_MISSING_RULES

- name: Run APPSEC_CUSTOM_RULES
if: needs.needs-run.outputs.outcome == 'success'
run: ./run.sh APPSEC_CUSTOM_RULES

- name: Run APPSEC_CORRUPTED_RULES
if: needs.needs-run.outputs.outcome == 'success'
run: ./run.sh APPSEC_CORRUPTED_RULES

- name: Run APPSEC_RULES_MONITORING_WITH_ERRORS
if: needs.needs-run.outputs.outcome == 'success'
run: ./run.sh APPSEC_RULES_MONITORING_WITH_ERRORS

- name: Run APPSEC_BLOCKING
if: needs.needs-run.outputs.outcome == 'success'
run: ./run.sh APPSEC_BLOCKING

- name: Run APPSEC_DISABLED
if: needs.needs-run.outputs.outcome == 'success'
run: ./run.sh APPSEC_DISABLED

- name: Run APPSEC_LOW_WAF_TIMEOUT
if: needs.needs-run.outputs.outcome == 'success'
run: ./run.sh APPSEC_LOW_WAF_TIMEOUT

- name: Run APPSEC_CUSTOM_OBFUSCATION
if: needs.needs-run.outputs.outcome == 'success'
run: ./run.sh APPSEC_CUSTOM_OBFUSCATION

- name: Run APPSEC_RATE_LIMITER
if: needs.needs-run.outputs.outcome == 'success'
run: ./run.sh APPSEC_RATE_LIMITER

- name: Run APPSEC_BLOCKING_FULL_DENYLIST
if: needs.needs-run.outputs.outcome == 'success'
run: ./run.sh APPSEC_BLOCKING_FULL_DENYLIST

- name: Run APPSEC_REQUEST_BLOCKING
if: needs.needs-run.outputs.outcome == 'success'
run: ./run.sh APPSEC_REQUEST_BLOCKING

- name: Run APPSEC_RUNTIME_ACTIVATION
if: needs.needs-run.outputs.outcome == 'success'
run: ./run.sh APPSEC_RUNTIME_ACTIVATION

- name: Run APPSEC_WAF_TELEMETRY
if: needs.needs-run.outputs.outcome == 'success'
run: ./run.sh APPSEC_WAF_TELEMETRY

- name: Run SAMPLING
if: needs.needs-run.outputs.outcome == 'success'
run: ./run.sh SAMPLING

# even on failures, we want to have artifact to be able to investigate
# The compress step speed up a lot the upload artifact process
- name: Compress artifact
if: needs.needs-run.outputs.outcome == 'success'
run: tar -czvf artifact.tar.gz $(ls | grep logs)

- name: Upload artifact
uses: actions/upload-artifact@v3
if: needs.needs-run.outputs.outcome == 'success'
with:
name: logs_${{ matrix.weblog-variant }}
path: artifact.tar.gz

parametric:
runs-on: ubuntu-latest
needs: needs-run
if: needs.needs-run.outputs.outcome == 'success' || github.event_name == 'schedule'
env:
TEST_LIBRARY: python
PYTHON_DDTRACE_PACKAGE: git+https://github.com/Datadog/dd-trace-py.git@${{ github.sha }}
steps:
- name: Checkout system tests
if: needs.needs-run.outputs.outcome == 'success'
uses: actions/checkout@v3
with:
repository: 'DataDog/system-tests'

- uses: actions/setup-python@v4
if: needs.needs-run.outputs.outcome == 'success'
with:
python-version: '3.9'

- name: Build
if: needs.needs-run.outputs.outcome == 'success'
run: ./build.sh -i runner

- name: Run
if: needs.needs-run.outputs.outcome == 'success'
run: ./run.sh PARAMETRIC

- name: Compress artifact
if: needs.needs-run.outputs.outcome == 'success'
run: tar -czvf artifact.tar.gz $(ls | grep logs)

- name: Upload artifact
uses: actions/upload-artifact@v3
if: needs.needs-run.outputs.outcome == 'success'
with:
name: logs_parametric
path: artifact.tar.gz

0 comments on commit b96f391

Please sign in to comment.