Skip to content

Commit

Permalink
Merge branch 'develop' into chore/l2ep-codeowners
Browse files Browse the repository at this point in the history
  • Loading branch information
chris-de-leon-cll authored Sep 24, 2024
2 parents 47a75fb + 8854def commit f6de05c
Show file tree
Hide file tree
Showing 64 changed files with 1,666 additions and 1,289 deletions.
5 changes: 5 additions & 0 deletions .changeset/gorgeous-walls-sit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"chainlink": patch
---

#added Soneium testnet chain configs
5 changes: 5 additions & 0 deletions .changeset/khaki-tigers-agree.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"chainlink": patch
---

#added #nops Add Zircuit Configs
5 changes: 5 additions & 0 deletions .changeset/olive-snails-sniff.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"chainlink": patch
---

Adding USDCReaderTester contract for CCIP integration tests #internal
85 changes: 2 additions & 83 deletions .github/E2E_TESTS_ON_GITHUB_CI.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,87 +53,6 @@ For security reasons, test secrets and sensitive information are not stored dire

If you need to run a GitHub workflow using custom secrets, please refer to the [guide on running GitHub workflows with your test secrets](https://github.com/smartcontractkit/chainlink-testing-framework/blob/main/lib/config/README.md#run-github-workflow-with-your-test-secrets).

## About the Reusable Workflow
## About the E2E Test Reusable Workflow

The [E2E Tests Reusable Workflow](https://github.com/smartcontractkit/chainlink/blob/develop/.github/workflows/run-e2e-tests-reusable-workflow.yml) is designed to run any type of E2E test on GitHub CI, including docker/testcontainers, old K8s tests, or tests in CRIB in the future.

Our goal is to migrate all workflows to use this reusable workflow for executing E2E tests. This approach will streamline our CI and allow for the automatic execution of tests at different stages of the software development process. Learn more about the advantages of using reusable workflows [here](https://smartcontract-it.atlassian.net/wiki/spaces/TT/pages/815497220/CI+Workflows+for+E2E+Tests).

**Examples of Workflows Utilizing the Reusable Workflow:**

- [Integration Tests](https://github.com/smartcontractkit/chainlink/blob/develop/.github/workflows/integration-tests.yml)
- [Nightly E2E Tests](https://github.com/smartcontractkit/chainlink/blob/develop/.github/workflows/run-nightly-e2e-tests.yml)
- [Selected E2E Tests](https://github.com/smartcontractkit/chainlink/blob/develop/.github/workflows/run-selected-e2e-tests.yml)
- [On-Demand Automation Tests](https://github.com/smartcontractkit/chainlink/blob/develop/.github/workflows/automation-ondemand-tests.yml)
- [CCIP Chaos Tests](https://github.com/smartcontractkit/chainlink/blob/develop/.github/workflows/ccip-chaos-tests.yml)

### E2E Test Configuration on GitHub CI

The [e2e-tests.yml](https://github.com/smartcontractkit/chainlink/blob/develop/.github/e2e-tests.yml) file lists all E2E tests configured for execution on CI. Each entry specifies the type of GitHub Runner needed and the workflows in which the test is included (for example, `smoke/ocr_test.go:*` is executed both in PRs and nightly).

### Slack Notification After Tests

To configure Slack notifications after tests executed via the reusable workflow, follow these steps:

- Set `slack_notification_after_tests` to either `always` or `on_failure` depending on when you want notifications to be sent.
- Assign `slack_notification_after_tests_channel_id` to the ID of the Slack channel where notifications should be sent.
- Provide a title for the notification by setting `slack_notification_after_tests_name`.
- Optionally use `slack_notification_after_tests_notify_user_id_on_failure` to reply in the thread and notify a user about the failed workflow

**Example:**

```yml
jobs:
call-run-e2e-tests-workflow:
name: Run E2E Tests
uses: ./.github/workflows/run-e2e-tests-reusable-workflow.yml
with:
chainlink_version: develop
test_workflow: Nightly E2E Tests
slack_notification_after_tests: true
slack_notification_after_tests_channel_id: "#team-test-tooling-internal"
slack_notification_after_tests_name: Nightly E2E Tests
slack_notification_after_tests_notify_user_id_on_failure: U0XXXXXXX
```
## Guides
### How to Run Selected Tests on GitHub CI
The [Run Selected E2E Tests Workflow](https://github.com/smartcontractkit/chainlink/actions/workflows/run-selected-e2e-tests.yml) allows you to execute specific E2E tests as defined in the [e2e-tests.yml](https://github.com/smartcontractkit/chainlink/blob/develop/.github/e2e-tests.yml). This is useful for various purposes such as testing specific features on a particular branch or verifying that modifications to a test have not introduced any issues.
For details on all available inputs that the workflow supports, refer to the [workflow definition](https://github.com/smartcontractkit/chainlink/actions/workflows/run-selected-e2e-tests.yml).
**Example Usage:**
To run a set of VRFv2Plus tests from a custom branch, use the following command:
```bash
gh workflow run run-selected-e2e-tests.yml \
-f test_ids="TestVRFv2Plus_LinkBilling,TestVRFv2Plus_NativeBilling,TestVRFv2Plus_DirectFunding,TestVRFV2PlusWithBHS" \
-f chainlink_version=develop \
--ref TT-1550-Provide-PoC-for-keeping-test-configs-in-git
```

### How to Run Custom Tests with Reusable Workflow

To run a specific list of tests, utilize the `custom_test_list_json` input. This allows you to provide a customized list of tests. If your test list is dynamic, you can generate it during a preceding job and then reference it using: `custom_test_list_json: ${{ needs.gen_test_list.outputs.test_list }}`.

```yml
run-e2e-tests-workflow:
name: Run E2E Tests
uses: ./.github/workflows/run-e2e-tests-reusable-workflow.yml
with:
custom_test_list_json: >
{
"tests": [
{
"id": "TestVRFv2Plus",
"path": "integration-tests/smoke/vrfv2plus_test.go",
"runs_on": "ubuntu-latest",
"test_env_type": "docker",
"test_cmd": "cd integration-tests/smoke && go test vrfv2plus_test.go"
}
]
}
```
For information on the E2E Test Reusable Workflow, visit the documentation in the [smartcontractkit/.github repository](https://github.com/smartcontractkit/.github/blob/main/.github/workflows/README.md).
Loading

0 comments on commit f6de05c

Please sign in to comment.