Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Docs/Contributing] Clarification on running non-OpenAI tests #1052

Merged
merged 3 commits into from
Dec 28, 2023
Merged

[Docs/Contributing] Clarification on running non-OpenAI tests #1052

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0a633eb
Update Contribute.md
maxim-saplin Dec 24, 2023
3629770
Removed mentioning of OAI_CONFIG_LIST
maxim-saplin Dec 25, 2023
29ebaae
Merge branch 'main' into contrib-run-tests-non-oai
qingyun-wu Dec 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion website/docs/Contribute.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,23 @@ Tests are automatically run via GitHub actions. There are two workflows:
1. [build.yml](https://github.com/microsoft/autogen/blob/main/.github/workflows/build.yml)
1. [openai.yml](https://github.com/microsoft/autogen/blob/main/.github/workflows/openai.yml)
ekzhu marked this conversation as resolved.
Show resolved Hide resolved

The first workflow is required to pass for all PRs. The second workflow is required for changes that affect the openai tests. The second workflow requires approval to run. When writing tests that require openai, please use [`pytest.mark.skipif`](https://github.com/microsoft/autogen/blob/main/test/test_client.py#L13) to make them run in one python version only when openai is installed. If additional dependency for this test is required, install the dependency in the corresponding python version in [openai.yml](https://github.com/microsoft/autogen/blob/main/.github/workflows/openai.yml).
The first workflow is required to pass for all PRs (and it doesn't do any OpenAI calls). The second workflow is required for changes that affect the OpenAI tests (and does actually call LLM). The second workflow requires approval to run. When writing tests that require OpenAI calls, please use [`pytest.mark.skipif`](https://github.com/microsoft/autogen/blob/b1adac515931bf236ac59224269eeec683a162ba/test/oai/test_client.py#L19) to make them run in one python version only when `openai` package is installed. If additional dependency for this test is required, install the dependency in the corresponding python version in [openai.yml](https://github.com/microsoft/autogen/blob/main/.github/workflows/openai.yml).

#### Run non-OpenAI tests

To run the subset of the tests not depending on `openai` (and not calling LLMs)):
- Create an empty file `OAI_CONFIG_LIST`, add `[]` (make it a valid JSON with empty collection)
sonichi marked this conversation as resolved.
Show resolved Hide resolved
- Install `pytest`
- Remove `openai` library, this is required to skip the tests which check for `openai` presence ([`pytest.mark.skipif`](https://github.com/microsoft/autogen/blob/b1adac515931bf236ac59224269eeec683a162ba/test/oai/test_client.py#L19))
- Run the tests from the `test` folder. Make sure they all pass, this is required for [build.yml](https://github.com/microsoft/autogen/blob/main/.github/workflows/build.yml) checks to pass
- Reinstall `openai`
ekzhu marked this conversation as resolved.
Show resolved Hide resolved

```
pip install pytest
pip uninstall -y openai
pytest test
pip install openai
```

### Coverage

Expand Down
Loading