The official Python adapter for Buildkite Test Analytics which collects information about your tests.
Supported python versions: >=3.8
⚒ Supported test frameworks: pytest.
📦 Supported CI systems: Buildkite, GitHub Actions, CircleCI, and others via the BUILDKITE_ANALYTICS_*
environment variables.
-
Create a test suite, and copy the API token that it gives you.
-
Add
buildkite-test-collector
to your list of dev dependencies insetup.py
extras_require={
"dev": [
"buildkite-test-collector"
]
}
- Set up your API token
Add the BUIDLKITE_ANALYTICS_TOKEN
environment variable to your build system's environment.
- Run your tests
Run your tests like normal. Note that we attempt to detect the presence of several common CI environments, however if this fails you can set the CI
environment variable to any value and it will work.
$ pytest
- Verify that it works
If all is well, you should see the test run in the test analytics section of the Buildkite dashboard.
The Buildkite Test Analytics REST API has support for tracing potentially slow operations within your tests, and can collect span data of four types: http, sql, sleep and annotations. This is documented as part of our public JSON API so anyone can instrument any code to send this data.
This library supports the ability to transmit tracing information to your Test Analytics output by using the new spans
pytest fixture. See the SpanCollector
documentation for more information.
You may also need to manually capture the data you wish to trace for your use case. For examples of how we've done this in our Ruby test collector, see:
- Instrumenting Rails to capture sql data
- Monkey patching to various libraries to capture http requests
- Monkey patching for sleep
Note: the Ruby test collector is the only Test Analytics collector that automatically captures and transmits span data. This Python collector can transmit information, but data capture must be done manually at this time.
See the GitHub 'enhancement' issues for planned features. Pull requests are always welcome, and we’ll give you feedback and guidance if you choose to contribute 💚
After cloning the repository, install the dependencies:
pip install -e '.[dev]'
And run the tests:
pytest
Useful resources for developing collectors include the Buildkite Test Analytics docs and the RSpec and Minitest collectors.
Bug reports and pull requests are welcome on GitHub at https://github.com/buildkite/test-collector-python
- Version bump the code, tag and push.
- Publish to the Python Package Index:
python3 -m build
python3 -m twine upload dist/*
- Create a new github release.
The package is available as open source under the terms of the MIT License.
Thanks to the folks at Alembic for building and maintaining this package.