-
Notifications
You must be signed in to change notification settings - Fork 133
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: Instrument unit tests with CI Test Visibility 2.5.0
#1999
Conversation
…t-testing` also enable gathering code coverage for tested targets
Datadog ReportBranch report: ✅ 0 Failed, 1848 Passed, 0 Skipped, 1m 15.14s Total Time |
blocking/ We also need to pass CI Provider variables to all tests. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good 👍 Hope to remove the xcodeproj in a near future 🤞
2.5.0-alpha3
2.5.0
What and why?
📦 This PR enables (back*) the Test Visibility instrumentation for our unit tests. With our tests being more stable in GitLab, we can leverage this instrumentation for more visibility into their performance and runs.
*
we disabled Test Visibility a while ago, due to increased number of flaky test runs in Bitrise CI.How?
Following the documentation, config ENVs are injected to every
xcscheme
and their values are exported intest.sh
. Each scheme is configured to gather code coverage data, so we can later enable ITR.🐶 This PR only integrates
dd-sdk-swift-testing
to an extra CI job meant for shadowing regular iOS unit tests runs. If we find our tests stable, we will integrate it to regular "Unit Test" jobs.Unfortunately, there is no simpler way than applying the same configuration to all 14 schemes. The fact that we centralize tests execution through
test.sh
helps to control values, but the actual ENV injection must include forwarding it from Environment Variables list inxscheme
definition. As explained in docs, the basicDD_TEST_RUNNER=1 ... xcodebuild test -scheme ...
only works forPackage.swift
and not for Xcode project.Review checklist