Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run tests with nextest and publish results #415

Merged
merged 1 commit into from
May 10, 2024
Merged

Run tests with nextest and publish results #415

merged 1 commit into from
May 10, 2024

Conversation

bantonsson
Copy link
Contributor

@bantonsson bantonsson commented May 6, 2024

What does this PR do?

This PR does the following:

  • Adds the nextest test runner and configuration to run single threaded tests separately.
  • Adds test result reporting as checks to the PR
  • Renames the modules for single threaded tests to single_threaded_tests, so they can be easily run separately with nextest.
  • Removes the use of ignore to separate out single threaded tests.
  • Adds a skip_root_tests module name for tests (that were previously ignored 🤷🏼‍♂️) that can't run as root on GitLab CI.
  • Changes the module name for most other tests to tests for consistency and easy skipping when building with cross, which doesn't support nextest.

Motivation

  • Tests were using skip as a way to separate out different test configuration needs (single threaded tests).
  • Failures were hard to find (scrolling through test output).

Additional Notes

Tests should now be run with cargo nextest run

To install nextest run:

cargo install --locked 'cargo-nextest@0.9.67'

or if you are on a rust version newer than 1.71.1 run:

cargo install --locked 'cargo-nextest@0.9.*'

Dependencies

This PR depends on new build images for the GitLab build.

@github-actions github-actions bot added profiling Relates to the profiling* modules. ci-build telemetry sidecar common labels May 6, 2024
@bantonsson bantonsson force-pushed the ban/nextest branch 5 times, most recently from 661b390 to e97fc53 Compare May 6, 2024 15:04
@codecov-commenter
Copy link

codecov-commenter commented May 6, 2024

Codecov Report

Attention: Patch coverage is 76.47059% with 4 lines in your changes are missing coverage. Please review.

Project coverage is 65.36%. Comparing base (00e924b) to head (bb191a5).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #415      +/-   ##
==========================================
+ Coverage   64.77%   65.36%   +0.59%     
==========================================
  Files         184      184              
  Lines       22326    22503     +177     
==========================================
+ Hits        14462    14710     +248     
+ Misses       7864     7793      -71     
Components Coverage Δ
crashtracker 20.31% <0.00%> (-0.07%) ⬇️
data-pipeline 51.45% <ø> (ø)
data-pipeline-ffi 0.00% <ø> (ø)
ddcommon 81.23% <ø> (ø)
ddcommon-ffi 74.93% <ø> (ø)
ddtelemetry 53.72% <ø> (ø)
ipc 81.27% <ø> (+1.96%) ⬆️
profiling 76.89% <ø> (ø)
profiling-ffi 60.05% <ø> (ø)
serverless 0.00% <ø> (ø)
sidecar 29.37% <ø> (+0.70%) ⬆️
sidecar-ffi 0.00% <ø> (ø)
spawn-worker 54.98% <100.00%> (+4.48%) ⬆️
trace-mini-agent 69.12% <ø> (ø)
trace-normalization 97.79% <ø> (ø)
trace-obfuscation 95.74% <ø> (ø)
trace-protobuf 25.64% <ø> (ø)
trace-utils 68.85% <ø> (ø)

@bantonsson bantonsson force-pushed the ban/nextest branch 7 times, most recently from 3af4d57 to 27b3cbf Compare May 8, 2024 12:06
@bantonsson bantonsson marked this pull request as ready for review May 8, 2024 12:28
@bantonsson bantonsson requested review from a team as code owners May 8, 2024 12:28
@bantonsson bantonsson changed the title [WIP] Run tests with nextest and publish results Run tests with nextest and publish results May 8, 2024
@bantonsson bantonsson requested a review from a team as a code owner May 8, 2024 12:52
@danielsn
Copy link
Contributor

danielsn commented May 8, 2024

What happens if someone runs cargo test?

.gitlab-ci.yml Outdated Show resolved Hide resolved
README.md Show resolved Hide resolved
Copy link
Contributor

@ekump ekump left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good! Glad we can get rid of the ignore workflow for single-threaded tests.

@bantonsson
Copy link
Contributor Author

@danielsn if you do cargo test, all tests will be run as normal, and some tests that need to be run single threaded will fail.

Copy link
Contributor

@morrisonlevi morrisonlevi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine to me, assuming CI passes and whatnot.

@bantonsson bantonsson force-pushed the ban/nextest branch 2 times, most recently from e3dece7 to 8090d6b Compare May 8, 2024 15:10
@bantonsson bantonsson merged commit 0d74198 into main May 10, 2024
34 of 35 checks passed
@bantonsson bantonsson deleted the ban/nextest branch May 10, 2024 06:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants