Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CIVIS-2844] Add test session public API #72

Merged
merged 16 commits into from
Nov 28, 2023

Conversation

anmarchenko
Copy link
Member

@anmarchenko anmarchenko commented Nov 22, 2023

What does this PR do?
First step towards achieving test suite level visibility (TSLV) in Ruby

Adds test session manual API that is used like that:

Datadog::CI.start_test_session(
  service: "my-web-site-tests",
  tags: { Datadog::CI::Ext::Test::TAG_FRAMEWORK => "my-test-framework" }
)

# .....

Datadog::CI.active_test_session.finish

On recorder level it adds:

  • Datadog::CI::TestSession domain model that wraps tracer's span that represents the whole session
  • Datadog::CI::ConcurrentSpan domain model that provides thread safety around tracer's span (TestSession is inherited from ConcurrentSpan as it is shared between threads)
  • Datadog::CI::Context::Global that represents process-global context that will store active test session, active test module, and a set of active test suites

On transport level it adds:

  • a new serializer factory Serializers::Factories::TestSuiteLevel that is capable of creating test suite level payload serializers
  • a new serializer Serializers::TestV2 that converts test span to citestcycle test event with version 2 (containing test_session_id field)
  • a new serializer Serializers::TestSession that converts session span to citestcycle event

To configuration it adds:

  • new temporary config value settings.ci.experimental_test_suite_level_visibility_enabled - it will exists only for transitionary period before test suite level visibility in Ruby is going to be tested and adopted

Notes

Issues to solve or test in the future:

  • currently all tests are part of a single test session trace when TSLV is enabled - this is not correct as each test must be a separate trace (will be solved in subsequent PR)
  • forking behaviour is not properly tested yet, to test using https://github.com/anmarchenko/openstreetmap-website with Rails parallel executor and adapt accordingly

How to test the change?
Code is covered by a set of unit tests, end2end testing will be possible as soon as test modules and test suites are implemented

@codecov-commenter
Copy link

codecov-commenter commented Nov 22, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (96281ad) 99.20% compared to head (d656b42) 99.30%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #72      +/-   ##
==========================================
+ Coverage   99.20%   99.30%   +0.09%     
==========================================
  Files         111      125      +14     
  Lines        3792     4328     +536     
  Branches      141      153      +12     
==========================================
+ Hits         3762     4298     +536     
  Misses         30       30              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@anmarchenko anmarchenko force-pushed the anmarchenko/test_session_tracing branch from f6c9a93 to b0537e3 Compare November 24, 2023 07:16
@anmarchenko anmarchenko requested review from a team and juan-fernandez November 24, 2023 13:14
@anmarchenko anmarchenko marked this pull request as ready for review November 24, 2023 13:14
@anmarchenko anmarchenko merged commit 8d8cead into main Nov 28, 2023
26 checks passed
@anmarchenko anmarchenko deleted the anmarchenko/test_session_tracing branch November 28, 2023 08:46
@github-actions github-actions bot added this to the 0.5.0 milestone Nov 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants