[CIVIS-2892] datadog_cov native extension for per test code coverage

[anmarchenko]

What does this PR do?
Adds datadog_cov native extension for per test code coverage.

Usage of Datadog::CI::ITR::Coverage::DDCov:

# track files coverage in the current folder
cov = Datadog::CI::ITR::Coverage::DDCov.new(root: Dir.pwd)
cov.start_coverage
Calculator.new.add(1, 2) # => 3
cov.stop_coverage # => { "/path/to/current-folder/calculator.rb" => true }

# track lines coverage
cov = Datadog::CI::ITR::Coverage::DDCov.new(root: Dir.pwd, mode: :lines)
cov.start_coverage
Calculator.new.add(1, 2) # => 3
cov.stop_coverage # => { "/path/to/current-folder/calculator.rb" => {12 => true, 13 => true} }

Additional Notes
Current limitations:

• UTF-8 in source file paths is not supported because of additional overhead of parsing UTF-8 strings: will be added later as an opt-in via environment variable as non-ASCII characters in paths and filenames are rare
• Lines coverage is barebones and not optimized: ITR will use only files coverage in the first stage

How to test the change?
Unit tests are provided in spec/datadog/ci/cov_spec.rb

Scenarios tested:

• coverage with different root values
• root folder path is longer than covered files' paths
• coverage is not collected after stop_coverage call
• coverage of mixed in code from helpers
• multi threaded coverage (coverage is collected per thread)
• UTF-8 in file names does not fail (but produces incorrect result)

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 99.57447% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 99.22%. Comparing base (6a994b6) to head (e4e86f3).

Files	Patch %	Lines
lib/datadog/ci/utils/git.rb	50.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##              1.0     #137      +/-   ##
==========================================
+ Coverage   99.19%   99.22%   +0.02%     
==========================================
  Files         163      171       +8     
  Lines        7240     7468     +228     
  Branches      302      310       +8     
==========================================
+ Hits         7182     7410     +228     
  Misses         58       58              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[juan-fernandez]

🎉 very nice!

[tonyredondo]

LGTM, I left a couple of nits, and I have a question regarding what happens if a test spawn new threads, is that possible? how the code coverage works in that case?

[ivoanjo]

Left a few notes! :)

[ivoanjo]

Ahaha clearly my head kept reviewing the code as an async background task and when I took a break to go get some tea the async task resolved and pointed out these two issues 🤣

[ivoanjo]

Left a few more notes, I think you're very close ;)

One thing that occurred to me is that we have a weird setup for packaging dd-trace-rb when used by single step instrumentation (sometimes known as "lib injection" in dd-trace-rb).

TL;DR we handle gems with native extensions in a specific way, so if this native gem is still going to become a dependency of ddtrace 1.x, it's probably worth checking nothing is broken there.

I'm not entirely sure of all the details, @TonyCTHsu is the right person to ask "if this new native extension going to break or not".

[anmarchenko]

This pull request is now rebased on top of 1.0 branch and the base is set to 1.0.

From now on, all ITR functionality will go into 1.0 branch so that we don't add additional risks for ddtrace 1.x gem.

[anmarchenko]

@ivoanjo this is ready for another pass

[ivoanjo]

👍 Native bits look good to me! I didn't re-review the other things that came in from the rebase/changing the branch :)