Relevant lines differs for files with no coverage vs. with coverage

[elliottmason]

Running specs and cucumber features separately yield different statistics in terms of relevant lines. What I am noticing is that both suites load all of the same files. However, what registers as a "relevant line" seems to change based upon whether or not that file has any test coverage.

At a glance, it seems that for files with coverage, single statements that span multiple lines are counted as a single line. If the file has no coverage, each individual line of code is counted.

e.g. this:

def show
  @product = base_scope
             .includes(colors_products: :color)
             .find(params[:id])
end 

Seems to count as 3 lines when the file has any amount of coverage, but 5 lines when the file has no coverage. The same file in the former example has a total number of relevant lines of 56 where as in the latter scenario (no coverage) the number of relevant lines is 106. The total number of lines (133) is picked up accurately in either scenario.

I'm using SimpleCov 0.15.1 on ruby 2.3.3p222 (2016-11-21 revision 56859) [x86_64-darwin17] and Rails 5.0.0.1

[PragTob]

Hi!

Thanks for the report, that does seem strange indeed. I vaguely recall that there might be a branch for that where we could fix it... shouldn't happen. Thanks a ton!

[joroshiba]

Seeing this same behaivior in 0.16.1

[jeffclemens-ab]

Tested and untested files each use a different method to calculate "relevant lines":

• Tested files: Determined by the Coverage module as it interprets the file
• Untested files: Determined by SimpleCov's line classifier

While the Coverage module intelligently handles multi-line statements, like the one posted by @elliottmason , the line classifier does not.

Ideally, the same method (ie. the Coverage module) would be used for both tested and untested files so that "relevant lines" would be consistent either way. (This problem is actually older than the line classifier.)

I think it would be possible use the Coverage module for untested files by simply loading them from SimpleCov after the tests are finished. Before I start a PR, I'm wondering if any maintainers have feedback on that approach?

For now, Rails projects can eager_load all their files to ensure that the Coverage module calculates "relevant lines" for all files, tested and untested.

[PragTob]

Hi @jeffclemens-ab - in general the idea is good and thanks for looking into it.

The problem however is that to the best of my knowledge the Coverage library needs you to require a ruby file. We can't just go around requiring ruby files as they might have a side effect.

if there's another way that's probably a good appraoch.