Add branch details to JSON report

[christoph-blessing]

• Extend json report test
• Add branch details to json report

Fixes: #1425

[jaltmayerpizzorno]

Mm, in Slipcover, I chose to output executed_branches and missing_branches directly next to executed_lines, etc., as a list of tuples (which in JSON becomes a list of 2-element lists). With ->exit branches being shown as [x,0] "tuples" (where x is the source line).

We don't have to have the same format, but it would be nice...

[christoph-blessing]

Hey, thanks for the feedback. I changed the format to the one you requested.

[jaltmayerpizzorno]

Cool! Thank you.

[nedbat]

I'm curious why 0 is better than -1 for exit?

[jaltmayerpizzorno]

Mm, I think both work well. I just a hope Ed to pick 0. Do you have a reason why -1 is better?

.. Juan

[nedbat]

Internally, entering and exiting a code object is represented in arcs as negative numbers. From one of the docstrings:

        Negative numbers have special meaning.  If the starting line number is
        -N, it represents an entry to the code object that starts at line N.
        If the ending ling number is -N, it's an exit from the code object that
        starts at line N.

Here's some actual data from the .coverage file:

.coverage> select * from arc;
 +---------+------------+--------+------+
 | file_id | context_id | fromno | tono |
 +---------+------------+--------+------+
 | 1       | 1          | 3      | 4    |
 | 1       | 1          | 5      | -1   |
 | 1       | 1          | 12     | 13   |
 | 1       | 1          | 17     | 18   |
 | 1       | 1          | 9      | 17   |
 | 1       | 1          | 1      | 9    |
 | 1       | 1          | 18     | -1   |
 | 1       | 1          | 15     | 11   |
 | 1       | 1          | -1     | 1    |
 | 1       | 1          | 4      | 5    |
 | 1       | 1          | 12     | 15   |
 | 1       | 1          | 10     | 11   |
 | 1       | 1          | -9     | 10   |
 | 1       | 1          | 7      | 3    |
 | 1       | 1          | 4      | 7    |
 | 1       | 1          | 13     | -9   |
 | 1       | 1          | 2      | 3    |
 | 1       | 1          | 11     | 12   |
 | 1       | 1          | -1     | 2    |
 +---------+------------+--------+------+

I'm curious why you haven't seen negative numbers other than -1 in testing.

[nedbat]

It's easy to get other negative numbers. This is using the code from this pull request:

# /tmp/b1438.py

def never(x):
    print("ok")
    if x: return 17
    print("nope")

never(0)

then:

% coverage run --branch /tmp/b1438.py
ok
nope

% coverage report -m
Name                    Stmts   Miss Branch BrPart  Cover   Missing
-------------------------------------------------------------------
/private/tmp/b1438.py       5      0      2      1    86%   5->exit
-------------------------------------------------------------------
TOTAL                       5      0      2      1    86%

% coverage json
Wrote JSON report to coverage.json

% jq <coverage.json
{
  "meta": {
    "version": "6.4.5a0",
    "timestamp": "2022-09-05T07:25:02.254430",
    "branch_coverage": true,
    "show_contexts": false
  },
  "files": {
    "/private/tmp/b1438.py": {
      "executed_lines": [
        3,
        4,
        5,
        6,
        8
      ],
      "summary": {
        "covered_lines": 5,
        "num_statements": 5,
        "percent_covered": 85.71428571428571,
        "percent_covered_display": "86",
        "missing_lines": 0,
        "excluded_lines": 0,
        "num_branches": 2,
        "num_partial_branches": 1,
        "covered_branches": 1,
        "missing_branches": 1
      },
      "missing_lines": [],
      "excluded_lines": [],
      "executed_branches": [
        [
          5,
          6
        ]
      ],
      "missing_branches": [
        [
          5,
          -3
        ]
      ]
    }
  },
  "totals": {
    "covered_lines": 5,
    "num_statements": 5,
    "percent_covered": 85.71428571428571,
    "percent_covered_display": "86",
    "missing_lines": 0,
    "excluded_lines": 0,
    "num_branches": 2,
    "num_partial_branches": 1,
    "covered_branches": 1,
    "missing_branches": 1
  }
}

missing_branches mentions -3, because the exit was never taken from the code starting at line 3.

[jaltmayerpizzorno]

For that .py, Slipcover outputs a missing (5,5) branch, since really it is a same-line branch (to the return statement), below. For branches that exit a function, it outputs 0 as the destination... As in the (2,0) branch for

def foo(x):
    if x == 0:
        x += 1
foo(0)

I could change that to -1, but to change that to something dependent upon where the code starts would take more effort.

{
    "meta": {
        "software": "slipcover",
        "version": "0.2.0",
        "timestamp": "2022-09-05T14:24:30.974682",
        "branch_coverage": true
    },
    "files": {
        "b1438.py": {
            "executed_lines": [
                3,
                4,
                5,
                6,
                8
            ],
            "missing_lines": [],
            "summary": {
                "covered_lines": 5,
                "missing_lines": 0,
                "covered_branches": 1,
                "missing_branches": 1,
                "percent_covered": 85.71428571428571
            },
            "executed_branches": [
                [
                    5,
                    6
                ]
            ],
            "missing_branches": [
                [
                    5,
                    5
                ]
            ]
        }
    },
    "summary": {
        "covered_lines": 5,
        "missing_lines": 0,
        "covered_branches": 1,
        "missing_branches": 1,
        "percent_covered": 85.71428571428571
    }
}

[nedbat]

Maybe we need to talk about the value of coverage.py and slipcover producing the same data, vs coverage.py producing data with more information?

[jaltmayerpizzorno]

Right... Is there truly value in encoding the starting line of the code in the branch destination? How would that help a user?

I guess the more the packages differ in the data they collect, potentially the more difficult a Slipcover-based collector for coverage.py might become... which, of course, at this point is just an idea.

In this case in particular, it seems like we'd generate different data anyway, because of the same-line branch.

[nedbat]

I've squashed and merged this as of 95195b1. I've kept the negative line numbers for entrances and exits, to keep the data uniform with the rest of coverage.py.

Thanks!

[nedbat]

This is now released as part of coverage 6.5.0.