Compact STDOUT/JSON Output

[dclayton-godaddy]

Feature Request

Add --compact flag to CLI to enable abbreviated stdout.

Is your feature request related to a problem? Please describe.

The default outputs product such large output (300K+ lines) which can be painful to review.

Describe the solution you'd like

Add --compact flag to CLI to display the most relevant information used to ignore false flags.

STDOUT

[{IssueType} ({Signature})] {file_path}@L{matched_line_no}: {matched_line}

[High Entropy (49d8990db16295778fdb98560a4b9149ca9161c88423b6dc5cc4ffce74521483)] src/file1.js@L1: // flow-typed signature: dc381ee55406f66b7272c6343db0834b

JSON

{
  "found_issues": [
    {
      "file_path": "src/file1.js",
      "matched_string": "dc381ee55406f66b7272c6343db0834b",
      "matched_line": "// flow-typed signature: dc381ee55406f66b7272c6343db0834b",
      "matched_line_no": 1,
      "issue_type": "High Entropy",
      "issue_detail": ""
    }
  ]
}

Describe alternatives you've considered

The current json/stdout.

Teachability, Documentation, Adoption, Migration Strategy

[tarkatronic]

This looks great! I love the idea. I could definitely see eventually switching to this as the default behavior, and allowing people to turn on the more verbose output when necessary.

That actually brings up another interesting though. When coupled with interactive mode (as proposed in #105), we should give the user the option to view the full verbose output.

[dclayton-godaddy]

Compact mode is intended to be limited to 1 line per issue. I wanted interactive mode to show more lines around it to get context on the usage of the flagged item. I don't want this mode to take away from the intent of the two options. Compact is intended to be compact and if you needed more info, you can look at the file/line yourself. Interactive is intended to view and take action on-the-fly (which you might need more info than compact).

[tarkatronic]

Sure, I understand the intent. But should these necessarily be mutually exclusive? I would think no. Just because this is how you use it doesn't mean that's how everybody will use it. 😄 So I say why limit the users? If they want to do --compact --ineractive, I say we support that use case.

[dclayton-godaddy]

True. I guess compact can have different output for interactive.

[jgowdy]

+1 Good idea

[rdrey]

For compact JSON I've been using this:

tartufo --json scan-local-repo . \
    | jq '[.found_issues[] | with_entries(select(.key == ("file_path", "matched_string", "signature")))] | unique' -C \
    | less -r

I also needed unique because I often have the same secret showing up in the diffs of multiple commits, which leads to duplicate reports otherwise.

A related issue we might be able to address with compact output:

The stdout output is difficult to use at the moment because tartufo doesn't highlight the matched string when stdout is a pipe/file. We might want to report the matched string separately, so it's still easy to spot in non-colour output or add a flag to always colorize output.