🌱 Add RepoClient re-use E2E tests.

[spencerschrock]

What kind of change does this PR introduce?

e2e test
(Is it a bug fix, feature, docs update, something else?)

• PR title follows the guidelines defined in our pull request documentation

What is the current behavior?

No e2e tests which test for accumulation of results.

What is the new behavior (if this is a feature change)?**

Adds a simple test that the results for repository B are the same whether calculated:

• B in isolation
• A then B, reusing the repo client

• Tests for the changes have been added (for bug fixes/features)

This e2e test was backported to 28b116f to test that it would have caught the bug.
It found issues with both Contributors and License accumulating results.

Contributors

  "Contributors" reason mismatch: "55 different organizations found -- score normalized to 10" vs "21 different organizations found -- score normalized to 10"
  "Contributors" reason mismatch: "21 different organizations found -- score normalized to 10" vs "55 different organizations found -- score normalized to 10"
  "Contributors" reason mismatch: "21 different organizations found -- score normalized to 10" vs "55 different organizations found -- score normalized to 10"
  "Contributors" reason mismatch: "55 different organizations found -- score normalized to 10" vs "21 different organizations found -- score normalized to 10"
  "Contributors" reason mismatch: "21 different organizations found -- score normalized to 10" vs "55 different organizations found -- score normalized to 10"
  "Contributors" reason mismatch: "55 different organizations found -- score normalized to 10" vs "21 different organizations found -- score normalized to 10"

License

"License" details mismatch:
      debug: 0-0
      info: 4-2
      warn: 0-0
"License" details mismatch:
      debug: 0-0
      info: 2-4
      warn: 0-0
"License" details mismatch:
      debug: 0-0
      info: 2-4
      warn: 0-0
"License" details mismatch:
      debug: 0-0
      info: 4-2
      warn: 0-0
"License" details mismatch:
      debug: 0-0
      info: 2-4
      warn: 0-0
"License" details mismatch:
      debug: 0-0
      info: 4-2
      warn: 0-0

Which issue(s) this PR fixes

Fixes #2533

Special notes for your reviewer

I'm not 100% certain why the results are being outputted multiple times.
The E2E test currently runs all scorecard checks 3 times, which results in the following increase in token API usage:

• REST: 152 / 5000
• Search: 6/30
• graphql: 6/5000

Does this PR introduce a user-facing change?

For user-facing changes, please add a concise, human-readable release note to
the release-note

(In particular, describe what changes users might need to make in their
application as a result of this pull request.)

NONE

[spencerschrock]

We could also structure this in a way such that it only runs on a certain make command, but not part of the normal E2E suite if we're concerned about the increase in API usage.

[laurentsimon]

We could also structure this in a way such that it only runs on a certain make command, but not part of the normal E2E suite if we're concerned about the increase in API usage.

how would the make command avoid increasing API usage?

[spencerschrock]

We could also structure this in a way such that it only runs on a certain make command, but not part of the normal E2E suite if we're concerned about the increase in API usage.

how would the make command avoid increasing API usage?

By making it available for contributors/maintainers to run on their own, without including it in the e2e suite that runs on every PR. Of course, the trade-off is remembering to run it when it is needed.

[github-actions]

Stale pull request message

[codecov]

Codecov Report

Merging #2625 (76db929) into main (a7e81bb) will not change coverage.
The diff coverage is n/a.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2625   +/-   ##
=======================================
  Coverage   49.86%   49.86%           
=======================================
  Files         156      156           
  Lines       11576    11576           
=======================================
  Hits         5772     5772           
  Misses       5454     5454           
  Partials      350      350           

[github-actions]

Stale pull request message

[naveensrinivasan]

@spencerschrock A friendly ping.

[spencerschrock]

This test exposed a data race that has been present for a while, which will need to be fixed before these tests can be merged.