feat: run scorecard only once

[laurentsimon]

The Action runs scorecard twice: once for the SARIF results, and once to upload the results to the API server. This increases rate limits and is also slower.

We need to run scorecard once only, then:

1. filter out the results depending on trigger type (PR vs not PR), and generate the SARIF output
2. Use all the results and generate the JSON result to upload to the API server

/cc @naveensrinivasan @azeemsgoogle

[spencerschrock]

@justaugustus this is another problem with the current cobra-hooking entrypoint method we do today. Has Cisco run into any rate limiting that this could remediate? (@lelia may have more context?)

[spencerschrock]

Note: I temporarily instrumented my Scorecard fork's call to ossf/scorecard-action with calls to GitHub's /rate_limit endpoint. Analyzing ossf/scorecard takes 66 core API calls, and 4 graphql calls without the publishing step.

The way we run Scorecard twice when publishing results, would therefore waste an extra 66 core and 4 graphql, or ~7% of a repo's hourly quota.

Primary rate limit for GITHUB_TOKEN in GitHub Actions
The rate limit for GITHUB_TOKEN is 1,000 requests per hour per repository. For requests to resources that belong to a GitHub Enterprise Cloud account, the limit is 15,000 requests per hour per repository.

So with the duplicate call, a repo would currently exhaust their hourly quota in 8 commits (assuming no other usage), which isn't great but also not the end of the world. (Runtime is another factor though).

[lelia]

@justaugustus this is another problem with the current cobra-hooking entrypoint method we do today. Has Cisco run into any rate limiting that this could remediate? (@lelia may have more context?)

As far as I'm aware, we haven't hit this rate limit with any of our projects yet — most of the scaling challenges we faced were due to the employment of reusable workflows across multiple orgs. The duplicate call is definitely something I'll keep an eye on as we start leveraging Scorecard across more projects, though.