Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update Scorecard Action hashes and version comments #2348

Merged
merged 4 commits into from
Mar 27, 2024

Conversation

spencerschrock
Copy link
Contributor

@spencerschrock spencerschrock commented Mar 20, 2024

ossf/scorecard-action v2.1.2 is old and is broken after a Sigstore change. https://blog.sigstore.dev/tuf-root-update/

(Also fixes #2138)

Pre-requisites


Please note that at this time we are only accepting new starter workflows for Code Scanning. Updates to existing starter workflows are fine.


Tasks

For all workflows, the workflow:

  • Should be contained in a .yml file with the language or platform as its filename, in lower, kebab-cased format (for example, docker-image.yml). Special characters should be removed or replaced with words as appropriate (for example, "dotnet" instead of ".NET").
  • Should use sentence case for the names of workflows and steps (for example, "Run tests").
  • Should be named only by the name of the language or platform (for example, "Go", not "Go CI" or "Go Build").
  • Should include comments in the workflow for any parts that are not obvious or could use clarification.
  • Should specify least privileged permissions for GITHUB_TOKEN so that the workflow runs successfully.

For CI workflows, the workflow:

  • Should be preserved under the ci directory.
  • Should include a matching ci/properties/*.properties.json file (for example, ci/properties/docker-publish.properties.json).
  • Should run on push to branches: [ $default-branch ] and pull_request to branches: [ $default-branch ].
  • Packaging workflows should run on release with types: [ created ].
  • Publishing workflows should have a filename that is the name of the language or platform, in lower case, followed by "-publish" (for example, docker-publish.yml).

For Code Scanning workflows, the workflow:

  • Should be preserved under the code-scanning directory.
  • Should include a matching code-scanning/properties/*.properties.json file (for example, code-scanning/properties/codeql.properties.json), with properties set as follows:
    • name: Name of the Code Scanning integration.
    • creator: Name of the organization/user producing the Code Scanning integration.
    • description: Short description of the Code Scanning integration.
    • categories: Array of languages supported by the Code Scanning integration.
    • iconName: Name of the SVG logo representing the Code Scanning integration. This SVG logo must be present in the icons directory.
  • Should run on push to branches: [ $default-branch, $protected-branches ] and pull_request to branches: [ $default-branch ]. We also recommend a schedule trigger of cron: $cron-weekly (for example, codeql.yml).

Some general notes:

  • This workflow must only use actions that are produced by GitHub, in the actions organization, or
  • This workflow must only use actions that are produced by the language or ecosystem that the workflow supports. These actions must be published to the GitHub Marketplace. We require that these actions be referenced using the full 40 character hash of the action's commit instead of a tag. Additionally, workflows must include the following comment at the top of the workflow file:
    # This workflow uses actions that are not certified by GitHub.
    # They are provided by a third-party and are governed by
    # separate terms of service, privacy policy, and support
    # documentation.
    
  • Automation and CI workflows should not send data to any 3rd party service except for the purposes of installing dependencies.
  • Automation and CI workflows cannot be dependent on a paid service or product.

ossf/scorecard-action v2.1.2 is old and doesnt work after a Sigstore
change. https://blog.sigstore.dev/tuf-root-update/

Signed-off-by: Spencer Schrock <sschrock@google.com>
@spencerschrock spencerschrock requested review from a team as code owners March 20, 2024 21:00
@github-actions github-actions bot added the code-scanning Related to workflows that show on the Code Scanning setup page label Mar 20, 2024
@jonjanego jonjanego removed the request for review from a team March 20, 2024 21:57
Copy link
Contributor

@jhutchings1 jhutchings1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

Copy link
Contributor

@alexisabril alexisabril left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @spencerschrock for the PR!

I'm requesting a single change as we don't currently support the latest version of upload-artifact in our enterprise-server offering: https://github.com/actions/upload-artifact?tab=readme-ov-file#v4---whats-new.

If that change looks good to you, I'm happy to merge this in.

@@ -59,14 +59,15 @@ jobs:
# Upload the results as artifacts (optional). Commenting out will disable uploads of run results in SARIF
# format to the repository Actions tab.
- name: "Upload artifact"
uses: actions/upload-artifact@3cea5372237819ed00197afe530f5a7ea3e805c8 # v3.1.0
uses: actions/upload-artifact@5d5d22a31266ced268874388b861e4b58bb5c2f3 # v4.3.1
Copy link
Contributor

@alexisabril alexisabril Mar 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
uses: actions/upload-artifact@5d5d22a31266ced268874388b861e4b58bb5c2f3 # v4.3.1
uses: actions/upload-artifact@97a0fba1372883ab732affbe8f94b823f91727db # v3-node20

Copy link
Contributor Author

@spencerschrock spencerschrock Mar 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the Scorecard side, v3.1.3 is fine by me.

Are there any expected issues on GitHub's end from v3.1.3 specifying node16? Given the Spring 2024 transition?

I see there's also a v3-node20 tag, but due to some Scorecard imposed workflow restrictions when publish_results: true I would need to make changes in our backend to support that tag since it doesn't live in main.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@spencerschrock That's a great callout and a note to my future self that we may need an additional tag for semantic versioning.

I am curious about the consideration you mentioned in your backend. Would this not need to be changed regardless as we may always introduce a new tag to fix bugs in prior versions? For instance, a 3.1.9 could be created in the future to address some newly discovered issue, which would be a divergence from main.

Copy link
Contributor Author

@spencerschrock spencerschrock Mar 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When publish_results is true, we want to check the workflow is generating legitimate scores before accepting them, as GitHub Actions allows the pinned SHA to be from the repo or a fork. So we don't want a forked ossf/scorecard-action (or similar) to generate bogus scores.

This is specifically something we check only when pinned to a hash, so someone could always pin to a tag (e.g. @v3.1.9).

Several of us have looked at the available APIs and there's no perfect option for "does this commit belong to the repo or a fork?". https://github.com/orgs/community/discussions/78161.

So for most repos we currently only check main for latency purposes, based primarily on their past behavior. For github/codeql-action we also check releases/v2 and releases/v3 as they started maintaining multiple release branches.

Copy link
Contributor Author

@spencerschrock spencerschrock Mar 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've made the proposed changes to this workflow. I included a github/codeql-action/upload-sarif bump as well to move it from node16 -> node20 as well.

Done. I still need to merge ossf/scorecard-webapp#599 on my side.

spencerschrock and others added 3 commits March 26, 2024 14:12
dependabot will suggest upgrade to v4.3.1 for repos that can upgrade.
note: v3.pre.node20 is how dependabot refers to the pinned hash, so
use that so it can upgrade the comment

Signed-off-by: Spencer Schrock <sschrock@google.com>
Signed-off-by: Spencer Schrock <sschrock@google.com>
@alexisabril alexisabril merged commit 4620c76 into actions:main Mar 27, 2024
3 checks passed
@spencerschrock spencerschrock deleted the scorecard-use-major-versions branch March 27, 2024 20:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
code-scanning Related to workflows that show on the Code Scanning setup page
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants