-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve OpenSSF Scorecard checks #12031
Labels
area/build
Build or GithubAction/CI issues
area/docs
Incorrect, missing, or mistakes in docs
solution/suggested
A solution to the bug has been suggested. Someone needs to implement it.
type/feature
Feature request
type/security
Security related
Comments
This was referenced Oct 18, 2023
Some results are in and after #12035, #12036, and #12058, our Overall Score is now 8.3! Nearly 2 points higher 🙂 New/Current Scorecard JSON output, pretty-printed{
"date": "2023-10-23",
"repo": {
"name": "github.com/argoproj/argo-workflows",
"commit": "08096fc0512ed57a89e4a95ced56512631d8c94b"
},
"scorecard": {
"version": "v4.13.0-29-g49c0eed3",
"commit": "49c0eed3a423f00c872b5c3c9f1bbca9e8aae799"
},
"score": 8.3,
"checks": [
{
"name": "Maintained",
"score": 10,
"reason": "30 commit(s) out of 30 and 20 issue activity out of 30 found in the last 90 days -- score normalized to 10",
"details": null,
"documentation": {
"short": "Determines if the project is \"actively maintained\".",
"url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#maintained"
}
},
{
"name": "Code-Review",
"score": 10,
"reason": "all changesets reviewed",
"details": null,
"documentation": {
"short": "Determines if the project requires human code review before pull requests (aka merge requests) are merged.",
"url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#code-review"
}
},
{
"name": "License",
"score": 10,
"reason": "license file detected",
"details": [
"Info: License file found in expected location: LICENSE:1",
"Info: FSF or OSI recognized license: LICENSE:1"
],
"documentation": {
"short": "Determines if the project has defined a license.",
"url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#license"
}
},
{
"name": "CII-Best-Practices",
"score": 5,
"reason": "badge detected: passing",
"details": null,
"documentation": {
"short": "Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.",
"url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#cii-best-practices"
}
},
{
"name": "Branch-Protection",
"score": -1,
"reason": "internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration",
"details": null,
"documentation": {
"short": "Determines if the default and release branches are protected with GitHub's branch protection settings.",
"url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#branch-protection"
}
},
{
"name": "Signed-Releases",
"score": 8,
"reason": "5 out of 5 artifacts are signed or have provenance",
"details": [
"Warn: release artifact v3.4.12 does not have provenance: https://api.github.com/repos/argoproj/argo-workflows/releases/125983584",
"Info: signed release artifact: argo-workflows-cli-checksums.sig: https://api.github.com/repos/argoproj/argo-workflows/releases/assets/131544568",
"Warn: release artifact v3.5.0 does not have provenance: https://api.github.com/repos/argoproj/argo-workflows/releases/124999540",
"Info: signed release artifact: argo-workflows-cli-checksums.sig: https://api.github.com/repos/argoproj/argo-workflows/releases/assets/130455044",
"Warn: release artifact v3.5.0-rc2 does not have provenance: https://api.github.com/repos/argoproj/argo-workflows/releases/121928138",
"Info: signed release artifact: argo-workflows-cli-checksums.sig: https://api.github.com/repos/argoproj/argo-workflows/releases/assets/126960660",
"Warn: release artifact v3.4.11 does not have provenance: https://api.github.com/repos/argoproj/argo-workflows/releases/120364554",
"Info: signed release artifact: argo-workflows-cli-checksums.sig: https://api.github.com/repos/argoproj/argo-workflows/releases/assets/125063416",
"Warn: release artifact v3.5.0-rc1 does not have provenance: https://api.github.com/repos/argoproj/argo-workflows/releases/117361096",
"Info: signed release artifact: argo-workflows-cli-checksums.sig: https://api.github.com/repos/argoproj/argo-workflows/releases/assets/121637664"
],
"documentation": {
"short": "Determines if the project cryptographically signs release artifacts.",
"url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#signed-releases"
}
},
{
"name": "Dangerous-Workflow",
"score": 10,
"reason": "no dangerous workflow patterns detected",
"details": null,
"documentation": {
"short": "Determines if the project's GitHub Action workflows avoid dangerous patterns.",
"url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#dangerous-workflow"
}
},
{
"name": "Security-Policy",
"score": 10,
"reason": "security policy file detected",
"details": [
"Info: security policy file detected: SECURITY.md:1",
"Info: Found linked content: SECURITY.md:1",
"Info: Found disclosure, vulnerability, and/or timelines in security policy: SECURITY.md:1",
"Info: Found text in security policy: SECURITY.md:1"
],
"documentation": {
"short": "Determines if the project has published a security policy.",
"url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#security-policy"
}
},
{
"name": "Token-Permissions",
"score": 10,
"reason": "GitHub workflow tokens follow principle of least privilege",
"details": [
"Info: topLevel 'contents' permission set to 'read': .github/workflows/changelog.yaml:10",
"Warn: jobLevel 'contents' permission set to 'write': .github/workflows/changelog.yaml:17: Verify which permissions are needed and consider whether you can reduce them. (High effort)",
"Info: topLevel 'contents' permission set to 'read': .github/workflows/ci-build.yaml:17",
"Info: topLevel 'contents' permission set to 'read': .github/workflows/dependabot-reviewer.yml:6",
"Warn: jobLevel 'contents' permission set to 'write': .github/workflows/dependabot-reviewer.yml:13: Verify which permissions are needed and consider whether you can reduce them. (High effort)",
"Info: topLevel 'contents' permission set to 'read': .github/workflows/docs.yaml:16",
"Info: topLevel 'contents' permission set to 'read': .github/workflows/pr.yaml:12",
"Info: topLevel 'contents' permission set to 'read': .github/workflows/release.yaml:20",
"Warn: jobLevel 'contents' permission set to 'write': .github/workflows/release.yaml:278: Verify which permissions are needed and consider whether you can reduce them. (High effort)",
"Info: topLevel 'contents' permission set to 'read': .github/workflows/sdks.yaml:8",
"Warn: jobLevel 'packages' permission set to 'write': .github/workflows/sdks.yaml:15: Verify which permissions are needed and consider whether you can reduce them. (High effort)",
"Warn: jobLevel 'contents' permission set to 'write': .github/workflows/sdks.yaml:16: Verify which permissions are needed and consider whether you can reduce them. (High effort)",
"Info: topLevel 'contents' permission set to 'read': .github/workflows/snyk.yml:11"
],
"documentation": {
"short": "Determines if the project's workflows follow the principle of least privilege.",
"url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#token-permissions"
}
},
{
"name": "Pinned-Dependencies",
"score": -1,
"reason": "internal error: error parsing shell code: Dockerfile.windows:1:5: (( can only be used to open an arithmetic cmd",
"details": null,
"documentation": {
"short": "Determines if the project has declared and pinned the dependencies of its build process.",
"url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#pinned-dependencies"
}
},
{
"name": "Binary-Artifacts",
"score": 10,
"reason": "no binaries found in the repo",
"details": null,
"documentation": {
"short": "Determines if the project has generated executable (binary) artifacts in the source repository.",
"url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#binary-artifacts"
}
},
{
"name": "SAST",
"score": 0,
"reason": "SAST tool is not run on all commits -- score normalized to 0",
"details": [
"Warn: 0 commits out of 30 are checked with a SAST tool",
"Warn: CodeQL tool not detected"
],
"documentation": {
"short": "Determines if the project uses static code analysis.",
"url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#sast"
}
},
{
"name": "Fuzzing",
"score": 0,
"reason": "project is not fuzzed",
"details": [
"Warn: no OSSFuzz integration found: Follow the steps in https://github.com/google/oss-fuzz to integrate fuzzing for your project.\nOver time, try to add fuzzing for more functionalities of your project. (High effort)",
"Warn: no OneFuzz integration found: Follow the steps in https://github.com/microsoft/onefuzz to start fuzzing for your project.\nOver time, try to add fuzzing for more functionalities of your project. (High effort)",
"Warn: no GoBuiltInFuzzer integration found: Follow the steps in https://go.dev/doc/fuzz/ to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)",
"Warn: no PythonAtherisFuzzer integration found: Follow the steps in https://github.com/google/atheris to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)",
"Warn: no CLibFuzzer integration found: Follow the steps in https://llvm.org/docs/LibFuzzer.html to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)",
"Warn: no CppLibFuzzer integration found: Follow the steps in https://llvm.org/docs/LibFuzzer.html to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)",
"Warn: no SwiftLibFuzzer integration found: Follow the steps in https://google.github.io/oss-fuzz/getting-started/new-project-guide/swift-lang/ to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)",
"Warn: no RustCargoFuzzer integration found: Follow the steps in https://rust-fuzz.github.io/book/cargo-fuzz.html to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)",
"Warn: no JavaJazzerFuzzer integration found: Follow the steps in https://github.com/CodeIntelligenceTesting/jazzer to enable fuzzing on your project.\nOver time, try to add fuzzing for more functionalities of your project. (Medium effort)",
"Warn: no ClusterFuzzLite integration found: Follow the steps in https://github.com/google/clusterfuzzlite to integrate fuzzing as part of CI.\nOver time, try to add fuzzing for more functionalities of your project. (High effort)",
"Warn: no HaskellPropertyBasedTesting integration found: Use one of the following frameworks to fuzz your project:\nQuickCheck: https://hackage.haskell.org/package/QuickCheck\nhedgehog: https://hedgehog.qa/\nvalidity: https://github.com/NorfairKing/validity\nsmallcheck: https://hackage.haskell.org/package/smallcheck\nhspec: https://hspec.github.io/\ntasty: https://hackage.haskell.org/package/tasty (High effort)",
"Warn: no TypeScriptPropertyBasedTesting integration found: Use fast-check: https://github.com/dubzzz/fast-check (High effort)",
"Warn: no JavaScriptPropertyBasedTesting integration found: Use fast-check: https://github.com/dubzzz/fast-check (High effort)"
],
"documentation": {
"short": "Determines if the project uses fuzzing.",
"url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#fuzzing"
}
},
{
"name": "Packaging",
"score": 10,
"reason": "publishing workflow detected",
"details": [
"Info: GitHub/GitLab publishing workflow used in run https://api.github.com/repos/argoproj/argo-workflows/actions/runs/6621149784: .github/workflows/ci-build.yaml:144"
],
"documentation": {
"short": "Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.",
"url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#packaging"
}
},
{
"name": "Vulnerabilities",
"score": 9,
"reason": "1 existing vulnerabilities detected",
"details": [
"Warn: Project is vulnerable to: GO-2022-0646"
],
"documentation": {
"short": "Determines if the project has open, known unfixed vulnerabilities.",
"url": "https://github.com/ossf/scorecard/blob/49c0eed3a423f00c872b5c3c9f1bbca9e8aae799/docs/checks.md#vulnerabilities"
}
}
]
} |
This was referenced Oct 29, 2023
7 tasks
This was referenced Feb 4, 2024
This was referenced Jul 16, 2024
4 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
area/build
Build or GithubAction/CI issues
area/docs
Incorrect, missing, or mistakes in docs
solution/suggested
A solution to the bug has been suggested. Someone needs to implement it.
type/feature
Feature request
type/security
Security related
Summary
Closely related to #9769 and #11953
Can see our scores for OpenSSF Scorecard from the badge link here: https://api.securityscorecards.dev/projects/github.com/argoproj/argo-workflows
Overall Score: 6.5 -> 8.6
Scorecard JSON output, pretty-printed
More specifically, here's a few scores to improve:
Branch Protection - Score: -1
internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration
. I don't think we can actually improve this as it requires an admin PAT to check that branch protection is set properly. Unless there's something new with the branch rules feature that allows them to be checked publiclySigned Releases - Score: 8
Pinned Dependencies - Score: -1 -> 5
This is actually giving an error, not sure why though:
internal error: error parsing shell code: Dockerfile.windows:1:5: (( can only be used to open an arithmetic cmd
EDIT2: See issue in Scorecard BUG: Runtime error on Pinned-Dependencies check causes a -1 on its score ossf/scorecard#3316 and PR 🐛 Pinned-Dependencies continues on error ossf/scorecard#3515
Pinned Dependencies check JSON
Our dependencies for JS and Go are all pinned, but I think we could improve
Dockerfile
s by using digests and GitHub Actions by using SHAs (as both have mutable tags, so a SemVer number could just be arbitrarily changed by an attacker)Token Permissions - Score: 0 -> 10
SAST - Score: 0
Fuzzing - Score: 0
Vulnerabilities - Score: 0 -> 9
osv-scanner.toml
to mark some of these as not impacting / ignored.audit fix
UI deps #12036 and fix(deps): upgradeswagger-ui-react
to latest 4.x.x #12058aws-sdk-go-v2
pkg#529. We could potentially ignore this withosv-scanner.toml
as we don't use the vulnerable functionality, but I decided to update instead since it was straightforward enough and should be done eventually anywayUse Cases
OpenSSF is being pushed by Google and CNCF and is also part of CLOMonitor checks that are similarly pushed. We have badges for these already, so we should strive to improve our scores on them.
We should also aim to have a more secure, trusted supply chain, which OpenSSF checks help us to achieve.
Message from the maintainers:
Love this enhancement proposal? Give it a 👍. We prioritise the proposals with the most 👍.
The text was updated successfully, but these errors were encountered: