🐛 Add pip installs to Pinned-Dependencies score #2922
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>
Considering the new pipInstalls dependencies in Pinned-Dependencies score, there are some changes. The "all dependencies pinned" test generates now one more Info log for "pip installs are all pinned". The "has 2 dependencies unpinned" test now one more Info log for "pip installs are all pinned" too, plus the total score has to weight now 5 scores instead of 4. The previous score was 10 for actionScore, 10 for dockerFromScore, 0 for dockerDownloadScore, because it's a downloadThenRun problem, another 0 for scriptScore, because it's a downloadThenRun problem, an here we have a bug of the same problem counting twice, but that gives a 20/4=5 for total score. Since we have pip install score, the current score counts a 10 for pipScore and that gives a 30/5=6 for total score. Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>
Considering the new pipInstalls dependencies in Pinned-Dependencies score, there are some changes. The repo being tested, ossf-tests/scorecard-check-pinned-dependencies-e2e, has only third-party GitHub actions pinned. All other dependencies types are unpinned, including pip installs. This gives us 8 for actionScore and 0 for all other scores. Previously the total score was 8/4=2, and now the total score is 8/5=1.6, which rounds down to 1. Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>
gabibguti
requested review from
azeemshaikh38,
justaugustus,
laurentsimon,
naveensrinivasan,
spencerschrock and
raghavkaul
as code owners
gabibguti
temporarily deployed
to
integration-test
GitHub Actions
Inactive
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## main #2922 +/- ##
==========================================
+ Coverage 52.66% 52.67% +0.01%
==========================================
Files 158 158
Lines 12057 12066 +9
==========================================
+ Hits 6350 6356 +6
- Misses 5337 5339 +2
- Partials 370 371 +1 |
When having one unpinned pip install and all other dependencies pinned, the score should be 40/5=8. Also, it should raise 1 warning for the unpinned pip install, 5 infos saying the other dependency types are pinned (2 for GHAs, 2 for dockerfile image and downdloads and 1 for script downdloads), and 0 debug logs since the pip install dependency does not have an error message. Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>
When an error happens to parse a pip install dependency, the error/debug message is saved in "Msg" field. In this case, we were not able to define if the pip install is pinned or not. This dependency is classified as pinned undefined. We treat such cases as pinned cases, so it logs as Info that pip installs are all pinned and counts the score as 10. Then, the final score makes it to 10 as well. Since it logs the error/debug message, the Debug log goes to 1. Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>
Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>
gabibguti
temporarily deployed
to
integration-test
GitHub Actions
Inactive
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.3.0 to 2.3.1. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@b2c19fb...8662eab) --- updated-dependencies: - dependency-name: github/codeql-action dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
gabibguti
temporarily deployed
to
integration-test
GitHub Actions
Inactive
gabibguti
temporarily deployed
to
integration-test
GitHub Actions
Inactive
naveensrinivasan
temporarily deployed
to
integration-test
GitHub Actions
Inactive
balteravishay
pushed a commit
to balteravishay/scorecard
that referenced
this pull request
May 29, 2023
* feat: Add pip install to pinned dependencies score Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com> * test: Fix pinned dependencies evaluation tests Considering the new pipInstalls dependencies in Pinned-Dependencies score, there are some changes. The "all dependencies pinned" test generates now one more Info log for "pip installs are all pinned". The "has 2 dependencies unpinned" test now one more Info log for "pip installs are all pinned" too, plus the total score has to weight now 5 scores instead of 4. The previous score was 10 for actionScore, 10 for dockerFromScore, 0 for dockerDownloadScore, because it's a downloadThenRun problem, another 0 for scriptScore, because it's a downloadThenRun problem, an here we have a bug of the same problem counting twice, but that gives a 20/4=5 for total score. Since we have pip install score, the current score counts a 10 for pipScore and that gives a 30/5=6 for total score. Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com> * test: Fix pinned dependencies e2e tests Considering the new pipInstalls dependencies in Pinned-Dependencies score, there are some changes. The repo being tested, ossf-tests/scorecard-check-pinned-dependencies-e2e, has only third-party GitHub actions pinned. All other dependencies types are unpinned, including pip installs. This gives us 8 for actionScore and 0 for all other scores. Previously the total score was 8/4=2, and now the total score is 8/5=1.6, which rounds down to 1. Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com> * test: Unpinned pip install score When having one unpinned pip install and all other dependencies pinned, the score should be 40/5=8. Also, it should raise 1 warning for the unpinned pip install, 5 infos saying the other dependency types are pinned (2 for GHAs, 2 for dockerfile image and downdloads and 1 for script downdloads), and 0 debug logs since the pip install dependency does not have an error message. Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com> * test: Undefined pip install score When an error happens to parse a pip install dependency, the error/debug message is saved in "Msg" field. In this case, we were not able to define if the pip install is pinned or not. This dependency is classified as pinned undefined. We treat such cases as pinned cases, so it logs as Info that pip installs are all pinned and counts the score as 10. Then, the final score makes it to 10 as well. Since it logs the error/debug message, the Debug log goes to 1. Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com> * test: All dependencies pinned score Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com> * 🌱 Bump github/codeql-action from 2.3.0 to 2.3.1 (ossf#2920) Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.3.0 to 2.3.1. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@b2c19fb...8662eab) --- updated-dependencies: - dependency-name: github/codeql-action dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --------- Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Naveen <172697+naveensrinivasan@users.noreply.github.com> Signed-off-by: Avishay <avishay.balter@gmail.com>
2 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What kind of change does this PR introduce?
This is a bug fix. Unpinned pip installs were identified but did not count for Pinned-Dependencies score. Now, they count.
What is the current behavior?
Pip installs are identified as pinned or unpinned and reported by Scorecard, but do not count for Pinned-Dependencies score.
What is the new behavior (if this is a feature change)?**
Pip installs now count in the aggregated score for Pinned-Dependencies.
Which issue(s) this PR fixes
Fixes #2908
Plus, related to #2582
Special notes for your reviewer
@raghavkaul This PR addresses one of the changes made in #2690.
Does this PR introduce a user-facing change?
For user-facing changes, please add a concise, human-readable release note to
the
release-note(In particular, describe what changes users might need to make in their
application as a result of this pull request.)