Skip to content

Commit

Permalink
[#1554] Fix code scanning alert - Pinned-Dependencies
Browse files Browse the repository at this point in the history 
Closes #1554
  • Loading branch information
@remkop
remkop committed Jan 31, 2022
1 parent 0e36711 commit 87ff7a6
Show file tree
Hide file tree
Showing 2 changed files with 17 additions and 6 deletions.
22 changes: 16 additions & 6 deletions .github/workflows/ci.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,19 @@ jobs:

runs-on: ${{ matrix.os }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@14dc64f30986eaa2ad2dddcec073f5aab18e5a24 # v1
with:
egress-policy: audit

- name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@629c2de402a417ea7690ca6ce3f33229e27606a5 # v2

- name: Validate Gradle Wrapper
uses: gradle/wrapper-validation-action@v1
uses: gradle/wrapper-validation-action@e6e38bacfdf1a337459f332974bb2327a31aaf4b # v1

- name: Configure JDK ${{ matrix.java-version }}
uses: actions/setup-java@v2
uses: actions/setup-java@f0bb91606209742fe3ea40199be2f3ef195ecabf # v2
with:
distribution: 'zulu'
java-version: ${{ matrix.java-version }}
Expand All @@ -35,18 +40,23 @@ jobs:

runs-on: ${{ matrix.os }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@14dc64f30986eaa2ad2dddcec073f5aab18e5a24 # v1
with:
egress-policy: audit

- name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@629c2de402a417ea7690ca6ce3f33229e27606a5 # v2

- name: Configure JDK ${{ matrix.java-version }}
uses: actions/setup-java@v2
uses: actions/setup-java@f0bb91606209742fe3ea40199be2f3ef195ecabf # v2
with:
distribution: 'zulu'
java-version: ${{ matrix.java-version }}
cache: gradle

- name: Build
uses: gradle/gradle-build-action@v2
uses: gradle/gradle-build-action@c6619898ec857b418d6436d3efe8a0becf74eb9e # v2
with:
gradle-version: 1.12
build-root-directory: picocli-legacy-tests/
Expand Down
1 change: 1 addition & 0 deletions RELEASE-NOTES.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ Picocli follows [semantic versioning](http://semver.org/).
* [#1457] DOC: add caution about arguments in @files with quoted option parameters. Thanks to [Andreas Deininger](https://github.com/deining) for the pull request.
* [#1544][#1545] DOC: Add NOTICE file with GPL v2 + CPE license. Thanks to [Keith M Swartz](https://github.com/kswartz26) for the pull request.
* [#1553] SECURITY: Fix code scanning alert - Token-Permissions
* [#1554] Fix code scanning alert - Pinned-Dependencies
* [#1491] BUILD: Add build job in CI; Thanks to [Goooler](https://github.com/Goooler) for the pull request.
* [#1482] BUILD: Optimize gradle; Thanks to [Goooler](https://github.com/Goooler) for the pull request.
* [#1461] BUILD: Allow publishing without signing for non-release versions. Thanks to [Andreas Deininger](https://github.com/deining) for raising this.
Expand Down

11 comments on commit 87ff7a6

@varunsh-coder
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @remkop happy to see you trying out harden-runner. You get a report of the outbound calls and insights in a link in the build log, e.g.
https://github.com/remkop/picocli/runs/5002447373?check_suite_focus=true#step:3:5
You can then use the recommended policy. I am curious if there was a reason you did not use the policy in block mode?
Also, I am building automation to make it easy to meet scorecard requirements. Please let me know if you have feedback on that, or if there were pain points that can be automated...Thanks!

@remkop
Copy link
Owner Author

@remkop remkop commented on 87ff7a6 Feb 1, 2022 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @varunsh-coder, thank you for reaching out!

I am curious if there was a reason you did not use the policy in block mode?

I basically used StepSecurity's online tool: copy-and-pasted my GitHub action CI workflow config file, checked that the result was not obviously bad, and used that as my new config.
I just used what was suggested by the tool. I will take a look at the block policy, I haven't looked in detail at what each of these policies do yet.

Regarding feedback/pain points:

  1. Looking at the result of all this on the openssf dashboard, there are a few things that stand out:
  • Summary does not show up (what can I do to fix that?)
  • Detected URLs does not point to the project's web page or GitHub page (what can I do to fix that?)
  • OpenSSF Scorecard section: last scan was Jan 8th; any idea what the refresh rate is of this dashboard?
  1. Branch Protection - the bar is too high
  • There is a lengthy explanation about Tiers (Tier 1-5), probably copied from the Scorecard Check documentation. This is misleading. I enabled Branch Protection on the main branch and configured everything to meet the requirements for Tier 1-3. I cannot currently meet Tier 4 as it "requires reviewers >= 2". However, I get zero credit for meeting Tier 1-3... So it appears that, at least for now, this is actually simply a lump sum: either you meet all requirements up to and including Tier 5 or you don't. Can the implementation be improved to receive partial credit for meeting some of these Tiers?
  • picocli is maintained by a single maintainer. The Branch Protection requirement seems to simply not cater for this model. The suggested remedy is "(...) try to recruit more maintainers to the project (...)". Alternatives are needed for single-maintainer projects to prevent such projects from being considered inherently unsafe.

Finally, thank you for your work on this. I was impressed by how easy StepSecurity's online tool made it to fix the Pinned Dependencies item! Great work!

@remkop
Copy link
Owner Author

@remkop remkop commented on 87ff7a6 Feb 1, 2022 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought of two more things (related to the openssf dashboard):

  • Packages: please also provide support for packages published to Maven Central. Picocli's binaries are published to Maven Central as this is the central artifact repository for JVM languages. How can I get these to show on the dashboard?
  • Signed Releases: Maven Central is very strict about signed releases. All of picocli's releases are available on Maven Central and have been signed (e.g. jar with signature jar.asc.sha256, same for the test jar, javadocs, sources, test-sources, and pom). How can I get these to show on the dashboard?

@varunsh-coder
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @varunsh-coder, thank you for reaching out!

I am curious if there was a reason you did not use the policy in block mode?

I basically used StepSecurity's online tool: copy-and-pasted my GitHub action CI workflow config file, checked that the result was not obviously bad, and used that as my new config. I just used what was suggested by the tool. I will take a look at the block policy, I haven't looked in detail at what each of these policies do yet.

Ok, whenever you check it out, feel free to tag me if you have feedback.

Regarding feedback/pain points:

  1. Looking at the result of all this on the openssf dashboard, there are a few things that stand out:
  • Summary does not show up (what can I do to fix that?)
  • Detected URLs does not point to the project's web page or GitHub page (what can I do to fix that?)
  • OpenSSF Scorecard section: last scan was Jan 8th; any idea what the refresh rate is of this dashboard?

@laurentsimon - hope you don't mind me adding you to this thread. I do not know what is the source of this ossf dashboard. Where should these issues be reported?

  1. Branch Protection - the bar is too high
  • There is a lengthy explanation about Tiers (Tier 1-5), probably copied from the Scorecard Check documentation. This is misleading. I enabled Branch Protection on the main branch and configured everything to meet the requirements for Tier 1-3. I cannot currently meet Tier 4 as it "requires reviewers >= 2". However, I get zero credit for meeting Tier 1-3... So it appears that, at least for now, this is actually simply a lump sum: either you meet all requirements up to and including Tier 5 or you don't. Can the implementation be improved to receive partial credit for meeting some of these Tiers?
  • picocli is maintained by a single maintainer. The Branch Protection requirement seems to simply not cater for this model. The suggested remedy is "(...) try to recruit more maintainers to the project (...)". Alternatives are needed for single-maintainer projects to prevent such projects from being considered inherently unsafe.

Yes, I think this is fair feedback. Lot of open source projects are maintained by a single maintainer. @laurentsimon - FYI. Please let me know if want me to create an issue for this or may be this is already tracked?

Finally, thank you for your work on this. I was impressed by how easy StepSecurity's online tool made it to fix the Pinned Dependencies item! Great work!

Thanks @remkop, to clarify I am the founder of StepSecurity, I am not a scorecards maintainer, but I work with @laurentsimon and others who maintain scorecards...

@laurentsimon
Copy link

@laurentsimon laurentsimon commented on 87ff7a6 Feb 1, 2022 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @varunsh-coder, thank you for reaching out!

I am curious if there was a reason you did not use the policy in block mode?

I basically used StepSecurity's online tool: copy-and-pasted my GitHub action CI workflow config file, checked that the result was not obviously bad, and used that as my new config. I just used what was suggested by the tool. I will take a look at the block policy, I haven't looked in detail at what each of these policies do yet.

Ok, whenever you check it out, feel free to tag me if you have feedback.

Regarding feedback/pain points:

  1. Looking at the result of all this on the openssf dashboard, there are a few things that stand out:
  • Summary does not show up (what can I do to fix that?)
  • Detected URLs does not point to the project's web page or GitHub page (what can I do to fix that?)
  • OpenSSF Scorecard section: last scan was Jan 8th; any idea what the refresh rate is of this dashboard?

the OSSF dashboard currently does not refresh using the results of the scorecard action analysis. We are working on it actively and will also launch a scorecard.dev website soon with that information. Thank you for your patience.

@azeemshaikh38 @inferno-chromium @scovetta @david-a-wheeler

@laurentsimon - hope you don't mind me adding you to this thread. I do not know what is the source of this ossf dashboard. Where should these issues be reported?

  1. Branch Protection - the bar is too high
  • There is a lengthy explanation about Tiers (Tier 1-5), probably copied from the Scorecard Check documentation. This is misleading. I enabled Branch Protection on the main branch and configured everything to meet the requirements for Tier 1-3. I cannot currently meet Tier 4 as it "requires reviewers >= 2". However, I get zero credit for meeting Tier 1-3... So it appears that, at least for now, this is actually simply a lump sum: either you meet all requirements up to and including Tier 5 or you don't. Can the implementation be improved to receive partial credit for meeting some of these Tiers?
  • picocli is maintained by a single maintainer. The Branch Protection requirement seems to simply not cater for this model. The suggested remedy is "(...) try to recruit more maintainers to the project (...)". Alternatives are needed for single-maintainer projects to prevent such projects from being considered inherently unsafe.

Yes, I think this is fair feedback. Lot of open source projects are maintained by a single maintainer. @laurentsimon - FYI. Please let me know if want me to create an issue for this or may be this is already tracked?

Thank you @remkop for the feedback. You are receiving credits for enabling branch protection. I ran scorecard manually and your score for branch protection is 8, which is as good as it gets. We still provide the results to get to 10, but we understand those criteria cannot be met by everyone. You can ignore the remaining alerts for branch protection.

Please let me know if there's a problem with the messaging in the results that we could improve. I think the main issue is that the OSSF dashboard does not reflect the results. Is there another place we could improve?

cc @olivekl

Finally, thank you for your work on this. I was impressed by how easy StepSecurity's online tool made it to fix the Pinned Dependencies item! Great work!

Thanks @remkop, to clarify I am the founder of StepSecurity, I am not a scorecards maintainer, but I work with @laurentsimon and others who maintain scorecards...

@varunsh-coder
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought of two more things (related to the openssf dashboard):

  • Packages: please also provide support for packages published to Maven Central. Picocli's binaries are published to Maven Central as this is the central artifact repository for JVM languages. How can I get these to show on the dashboard?

Sorry, I had missed this feedback. As per scorecards

"The check currently looks for GitHub packaging workflows and language-specific GitHub Actions that upload the package to a corresponding hub, e.g., Npm. We plan to add better support to query package manager hubs directly in the future, e.g., for Npm, PyPi."

My understanding is that since you do not have a packaging workflow in the repo that publishes to maven, this is not showing up on the dashboard.

  • Signed Releases: Maven Central is very strict about signed releases. All of picocli's releases are available on Maven Central and have been signed (e.g. jar with signature jar.asc.sha256, same for the test jar, javadocs, sources, test-sources, and pom). How can I get these to show on the dashboard?

As per scorecards,

"This check looks for the following filenames in the project's last five releases: *.minisig, *.asc (pgp), *.sig, *.sign."

@remkop I see that you publish .jar files in releases, but not the .asc.sha256 file. Not sure if consumers will use the signed file, but also publishing the signature files in the release assets, will make it equivalent to what is published to maven central. @laurentsimon I observed the extension for signed file is .asc.sha256 in this case. So even if they were published, it would not get detected by the check...

@laurentsimon
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought of two more things (related to the openssf dashboard):

  • Packages: please also provide support for packages published to Maven Central. Picocli's binaries are published to Maven Central as this is the central artifact repository for JVM languages. How can I get these to show on the dashboard?

Sorry, I had missed this feedback. As per scorecards

"The check currently looks for GitHub packaging workflows and language-specific GitHub Actions that upload the package to a corresponding hub, e.g., Npm. We plan to add better support to query package manager hubs directly in the future, e.g., for Npm, PyPi."

My understanding is that since you do not have a packaging workflow in the repo that publishes to maven, this is not showing up on the dashboard.

that's correct. Currently it's hard to map a package name back to its repo, so we're only looking for workflows that build the package. In your case, we're looking for the use of actions/setup-java action or mvn [...] deploy commands. Is the a reasonable assumption?

  • Signed Releases: Maven Central is very strict about signed releases. All of picocli's releases are available on Maven Central and have been signed (e.g. jar with signature jar.asc.sha256, same for the test jar, javadocs, sources, test-sources, and pom). How can I get these to show on the dashboard?

As per scorecards,

"This check looks for the following filenames in the project's last five releases: *.minisig, *.asc (pgp), *.sig, *.sign."

@remkop I see that you publish .jar files in releases, but not the .asc.sha256 file. Not sure if consumers will use the signed file, but also publishing the signature files in the release assets, will make it equivalent to what is published to maven central. @laurentsimon I observed the extension for signed file is .asc.sha256 in this case. So even if they were published, it would not get detected by the check...

Great, I've created a tracking issue on scorecard repo ossf/scorecard#1582. Can you confirm .asc.sha256 is the standard extension for maven's signed packages? Is it fair to expect maintainers to add this to their GitHub release? Please comment on the issue for additional information how we should implement this.

@remkop
Copy link
Owner Author

@remkop remkop commented on 87ff7a6 Feb 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought of two more things (related to the openssf dashboard):

  • Packages: please also provide support for packages published to Maven Central. Picocli's binaries are published to Maven Central as this is the central artifact repository for JVM languages. How can I get these to show on the dashboard?

Sorry, I had missed this feedback. As per scorecards
"The check currently looks for GitHub packaging workflows and language-specific GitHub Actions that upload the package to a corresponding hub, e.g., Npm. We plan to add better support to query package manager hubs directly in the future, e.g., for Npm, PyPi."
My understanding is that since you do not have a packaging workflow in the repo that publishes to maven, this is not showing up on the dashboard.

that's correct. Currently it's hard to map a package name back to its repo, so we're only looking for workflows that build the package. In your case, we're looking for the use of actions/setup-java action or mvn [...] deploy commands. Is the a reasonable assumption?

I see! I was publishing releases manually from my PC. I now realize that the tool relies on the GitHub Action workflow config file to find/map packages. So, for the scorecard toolchain to detect the packages, the GitHub Action workflow would need to do the publishing, perhaps triggered by me pushing a release tag. I guess that makes sense, I can try that.

  • Signed Releases: Maven Central is very strict about signed releases. All of picocli's releases are available on Maven Central and have been signed (e.g. jar with signature jar.asc.sha256, same for the test jar, javadocs, sources, test-sources, and pom). How can I get these to show on the dashboard?

As per scorecards,
"This check looks for the following filenames in the project's last five releases: *.minisig, *.asc (pgp), *.sig, *.sign."
@remkop I see that you publish .jar files in releases, but not the .asc.sha256 file. Not sure if consumers will use the signed file, but also publishing the signature files in the release assets, will make it equivalent to what is published to maven central. @laurentsimon I observed the extension for signed file is .asc.sha256 in this case. So even if they were published, it would not get detected by the check...

Great, I've created a tracking issue on scorecard repo ossf/scorecard#1582. Can you confirm .asc.sha256 is the standard extension for maven's signed packages? Is it fair to expect maintainers to add this to their GitHub release? Please comment on the issue for additional information how we should implement this.

Ok will do.

@remkop
Copy link
Owner Author

@remkop remkop commented on 87ff7a6 Feb 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Looking at the result of all this on the openssf dashboard, there are a few things that stand out:
  • Summary does not show up (what can I do to fix that?)
  • Detected URLs does not point to the project's web page or GitHub page (what can I do to fix that?)
  • OpenSSF Scorecard section: last scan was Jan 8th; any idea what the refresh rate is of this dashboard?

the OSSF dashboard currently does not refresh using the results of the scorecard action analysis. We are working on it actively and will also launch a scorecard.dev website soon with that information. Thank you for your patience.

Understood. No worries! 😄

Can you let me know how I can impact the Summary and Detected URLs on the ossf dashboard? How is this retrieved? Is there some file I need to add, or some setting to add to a GitHub Action configuration?

@azeemshaikh38 @inferno-chromium @scovetta @david-a-wheeler

@laurentsimon - hope you don't mind me adding you to this thread. I do not know what is the source of this ossf dashboard. Where should these issues be reported?

  1. Branch Protection - the bar is too high
  • There is a lengthy explanation about Tiers (Tier 1-5), probably copied from the Scorecard Check documentation. This is misleading. I enabled Branch Protection on the main branch and configured everything to meet the requirements for Tier 1-3. I cannot currently meet Tier 4 as it "requires reviewers >= 2". However, I get zero credit for meeting Tier 1-3... So it appears that, at least for now, this is actually simply a lump sum: either you meet all requirements up to and including Tier 5 or you don't. Can the implementation be improved to receive partial credit for meeting some of these Tiers?
  • picocli is maintained by a single maintainer. The Branch Protection requirement seems to simply not cater for this model. The suggested remedy is "(...) try to recruit more maintainers to the project (...)". Alternatives are needed for single-maintainer projects to prevent such projects from being considered inherently unsafe.

Yes, I think this is fair feedback. Lot of open source projects are maintained by a single maintainer. @laurentsimon - FYI. Please let me know if want me to create an issue for this or may be this is already tracked?

Thank you @remkop for the feedback. You are receiving credits for enabling branch protection. I ran scorecard manually and your score for branch protection is 8, which is as good as it gets. We still provide the results to get to 10, but we understand those criteria cannot be met by everyone. You can ignore the remaining alerts for branch protection.

Please let me know if there's a problem with the messaging in the results that we could improve. I think the main issue is that the OSSF dashboard does not reflect the results. Is there another place we could improve?

I see. Thank you for checking that!
You are correct, I did not like having to Dismiss the branch protection alert as "false positive" in the end after making those changes. It felt that I had not accomplished anything, but with hindsight that is because I did not have the feedback that the branch protection score would show up as 8 on the ossf dashboard after the next refresh.

Yes, as you suggest, one idea to improve this is to show the current score in the alert messages, if that is feasible. 🤔

@laurentsimon
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's correct. Currently it's hard to map a package name back to its repo, so we're only looking for workflows that build the package. In your case, we're looking for the use of actions/setup-java action or mvn [...] deploy commands. Is the a reasonable assumption?

I see! I was publishing releases manually from my PC. I now realize that the tool relies on the GitHub Action workflow config file to find/map packages. So, for the scorecard toolchain to detect the packages, the GitHub Action workflow would need to do the publishing, perhaps triggered by me pushing a release tag. I guess that makes sense, I can try that.

awesome!

the OSSF dashboard currently does not refresh using the results of the scorecard action analysis. We are working on it actively and will also launch a scorecard.dev
website soon with that information. Thank you for your patience.

Understood. No worries! 😄

Can you let me know how I can impact the Summary and Detected URLs on the ossf dashboard? How is this retrieved? Is there some file I need to add, or some setting to > add to a GitHub Action configuration?

Unfortunately not. Once we have scorecard.dev website up and running, the data will automatically show up there. We're also preparing a badge system to display on your readme.

Yes, as you suggest, one idea to improve this is to show the current score in the alert messages, if that is feasible. 🤔

Can you create a tracking issue? This is do-able and we should have done it earlier. I think we can have this done in the next minor release. Thanks for the suggestion!

@laurentsimon
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've created the issue ossf/scorecard#1592 and sent a PR ossf/scorecard#1593 to address it.

Please sign in to comment.