You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
When running Pinned-Dependencies check on apache/arrow or apache/beam, it's returning a -1 and reporting an Internal Error. It's not the same error on both repository, but both errors are breaking the execution and disabling users to view any actual result that could have been achieved.
See above the errors in details:
diogoteles:~$ scorecard --repo=https://github.com/apache/beam --show-details --format=json --checks=Pinned-Dependencies | jq .
Error: check runtime error: Pinned-Dependencies: internal error: internal error: unable to determine OS for job: Build python wheels on ${{matrix.arch}} for ${{ matrix.os_python.os }}
2023/07/26 10:47:12 error during command execution: check runtime error: Pinned-Dependencies: internal error: internal error: unable to determine OS for job: Build python wheels on ${{matrix.arch}} for ${{ matrix.os_python.os }}
{
"date": "2023-07-26T10:47:05-03:00",
"repo": {
"name": "github.com/apache/beam",
"commit": "907c5110163b0efe52e9e12127fd013c7fc455d7"
},
"scorecard": {
"version": "4.11.0",
"commit": "4edb07802fdad892fa8d10f8fd47666b6ccc27c9"
},
"score": -1,
"checks": [
{
"details": null,
"score": -1,
"reason": "internal error: internal error: unable to determine OS for job: Build python wheels on ${{matrix.arch}} for ${{ matrix.os_python.os }}",
"name": "Pinned-Dependencies",
"documentation": {
"url": "https://github.com/ossf/scorecard/blob/4edb07802fdad892fa8d10f8fd47666b6ccc27c9/docs/checks.md#pinned-dependencies",
"short": "Determines if the project has declared and pinned the dependencies of its build process."
}
}
],
"metadata": null
}
diogoteles:~$ scorecard --repo=https://github.com/apache/arrow --show-details --format=json --checks=Pinned-Dependencies | jq .
Error: check runtime error: Pinned-Dependencies: internal error: error parsing shell code: ci/docker/python-wheel-windows-test-vs2017.dockerfile:1:2: "if <cond>" must be followed by "then"
2023/07/26 10:51:40 error during command execution: check runtime error: Pinned-Dependencies: internal error: error parsing shell code: ci/docker/python-wheel-windows-test-vs2017.dockerfile:1:2: "if <cond>" must be followed by "then"
{
"date": "2023-07-26T10:51:35-03:00",
"repo": {
"name": "github.com/apache/arrow",
"commit": "8503c869fa80090bc849bae0b4e68a84605ea82e"
},
"scorecard": {
"version": "4.11.0",
"commit": "4edb07802fdad892fa8d10f8fd47666b6ccc27c9"
},
"score": -1,
"checks": [
{
"details": null,
"score": -1,
"reason": "internal error: error parsing shell code: ci/docker/python-wheel-windows-test-vs2017.dockerfile:1:2: \"if <cond>\" must be followed by \"then\"",
"name": "Pinned-Dependencies",
"documentation": {
"url": "https://github.com/ossf/scorecard/blob/4edb07802fdad892fa8d10f8fd47666b6ccc27c9/docs/checks.md#pinned-dependencies",
"short": "Determines if the project has declared and pinned the dependencies of its build process."
}
}
],
"metadata": null
}
Reproduction steps
Steps to reproduce the behavior:
Run Scorecard (I've reproduced it on version 4.11.0 and 4.10.2) with Pinned-Dependencies check for apache/arrow or apache/beam
Expected behavior
At first, we should consider enhancing the error handling to avoid not showing any results to the user other than the internal error. I'd say that ideally the internal error should not break the whole execution of the check; the output should report the error(and probably keep reporting a -1), but should also contain any results that could be achieved before and after the internal error.
Additionally, we should work to avoid those specific internal errors, of course.
The text was updated successfully, but these errors were encountered:
We also get a runtime error when Scorecard runs this check on argoproj/argo-workflows. Mentioned in CNCF Slack for the current Security Slam.
It throws an error while trying to parse our Dockerfile.windows: internal error: error parsing shell code: Dockerfile.windows:1:5: (( can only be used to open an arithmetic cmd.
As far as I can tell, that seems to be because the shell parser is assuming Linux shell, but that code is actually Windows PowerShell.
#3515 successfully warns on that error and continues, outputting relevant information, even in the same file.
(and probably keep reporting a -1)
I think it should normalize an overall score, not just report a -1 if a single line gets a runtime error. Especially since the error is within Scorecard, and not within the project it's checking. Running #3515 on Argo Workflows gets a 4, which makes a lot more sense to me than a -1.
Describe the bug
When running Pinned-Dependencies check on apache/arrow or apache/beam, it's returning a -1 and reporting an Internal Error. It's not the same error on both repository, but both errors are breaking the execution and disabling users to view any actual result that could have been achieved.
See above the errors in details:
Reproduction steps
Steps to reproduce the behavior:
Expected behavior
At first, we should consider enhancing the error handling to avoid not showing any results to the user other than the internal error. I'd say that ideally the internal error should not break the whole execution of the check; the output should report the error(and probably keep reporting a -1), but should also contain any results that could be achieved before and after the internal error.
Additionally, we should work to avoid those specific internal errors, of course.
The text was updated successfully, but these errors were encountered: