-
Notifications
You must be signed in to change notification settings - Fork 474
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BUG: Parsing errors #839
Comments
@chrismcgehee are you interested in helping for this issue? It's in the Pinned-Dependency check that you're looking at to add lines. |
Sure, I'll take this one. |
There are several different categories these parsing errors fall into. I'll list them and state how I plan to address them:
Let me know if you have any suggestions. |
Thanks Chris, this is great! Comments inline:
How would we check for this? If we have parsers that can understand this, that can work. But if we are planning to use string matching to do this, I'm not too sure about this one.
All of this sound good to me. |
This is a pretty simple check, so I think string matching would be appropriate here. I was unable to find any open-source parsers for github workflows. |
workflows also have a
you should not need to parse the file if it's a shell, because it will be done by
this is already done in
|
Laurent, thanks for the comments.
Is there an example yml file you can point to that shows the |
a few places:
|
Ok, maybe if all the values in |
sounds like a reasonable approach. Also verify that if |
Found one more. Might be similar to previous ones I found.
|
One more from #954
|
The error from https://github.com/crawl/crawl is because there is a python file in
The error from https://github.com/sourcegraph/sourcegraph is actually failing on the CODENOTIFY file. The same fix should handle this one also. @nathan-415, fyi. |
@chrismcgehee Can we close this? |
Not yet. There's still one more issue I need to address: shell code that isn't able to be parsed. |
I found another one |
Ok, there are 4 repos that give us
That leads me to a broader question of how we handle shell code that we are unable to parse. Currently, the entire check fails with a score of '?'. What I would like to do instead is log a warning whenever we fail to parse a file, but continue on with the check. It would be good if we put a system in place to monitor for these warnings so we can evaluate if there is a bug in our parsing code. How does this sound? |
+1. See https://github.com/ossf/scorecard/blob/main/stats/views.go#L50. You can use that to record/monitor these parse errors. We can start with this to get a sense of how bad the problem is and then decide on whether we should log a warning and continue. |
@chrismcgehee thanks for fixing so many of the parsing issues. The number of parsing errors we see in the cron job has significantly reduced! Some new parsing issues I see as of now:
|
Monitoring data @chrismcgehee put in place for shell errors. We seem to consistently have some shell related failures when running the cron job. Although the number is not that high which is a good thing. But we clearly seem to fail parsing shell code on a significant number of repos. |
Another parsing issue reported by @asraa in #1163:
|
@asraa, can you provide an example yaml file (or a repo) where this would fail? I looked through https://securitylab.github.com/research/github-actions-preventing-pwn-requests/ and didn't see any yaml file that looked like it had |
Ooops! I failed to update this. I found two things
The YAML linked in that blog will return github internal error for parsing workflow for the Token-Permissions check scorecard/checks/permissions.go Line 211 in 53ae583
|
@chrismcgehee FYI, code is
|
On a similar note why not use this library for parsing https://github.com/rhysd/actionlint #989 ? |
Only briefly looked at https://pkg.go.dev/github.com/rhysd/actionlint#Parse. Sounds like a good idea to me @naveensrinivasan . @chrismcgehee wdyt? |
@azeemshaikh38 Thanks for fixing the panicky code. |
@chrismcgehee Do you want to do this? Or I can take it. |
I can do this one. |
Thanks |
Parsing errors that showed up with the new actionlint parser:
|
Noticing many parsing errors during cron job. Not blocking cron job, but good to fix since otherwise we just return
ErrScorecardInternal
. Some examples to reproduce shown below:There might be more cases. Will add as I find them. Would be good to fix these and add these repos to
cron/data/projects.release.csv
as and when we fix them.The text was updated successfully, but these errors were encountered: