-
Notifications
You must be signed in to change notification settings - Fork 474
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature: more robust way of logging errors in sub-checks #1327
Comments
relevant for shell parsing #1307 |
I do think we should simply log when we can't parse a file instead of failing the entire check, essentially do what #1312 does but for any parsing error. It would be nice if we could examine the files that can't be parsed in case we can improve our code to parse better, but inevitably we're going to encounter files that are formatted incorrectly. Maybe we can keep a list of files that we know are not parseable, and just ignore them. In regards to skipping template Dockerfiles, I think using the filename is a decent start. The file https://raw.githubusercontent.com/kubernetes/kubernetes/master/build/server-image/Dockerfile is not, strictly speaking, a template; it is a fully valid Dockerfile, just with build arguments. To handle a file like this would should enhance are logic to look for hashes as you suggest in #710 (comment), but we shouldn't skip it because it's a template. |
that'd be great, yes!
Do you know if developers commit their generated dockerfiles from a template? In the next iteration of the check, would is still be useful to try to parse the templates and just log a message of parsing fails (like you suggest above)? |
I think typically developers use a template dockerfile to build with different architectures or different distros as part of a pipeline. They might sometimes commit the dockerfiles generated from a template, but probably not usually.
This is the only one I'm aware of that caused errors for scorecard:
I think that would be a better approach. It looks like some of the dockerfiles that we're now skipping (this one for example) are able to be parsed just fine. |
do you know if the templating engine is part of thee official docker tooling or is it provided by a third-party?
|
It's third-party tooling. |
how about creating an |
Yeah, I think that's a good approach. |
Added to discussion for next sync. |
This issue is stale because it has been open for 60 days with no activity. |
Today, we return an inconclusive results (accompanied by an error) if a sub-check catches an error. For example, the Pinned-Dependencies check is currently fairly (too!) large, so a single error makes the entire check return
-1
.In #1324 (comment), we implemented a heuristic to avoid running the sub-check, using the filename. I feel this approach is fragile. Dockerfile templates are common, and are not always called
template
,tpl
, etc. See #710 and https://raw.githubusercontent.com/kubernetes/kubernetes/master/build/server-image/Dockerfile.It'd be useful to have a more robust solution. I'n not sure what a better solution is.
Maybe we should create an
Error()
function to log errors in sub-checks without affecting the entire check result.Any other ideas?
The text was updated successfully, but these errors were encountered: