Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
enhancement: External validations for Operator SDK #98
enhancement: External validations for Operator SDK #98
Changes from all commits
290ea24
279a347
cc99c9e
2817e7e
5e1fe8b
57054b2
ef2e918
18067b3
793fa25
aab11a8
a6e0751
ae285c3
d7f7225
File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
By generating the bundle via SDK we need to check it to ensure that the spec is correct.
Also, all checks under
operator-sdk bundle validate ./bundle --select-optional suite=operatorframework
are valid checks for any Operator using the OF solutions which would like to be distributed via OLM.In this way, all tests shipped under operator-framework/api must still be provided via SDK by default.
However, we can allow users to consume the operator-framework/api bin sooner as well.
I think we have the following uses cases that we still need to comply with:
make bundle
to create/update it for my project so that, I can ensure that the spec generated/updated by the tool using my configuration is valid.operator-sdk bundle validate
to check bundles that were not built using the default SDK scaffold.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@camilamacedo86 I still think there is value for the compiled in checks. Though we may have to think of a better way to update those.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are we going to expose this as a go type that validator implementers will use?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ryantking yes, we are going to expose it. There is a link to it in the EP: https://github.com/operator-framework/api/blob/master/pkg/validation/errors/error.go#L9-L16
That's already accessible from SDK.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let me know if I need to call that out better.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently, we do not have many external validators. However, in the future, if it grows we might face complexities such as:
To test your project against the criteria to publish in the index X:
operator-sdk bundle validate ./bundle --select-optional suite=operatorframework
and then against A,B,C validators as well.So, shows that would be very nice if we could just copy and paste a config and run just 1 command that would check the bundle against the common criteria + all external validators.
See that we download OPM CLI and use it (https://github.com/operator-framework/operator-sdk/blob/master/testdata/go/v3/memcached-operator/Makefile#L183-L198). Then, if we could make SDK download and use all external validators configured then, we maybe make it easier / possible SDK only uses this config with a bin provided by operator-framework/api instead of importing it to execute the checks. That could address the main motivations of this PE. (avoid SDK releases only to ship new versions of the of/api bundle validators implementation)
Then, if a user requires a new version of the default checks (of/api) we could only update the config for its upper version as we do in the case of OPM in the makefile target.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This config is going down the route of "discoverability".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is a repeat and I don't think it needs to be added as a con to the JavaScript or CUE alternative. These would be equally fast.