-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: migrate from kuttl to chainsaw #786
Conversation
Signed-off-by: Charles-Edouard Brétéché <charles.edouard@nirmata.com>
BTW this file is not valid tempo-operator/tests/e2e-openshift/red-metrics/03-assert.yaml Lines 52 to 57 in 46c037b
|
hi @eddycharly thanks for the PR. Is there any statement in kuttl project about not being activelly maintained? chainsaw seems to be only 3-4 old project (looking at releases) which concerns me. Is there any evidence of adoption by operaotr-sdk/kubebuilder? |
Hi @pavolloffay Chainsaw was created because kuttl stopped to be actively maintained (it's actually a man community now). We are also solving most of the kuttl limitations that exist today and wrote a command to automatically migrate from kuttl to chainsaw. We presented chainsaw to kubebuilder/operator-sdk folks last week and are discussing how to collaborate:
I agree this is a relatively young project but it has already been adopted by a couple of e2e tests heavy projects (kyverno, keptn, fairwinds, opentelementry-operator is the next one, and there is a PR for grafana-operator grafana/grafana-operator#1406). This is a solid kuttl replacement, i can change the PR to run both tools side by side if you prefer but maintaining two identical test suites is a pain. |
Could this be a drop-in replacement? I.e. we can just replace the |
Unfortunately no, cli args are not exactly the same and chainsaw doesn't support It would be nice to let the workflows run and look at the result, happy to discuss it more if i have a chance to convince you :) Here is the recording of the last KB meeting where we presented chainsaw if you want to see it in action https://www.youtube.com/watch?v=Ejof-wtAdQM |
Grafana-operator adopted chainsaw today grafana/grafana-operator#1406 🎉 ❤️ |
Signed-off-by: Charles-Edouard Brétéché <charles.edouard@nirmata.com>
@pavolloffay @andreasgerstmayr did you have a chance to look at this ? |
@pavolloffay I think we met today during the opentelemetry-operator SIG meeting. I hope you loved chainsaw :) |
LGTM I am fine with merging this. @andreasgerstmayr WDYT? |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #786 +/- ##
=======================================
Coverage 75.31% 75.31%
=======================================
Files 89 89
Lines 6369 6369
=======================================
Hits 4797 4797
Misses 1342 1342
Partials 230 230
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
- command: | ||
args: | ||
- -c | ||
- '! kubectl get --namespace $NAMESPACE configmap tempo-simplest -o jsonpath=''{.data.tempo\.yaml}'' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does the var $NAMESPACE work here same as KUTTL ? I need to check how the Chainsaw tests work when run from inside a pod as the OpenShift CI uses Prow CI to run the jobs and the tests are run from inside a pod.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes $NAMESPACE
works the same as in KUTTL.
Most scripts can be replaced with asserts/apply operations in chainsaw, i will look deeper at your tests once this PR merges to remove scripts when possible.
I'll fix the failing tests soon, thanks for triggering the workflow. |
Signed-off-by: Charles-Edouard Brétéché <charles.edouard@nirmata.com>
@pavolloffay @IshwarKanse i pushed a fix 🤞 |
@IshwarKanse please ping me on the PR, there's something i don't understand fully here. |
We released v0.1.6 today, fixing an important issue related to concurrency, shall i update this PR ? |
@eddycharly yeah sure. Please update the version. Thanks. |
Signed-off-by: Charles-Edouard Brétéché <charles.edouard@nirmata.com>
@IshwarKanse done |
I'm fine with this change. |
I took the time to remove scripts from upgrade tests, converting - script:
content: kubectl get --namespace $NAMESPACE tempo simplest -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}' | grep True to an assertion tree equivalent - assert:
resource:
apiVersion: tempo.grafana.com/v1alpha1
kind: TempoStack
metadata:
name: simplest
status:
(length(conditions[?(type == 'Ready' && status == 'True')]) == 0): false |
This would work too, maybe better than using - assert:
resource:
apiVersion: tempo.grafana.com/v1alpha1
kind: TempoStack
metadata:
name: simplest
status:
"(contains(conditions[*].{type: type, status: status}, {type: 'Ready', status: 'True'}))": true |
b4c1412
to
cf237df
Compare
@eddycharly Can you resolve this so that we can merge this PR. Auto-merge is enabled.
Once merged I'll retest on a OpenShift cluster. |
Head branch was pushed to by a user without write access
@IshwarKanse can you retry ?
I thought it was not going to be merged until a meeting happens 🤷 |
Hey folks, i didn't like the solution above and needed something similar while working on something else, there's a better solution: - assert:
resource:
apiVersion: tempo.grafana.com/v1alpha1
kind: TempoStack
metadata:
name: simplest
status:
(conditions[?type == 'Ready']):
- status: 'True' |
If one can trigger the workflow 🙏 What are the next steps ? Any idea when this could potentially land in ? 🤞 |
@eddycharly We discussed it internally and decided to merge. cc @frzifus @andreasgerstmayr |
Thanks folks! |
BTW if something breaks or you need help with anything feel free to ping me on slack. |
Migrate from kuttl to chainsaw.
Chainsaw is similar to kuttl but a lot more flexible and actively maintained.
If you need more infos 👇