Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Ability to MergePatch CRDs properly #488

Merged
merged 5 commits into from
Jul 9, 2019
Merged

Conversation

fabianbaier
Copy link
Member

What type of PR is this?

Uncomment only one /kind <> line, hit enter to put that in a new line, and remove leading whitespaces from that line:

/component operator
/kind bug

What this PR does / why we need it:

In #447 we've added StrategicMergePatchs, however custom resources apparently don't work currently with SMPs. With this patch, CustomResources can behave properly without throwing:

PlanExecutionController: CreateOrUpdate Patch: the body of the request was in an unknown format - accepted media types include: application/json-patch+json, application/merge-patch+json

It works by being more verbose and catching the error of a failed StrategicMergePatch and applying an old MergePatch before throwing an error. When testing this worked with Zookeeper, Kafka and Flink.

Without this, a more complex "extension", e.g. like in the Flink demo in which we require instantiation of e.g. other Operators first wasn't working. Having for example https://github.com/kudobuilder/operators/blob/master/repository/flink/docs/demo/financial-fraud/demo-operator/templates/zookeeper.yaml as an Instance to be applied in a Phase would throw the above error.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

First, this needs further testing. In particular for the scenario like in the flink financial fraud demo.
To try it out run in the kudobuilder/operators repo: kudo install flink/docs/demo/financial-fraud/demo-operator

The expected output should be:

NAME                                          READY   STATUS        RESTARTS   AGE
flink-demo-qx2x66-zk-flink-demo-qx2x66-zk-0   0/1     Pending       0          2d21h
flink-demo-qx2x66-zk-flink-demo-qx2x66-zk-0   0/1     Pending       0          2d21h
flink-demo-qx2x66-zk-flink-demo-qx2x66-zk-1   0/1     Pending       0          2d21h
flink-demo-qx2x66-zk-flink-demo-qx2x66-zk-1   0/1     Pending       0          2d21h
flink-demo-qx2x66-zk-flink-demo-qx2x66-zk-2   0/1     Pending       0          2d21h
flink-demo-qx2x66-zk-flink-demo-qx2x66-zk-2   0/1     Pending       0          2d21h
flink-demo-qx2x66-zk-flink-demo-qx2x66-zk-0   0/1     Pending       0          2d21h
flink-demo-qx2x66-zk-flink-demo-qx2x66-zk-1   0/1     Pending       0          2d21h
flink-demo-qx2x66-zk-flink-demo-qx2x66-zk-0   0/1     ContainerCreating   0          2d21h
flink-demo-qx2x66-zk-flink-demo-qx2x66-zk-1   0/1     ContainerCreating   0          2d21h
flink-demo-qx2x66-zk-flink-demo-qx2x66-zk-0   0/1     Running             0          2d21h
flink-demo-qx2x66-zk-flink-demo-qx2x66-zk-1   0/1     Running             0          2d21h
flink-demo-qx2x66-zk-flink-demo-qx2x66-zk-2   0/1     Pending             0          2d21h
flink-demo-qx2x66-zk-flink-demo-qx2x66-zk-2   0/1     ContainerCreating   0          2d21h
flink-demo-qx2x66-zk-flink-demo-qx2x66-zk-2   0/1     Running             0          2d21h
flink-demo-qx2x66-zk-flink-demo-qx2x66-zk-0   1/1     Running             0          2d21h
flink-demo-qx2x66-zk-flink-demo-qx2x66-zk-1   1/1     Running             0          2d21h
flink-demo-qx2x66-zk-flink-demo-qx2x66-zk-2   1/1     Running             0          2d21h
flink-demo-jzhct2-zk-flink-demo-jzhct2-zk-0   0/1     Pending             0          2d21h
flink-demo-jzhct2-zk-flink-demo-jzhct2-zk-0   0/1     Pending             0          2d21h
flink-demo-jzhct2-zk-flink-demo-jzhct2-zk-1   0/1     Pending             0          2d21h
flink-demo-jzhct2-zk-flink-demo-jzhct2-zk-1   0/1     Pending             0          2d21h
flink-demo-jzhct2-zk-flink-demo-jzhct2-zk-2   0/1     Pending             0          2d21h
flink-demo-jzhct2-zk-flink-demo-jzhct2-zk-2   0/1     Pending             0          2d21h
flink-demo-jzhct2-zk-flink-demo-jzhct2-zk-2   0/1     Pending             0          2d21h
flink-demo-jzhct2-zk-flink-demo-jzhct2-zk-2   0/1     ContainerCreating   0          2d21h
flink-demo-jzhct2-zk-flink-demo-jzhct2-zk-2   0/1     Running             0          2d21h
flink-demo-jzhct2-zk-flink-demo-jzhct2-zk-1   0/1     Pending             0          2d21h
flink-demo-jzhct2-zk-flink-demo-jzhct2-zk-0   0/1     Pending             0          2d21h
flink-demo-jzhct2-zk-flink-demo-jzhct2-zk-1   0/1     ContainerCreating   0          2d21h
flink-demo-jzhct2-zk-flink-demo-jzhct2-zk-0   0/1     ContainerCreating   0          2d21h
flink-demo-jzhct2-zk-flink-demo-jzhct2-zk-0   0/1     Running             0          2d21h
flink-demo-jzhct2-zk-flink-demo-jzhct2-zk-1   0/1     Running             0          2d21h
flink-demo-jzhct2-zk-flink-demo-jzhct2-zk-1   1/1     Running             0          2d21h
flink-demo-jzhct2-zk-flink-demo-jzhct2-zk-0   1/1     Running             0          2d21h
flink-demo-jzhct2-zk-flink-demo-jzhct2-zk-2   1/1     Running             0          2d21h
flink-demo-4mqrmq-zk-flink-demo-4mqrmq-zk-0   0/1     Pending             0          2d21h
flink-demo-4mqrmq-zk-flink-demo-4mqrmq-zk-0   0/1     Pending             0          2d21h
flink-demo-4mqrmq-zk-flink-demo-4mqrmq-zk-1   0/1     Pending             0          2d21h
flink-demo-4mqrmq-zk-flink-demo-4mqrmq-zk-1   0/1     Pending             0          2d21h
flink-demo-4mqrmq-zk-flink-demo-4mqrmq-zk-2   0/1     Pending             0          2d21h
flink-demo-4mqrmq-zk-flink-demo-4mqrmq-zk-2   0/1     Pending             0          2d21h
flink-demo-4mqrmq-zk-flink-demo-4mqrmq-zk-1   0/1     Pending             0          2d21h
flink-demo-4mqrmq-zk-flink-demo-4mqrmq-zk-0   0/1     Pending             0          2d21h
flink-demo-4mqrmq-zk-flink-demo-4mqrmq-zk-1   0/1     ContainerCreating   0          2d21h
flink-demo-4mqrmq-zk-flink-demo-4mqrmq-zk-0   0/1     ContainerCreating   0          2d21h
flink-demo-4mqrmq-zk-flink-demo-4mqrmq-zk-0   0/1     Running             0          2d21h
flink-demo-4mqrmq-zk-flink-demo-4mqrmq-zk-2   0/1     Pending             0          2d21h
flink-demo-4mqrmq-zk-flink-demo-4mqrmq-zk-2   0/1     ContainerCreating   0          2d21h
flink-demo-4mqrmq-zk-flink-demo-4mqrmq-zk-1   0/1     Running             0          2d21h
flink-demo-4mqrmq-zk-flink-demo-4mqrmq-zk-2   0/1     Running             0          2d21h
flink-demo-4mqrmq-zk-flink-demo-4mqrmq-zk-0   1/1     Running             0          2d21h
flink-demo-4mqrmq-zk-flink-demo-4mqrmq-zk-1   1/1     Running             0          2d21h
flink-demo-4mqrmq-zk-flink-demo-4mqrmq-zk-2   1/1     Running             0          2d21h
flink-demo-4mqrmq-kafka-kafka-0               0/1     Pending             0          2d21h
flink-demo-4mqrmq-kafka-kafka-0               0/1     Pending             0          2d21h
flink-demo-4mqrmq-kafka-kafka-0               0/1     Pending             0          2d21h
flink-demo-4mqrmq-kafka-kafka-0               0/1     ContainerCreating   0          2d21h
flink-demo-4mqrmq-kafka-kafka-0               0/1     Running             0          2d21h
flink-demo-4mqrmq-kafka-kafka-0               1/1     Running             0          2d21h
flink-demo-4mqrmq-kafka-kafka-1               0/1     Pending             0          2d21h
flink-demo-4mqrmq-kafka-kafka-1               0/1     Pending             0          2d21h
flink-demo-4mqrmq-kafka-kafka-1               0/1     Pending             0          2d21h
flink-demo-4mqrmq-kafka-kafka-1               0/1     ContainerCreating   0          2d21h
flink-demo-4mqrmq-kafka-kafka-1               0/1     Running             0          2d21h
flink-demo-4mqrmq-kafka-kafka-1               1/1     Running             0          2d21h
flink-demo-4mqrmq-kafka-kafka-2               0/1     Pending             0          2d21h
flink-demo-4mqrmq-kafka-kafka-2               0/1     Pending             0          2d21h
flink-demo-4mqrmq-kafka-kafka-2               0/1     Pending             0          2d21h
flink-demo-4mqrmq-kafka-kafka-2               0/1     ContainerCreating   0          2d21h
flink-demo-4mqrmq-kafka-kafka-2               0/1     Running             0          2d21h
flink-demo-4mqrmq-kafka-kafka-2               1/1     Running             0          2d21h
flink-demo-4mqrmq-flink-jobmanager-0          0/1     Pending             0          2d21h
flink-demo-4mqrmq-flink-jobmanager-0          0/1     Pending             0          2d21h
flink-demo-4mqrmq-flink-taskmanager-7b5b955d59-9dq45   0/1     Pending             0          2d21h
flink-demo-4mqrmq-flink-taskmanager-7b5b955d59-9dq45   0/1     Pending             0          2d21h
flink-demo-4mqrmq-flink-taskmanager-7b5b955d59-47d97   0/1     Pending             0          2d21h
flink-demo-4mqrmq-flink-taskmanager-7b5b955d59-47d97   0/1     Pending             0          2d21h
flink-demo-4mqrmq-flink-jobmanager-0                   0/1     Pending             0          2d21h
flink-demo-4mqrmq-flink-taskmanager-7b5b955d59-9dq45   0/1     ContainerCreating   0          2d21h
flink-demo-4mqrmq-flink-taskmanager-7b5b955d59-47d97   0/1     ContainerCreating   0          2d21h
flink-demo-4mqrmq-flink-jobmanager-0                   0/1     ContainerCreating   0          2d21h
flink-demo-4mqrmq-flink-taskmanager-7b5b955d59-9dq45   1/1     Running             0          2d21h
flink-demo-4mqrmq-flink-jobmanager-0                   1/1     Running             0          2d21h
flink-demo-4mqrmq-generator-7dddcf978f-t8nnb           0/1     Pending             0          2d21h
flink-demo-4mqrmq-generator-7dddcf978f-t8nnb           0/1     Pending             0          2d21h
flink-demo-4mqrmq-actor-54656b546-qw6z5                0/1     Pending             0          2d21h
flink-demo-4mqrmq-actor-54656b546-qw6z5                0/1     Pending             0          2d21h
flink-demo-4mqrmq-generator-7dddcf978f-t8nnb           0/1     ContainerCreating   0          2d21h
flink-demo-4mqrmq-flink-taskmanager-7b5b955d59-47d97   1/1     Running             0          2d21h
flink-demo-4mqrmq-actor-54656b546-qw6z5                0/1     ContainerCreating   0          2d21h
flink-demo-4mqrmq-generator-7dddcf978f-t8nnb           1/1     Running             0          2d21h
flink-demo-4mqrmq-actor-54656b546-qw6z5                1/1     Running             0          2d21h

Does this PR introduce a user-facing change?:


@gerred
Copy link
Member

gerred commented Jul 2, 2019

I need to think about this one and look at available methods/APIs. This is a pretty sledgehammery solution. Are there instances that AREN'T CRDs where a StrategicMergePatch would fail but a MergePatch would work, and cause unintended consequences? Can we mark this entire set of code as immediately deprecated pending removal once Server Side Apply is in?

@gerred
Copy link
Member

gerred commented Jul 2, 2019

Another solution we discussed that might be more viable - take the union of the object being applied and the server side object and apply the object's view of that as a merge patch, so that only the fields that the applying object are concerned with are applied. This solves the same semantic while being applicable to any JSON.

@oliviabarrick
Copy link
Contributor

Also we should add a test for the issue that this fixes.

Copy link
Member

@kensipe kensipe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we have a test for this?

@gerred
Copy link
Member

gerred commented Jul 2, 2019

@kensipe Probably. If so, this could be extracted as a function that returns a ConstantPatch for reconcile to use. This would be a great small step! That way, we're not trying to test this massive reconcile function.

@fabianbaier
Copy link
Member Author

@kensipe Probably. If so, this could be extracted as a function that returns a ConstantPatch for reconcile to use. This would be a great small step! That way, we're not trying to test this massive reconcile function.

We have now a test :)

We ended up discussing more around those lines here https://kubernetes.slack.com/archives/CG3HTFCMV/p1562610928382300 and decided to not extract it into a function.

@@ -499,7 +499,12 @@ func (r *ReconcilePlanExecution) Reconcile(request reconcile.Request) (reconcile
log.Printf("Error getting patch between truth and obj: %v\n", err)
} else {
err = r.Client.Patch(context.TODO(), truth, client.ConstantPatch(types.StrategicMergePatchType, rawObj))
log.Printf("PlanExecutionController: CreateOrUpdate Patch: %v", err)
if err != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we check the specific error here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 agreed

//
// Reason: "UnsupportedMediaType" Code: 415
switch e := err.(type) {
case *errors.StatusError:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. If you want, Kubernetes has some helper functions you can use for this: https://godoc.org/k8s.io/apimachinery/pkg/api/errors#IsUnsupportedMediaType

if errors.IsUnsuportedMediaType(err) {}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. If you want, Kubernetes has some helper functions you can use for this: https://godoc.org/k8s.io/apimachinery/pkg/api/errors#IsUnsupportedMediaType

if errors.IsUnsuportedMediaType(err) {}

Ah nice! Didn't know about this helper function!

@fabianbaier fabianbaier merged commit 8f5f345 into master Jul 9, 2019
@fabianbaier fabianbaier deleted the fb/smp-patch branch July 9, 2019 19:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants