Change logs support to track all changes being made to managed resources #754

jbw976 · 2024-08-03T15:45:47Z

Description of your changes

This PR contains the crossplane-runtime changes to support cloud change logs.

Issue: Proposal: Cloud Change Log crossplane#5477
Design: design: cloud change logs crossplane#5822

Further PRs for changes to the providers themselves (to consume these runtime changes) and for the change logs sidecar container are coming soon, but should not block this PR from being merged. Previews of those changes can be found in:

Testing

I have prepared some testing instructions at https://github.com/jbw976/change-log-sidecar/blob/main/README.md that reviewers can try out fairly easily on their local laptops.

Here is some example output that demonstrates what a subset of fields of a change log entry looks like, for example when an object is created, updated, then deleted:

❯ kubectl -n crossplane-system logs -l pkg.crossplane.io/provider=provider-kubernetes -c change-log-sidecar | jq '.timestamp + " " + .provider + " " + .name + " " + .operation'
"2024-08-09T04:48:43Z provider-kubernetes:v0.0.6 object-0 OPERATION_TYPE_CREATE"
"2024-08-09T04:49:22Z provider-kubernetes:v0.0.6 object-0 OPERATION_TYPE_UPDATE"
"2024-08-09T04:50:32Z provider-kubernetes:v0.0.6 object-0 OPERATION_TYPE_DELETE"

A full change log entry looks like the below:

{"apiVersion":"kubernetes.crossplane.io/v1alpha2","externalName":"object-0","kind":"Object","name":"object-0","operation":"OPERATION_TYPE_CREATE","provider":"provider-kubernetes:v0.0.6","snapshot":{"apiVersion":"kubernetes.crossplane.io/v1alpha2","kind":"Object","metadata":{"annotations":{"crossplane.io/composition-resource-name":"resource-object-0","crossplane.io/external-name":"object-0"},"creationTimestamp":"2024-08-09T04:48:43Z","generateName":"traceperf-tester-wnw9g-","generation":2,"labels":{"crossplane.io/claim-name":"traceperf-tester","crossplane.io/claim-namespace":"default","crossplane.io/composite":"traceperf-tester-wnw9g"},"managedFields":[{"apiVersion":"kubernetes.crossplane.io/v1alpha2","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{"f:crossplane.io/composition-resource-name":{}},"f:generateName":{},"f:labels":{"f:crossplane.io/claim-name":{},"f:crossplane.io/claim-namespace":{},"f:crossplane.io/composite":{}},"f:ownerReferences":{"k:{\"uid\":\"c21ef3e7-adb8-4b1f-9b10-b7fbf33bf623\"}":{}}},"f:spec":{"f:forProvider":{"f:manifest":{"f:apiVersion":{},"f:data":{".":{},"f:key":{}},"f:kind":{},"f:metadata":{"f:namespace":{}}}}}},"manager":"apiextensions.crossplane.io/composed/010d7ea543f4d932601bc4472c6b11a93716eecdfa070bfdcea16def1d7efd74","operation":"Apply","time":"2024-08-09T04:48:43Z"},{"apiVersion":"kubernetes.crossplane.io/v1alpha2","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{"f:crossplane.io/external-name":{}}},"f:spec":{"f:readiness":{".":{},"f:policy":{}}}},"manager":"crossplane-kubernetes-provider","operation":"Update","time":"2024-08-09T04:48:43Z"}],"name":"object-0","ownerReferences":[{"apiVersion":"trace-perf.crossplane.io/v1alpha1","blockOwnerDeletion":true,"controller":true,"kind":"XTracePerf","name":"traceperf-tester-wnw9g","uid":"c21ef3e7-adb8-4b1f-9b10-b7fbf33bf623"}],"resourceVersion":"998","uid":"905af6bb-eb03-4569-9fe1-b257fa8019e9"},"spec":{"deletionPolicy":"Delete","forProvider":{"manifest":{"apiVersion":"v1","data":{"key":"value-0-tester-foo"},"kind":"ConfigMap","metadata":{"namespace":"default"}}},"managementPolicies":["*"],"providerConfigRef":{"name":"default"},"readiness":{"policy":"SuccessfulCreate"}},"status":{"atProvider":{"manifest":null}}},"timestamp":"2024-08-09T04:48:43Z"}

I have:

Read and followed Crossplane's contribution process.
Run earthly +reviewable to ensure this PR is ready for review.
Added or updated unit tests.
Linked a PR or a docs tracking issue to document this change.
~~Added backport release-x.y labels to auto-backport this PR.~~

Need help with this checklist? See the cheat sheet.

apis/proto/v1alpha1/changelog.proto