Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improved status logging #742

Merged
merged 2 commits into from
Nov 7, 2024
Merged

improved status logging #742

merged 2 commits into from
Nov 7, 2024

Conversation

EronWright
Copy link
Contributor

@EronWright EronWright commented Nov 6, 2024

Proposed changes

Ensures that an info-level log line is emitted whenever the status is substantially changed on a given object. The resultant log is more useful for troubleshooting when we don't have access to the cluster or the status information within the Stack object.

For example, here's the series of status updates that occurs when a new stack is deployed.

2024-11-06T13:30:50-08:00	INFO	Status updated	{"controller": "stack-controller", "namespace": "default", "name": "random-yaml", "reconcileID": "47deadf8-cc74-4f30-92cf-a2781babd58b", "revision": "2119184", "observedGeneration": 0, "observedReconcileRequest": "", "lastUpdate": null, "currentUpdate": null, "conditions": [{"type":"Ready","status":"False","lastTransitionTime":"2024-11-06T21:30:50Z","reason":"NotReadyInProgress","message":"reconciliation is in progress"},{"type":"Reconciling","status":"True","lastTransitionTime":"2024-11-06T21:30:50Z","reason":"StackProcessing","message":"waiting for workspace readiness"}]}
2024-11-06T13:30:50-08:00	INFO	Status updated	{"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "ffe76e47-b703-4d07-bfb0-a50645113862", "revision": "2119189", "observedGeneration": 1, "address": "", "conditions": [{"type":"Ready","status":"False","observedGeneration":1,"lastTransitionTime":"2024-11-06T21:30:50Z","reason":"RollingUpdate","message":"Waiting for the statefulset to be updated"}]}
2024-11-06T13:30:50-08:00	INFO	Status updated	{"controller": "stack-controller", "namespace": "default", "name": "random-yaml", "reconcileID": "99af995b-303f-41a2-908d-d4427e8e0648", "revision": "2119198", "observedGeneration": 2, "observedReconcileRequest": "", "lastUpdate": null, "currentUpdate": null, "conditions": [{"type":"Ready","status":"False","lastTransitionTime":"2024-11-06T21:30:50Z","reason":"NotReadyInProgress","message":"reconciliation is in progress"},{"type":"Reconciling","status":"True","lastTransitionTime":"2024-11-06T21:30:50Z","reason":"StackProcessing","message":"waiting for workspace readiness"}]}
2024-11-06T13:30:54-08:00	INFO	Status updated	{"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "7dab6d15-1c5c-42fc-9bc7-d96dfbcffaf8", "revision": "2119226", "observedGeneration": 1, "address": "", "conditions": [{"type":"Ready","status":"False","observedGeneration":1,"lastTransitionTime":"2024-11-06T21:30:50Z","reason":"ConnectionFailed","message":"unable to connect to workspace: TRANSIENT_FAILURE"}]}
2024-11-06T13:30:59-08:00	INFO	Status updated	{"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "c12c6630-089f-4783-ab1f-856796961888", "revision": "2119239", "observedGeneration": 1, "address": "random-yaml-workspace.default.svc.cluster.local:50051", "conditions": [{"type":"Ready","status":"False","observedGeneration":1,"lastTransitionTime":"2024-11-06T21:30:50Z","reason":"Installing","message":"Installing packages and plugins required by the program"}]}
2024-11-06T13:31:01-08:00	INFO	Status updated	{"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "c12c6630-089f-4783-ab1f-856796961888", "revision": "2119244", "observedGeneration": 1, "address": "random-yaml-workspace.default.svc.cluster.local:50051", "conditions": [{"type":"Ready","status":"False","observedGeneration":1,"lastTransitionTime":"2024-11-06T21:30:50Z","reason":"Initializing","message":"Initializing and selecting a Pulumi stack"}]}
2024-11-06T13:31:04-08:00	INFO	Status updated	{"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "c12c6630-089f-4783-ab1f-856796961888", "revision": "2119253", "observedGeneration": 1, "address": "random-yaml-workspace.default.svc.cluster.local:50051", "conditions": [{"type":"Ready","status":"True","observedGeneration":1,"lastTransitionTime":"2024-11-06T21:30:50Z","reason":"Succeeded","message":""}]}
2024-11-06T13:31:04-08:00	INFO	Status updated	{"controller": "stack-controller", "namespace": "default", "name": "random-yaml", "reconcileID": "da8dcf8a-09cd-4705-a8a4-0f61c5fba359", "revision": "2119255", "observedGeneration": 2, "observedReconcileRequest": "", "lastUpdate": null, "currentUpdate": {"generation":2,"name":"random-yaml-193036360e1","commit":"sha256:3227b4cdbdb3492b7bdff13f9acbfe6e679eee55753b48f0467e2ab7e9342350"}, "conditions": [{"type":"Ready","status":"False","lastTransitionTime":"2024-11-06T21:30:50Z","reason":"NotReadyInProgress","message":"reconciliation is in progress"},{"type":"Reconciling","status":"True","lastTransitionTime":"2024-11-06T21:30:50Z","reason":"StackProcessing","message":"stack is being processed"}]}
2024-11-06T13:31:04-08:00	INFO	Status updated	{"controller": "update-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Update", "Update": {"name":"random-yaml-193036360e1","namespace":"default"}, "namespace": "default", "name": "random-yaml-193036360e1", "reconcileID": "e1fe3034-b49a-4937-ab79-181d5b2c798a", "revision": "2119256", "revision": "2119257", "observedGeneration": 1, "message": "", "conditions": [{"type":"Progressing","status":"True","observedGeneration":1,"lastTransitionTime":"2024-11-06T21:31:04Z","reason":"Progressing","message":""},{"type":"Failed","status":"False","observedGeneration":1,"lastTransitionTime":"2024-11-06T21:31:04Z","reason":"Progressing","message":""},{"type":"Complete","status":"False","observedGeneration":1,"lastTransitionTime":"2024-11-06T21:31:04Z","reason":"Progressing","message":""}]}
2024-11-06T13:31:12-08:00	INFO	Status updated	{"controller": "update-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Update", "Update": {"name":"random-yaml-193036360e1","namespace":"default"}, "namespace": "default", "name": "random-yaml-193036360e1", "reconcileID": "e1fe3034-b49a-4937-ab79-181d5b2c798a", "revision": "2119256", "revision": "2119275", "observedGeneration": 1, "message": "\"Stack Update (up)\"", "conditions": [{"type":"Progressing","status":"False","observedGeneration":1,"lastTransitionTime":"2024-11-06T21:31:12Z","reason":"Complete","message":""},{"type":"Failed","status":"False","observedGeneration":1,"lastTransitionTime":"2024-11-06T21:31:04Z","reason":"succeeded","message":"\"Stack Update (up)\""},{"type":"Complete","status":"True","observedGeneration":1,"lastTransitionTime":"2024-11-06T21:31:12Z","reason":"Updated","message":""}]}
2024-11-06T13:31:13-08:00	INFO	Status updated	{"controller": "stack-controller", "namespace": "default", "name": "random-yaml", "reconcileID": "225be101-a526-48b2-be77-01748cb31c4d", "revision": "2119277", "observedGeneration": 2, "observedReconcileRequest": "", "lastUpdate": {"generation":2,"name":"random-yaml-193036360e1","type":"up","state":"succeeded","lastAttemptedCommit":"sha256:3227b4cdbdb3492b7bdff13f9acbfe6e679eee55753b48f0467e2ab7e9342350","lastSuccessfulCommit":"sha256:3227b4cdbdb3492b7bdff13f9acbfe6e679eee55753b48f0467e2ab7e9342350","permalink":"https://app.pulumi.com/eron-pulumi-corp/random/dev/updates/4900","lastResyncTime":"2024-11-06T21:31:12Z","failures":0}, "currentUpdate": null, "conditions": [{"type":"Ready","status":"True","lastTransitionTime":"2024-11-06T21:31:12Z","reason":"ProcessingCompleted","message":"the stack has been processed and is up to date"}]}

Related issues (optional)

Closes #732
Closes #354

@EronWright EronWright requested a review from blampe November 6, 2024 21:32
Copy link

codecov bot commented Nov 6, 2024

Codecov Report

Attention: Patch coverage is 92.85714% with 2 lines in your changes missing coverage. Please review.

Project coverage is 50.47%. Comparing base (0b5b8e4) to head (cbe21e8).
Report is 3 commits behind head on master.

Files with missing lines Patch % Lines
...ator/internal/controller/auto/update_controller.go 80.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #742      +/-   ##
==========================================
+ Coverage   50.28%   50.47%   +0.19%     
==========================================
  Files          30       30              
  Lines        4091     4115      +24     
==========================================
+ Hits         2057     2077      +20     
- Misses       1850     1854       +4     
  Partials      184      184              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@EronWright EronWright merged commit ddd4ac4 into master Nov 7, 2024
6 checks passed
@EronWright EronWright deleted the issue-732 branch November 7, 2024 00:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Report status changes to the operator log Overhaul logging
2 participants