Completed Pod not getting cleaned after failed sync or success

[mozarik]

Environment

• Airbyte version: 0.40.4
• OS Version / Instance: Ubuntu 18.04
• Deployment: Kubernetes kube/overlays/stable
• Step where error happened: Sync Data -> Complete or Error happened when sync

Current Behavior

Is the pod not getting cleaned after it use ?

kubectl get pod --field-selector=status.phase==Succeeded

NAME                                    READY   STATUS      RESTARTS   AGE
orchestrator-norm-job-11-attempt-1      0/1     Completed   0          111m
source-github-check-11-2-eopmb          0/4     Completed   0          109m
destination-bigquery-check-11-2-lztdk   0/4     Completed   0          109m
orchestrator-norm-job-11-attempt-2      0/1     Completed   0          38m

Expected Behavior

This is what I asked. is this expected behaviour ?

Logs

Steps to Reproduce

1. Sync Data
2. After sync failed or Success

Are you willing to submit a PR?

I dont know the code base yet

[sh4sh]

There's a service called pod-sweeper that should clean up these pods after 2 hours, so hopefully by now they are gone. There is explanation on why that limit was chosen here: #4737 (comment)

If you want to adjust how long succeeded pods take to clean up, you can edit this line of the deployment yaml: https://github.com/airbytehq/airbyte/blob/v0.40.4/kube/resources/pod-sweeper.yaml#L23

[sh4sh]

Hey @mozarik is this issue resolved or is there an issue with pod-sweeper?

[mozarik]

Hey @mozarik is this issue resolved or is there an issue with pod-sweeper?

Sorry its resolved for now. Thank you