fix(upgrade): increase wait for deployment rollout status

[shubham14bajpai]

Signed-off-by: shubham shubham.bajpai@mayadata.io

Why is this PR required? What issue does it fix?:
This PR fixes the wait time for deployment to roll out after it is patched. Now the job will wait indefinitely for the deployment pod to come up once it is patched successfully. This PR fixes the issue openebs/openebs#3062

Does this PR require any upgrade changes?:
No

Manual test performed
Patched kubelet code to take 10 minutes for podSync.
Upgraded 1.8.0 to 1.11.0.
Job logs:

mayadata:upgrade$ k logs -f cstor-spc-1901100-8p72j
I0619 06:04:59.587201       1 cstor_spc.go:100] Started upgrading storagePoolClaim{sparse-claim} from 1.8.0 to 1.11.0
I0619 06:05:00.008563       1 helper.go:68] Patching deployment sparse-claim-j8jr
I0619 06:05:00.105448       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because deployment rollout in-progress: waiting for deployment spec update
I0619 06:05:10.122578       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:05:20.176773       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:05:30.198933       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:05:40.203717       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:05:50.207980       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:06:00.216758       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:06:10.226104       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:06:20.292787       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:06:30.328104       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:06:40.347484       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:06:50.351850       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:07:00.362315       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:07:10.372810       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:07:20.376606       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:07:30.385903       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:07:40.393821       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:07:50.402888       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:08:00.415207       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:08:10.424903       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:08:20.436386       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:08:30.445366       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:08:40.614508       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:08:50.622763       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:09:00.638711       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:09:10.646600       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:09:20.649335       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:09:30.700749       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:09:40.713632       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:09:50.726609       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:10:00.736274       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:10:10.746041       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:10:20.755769       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:10:30.763715       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:10:40.786255       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:10:50.795278       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:11:00.811728       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:11:10.823915       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:11:20.829340       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:11:30.839809       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:11:40.849994       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:11:50.861952       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:12:00.872854       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:12:10.881843       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:12:20.890242       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:12:30.912276       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:12:40.919687       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:12:50.929801       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:13:01.151707       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:13:11.155445       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 new replicas were updated
I0619 06:13:21.168289       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:13:31.181574       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:13:41.192666       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:13:51.195108       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:14:01.613765       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:14:11.622980       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:14:22.174044       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:14:32.197887       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:14:42.208489       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:14:52.254772       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:15:02.274310       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:15:12.285317       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:15:22.287639       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:15:32.299994       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:15:42.302223       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:15:52.313797       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:16:02.339443       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:16:12.351051       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:16:22.358659       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:16:32.369588       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:16:42.380975       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:16:52.394305       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:17:02.411640       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:17:12.424724       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:17:22.435591       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:17:32.446553       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:17:42.459887       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:17:52.473628       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:18:02.487836       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:18:12.491117       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:18:22.504014       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:18:32.558311       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:18:42.570388       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:18:52.573824       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:19:02.585836       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:19:12.592676       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:19:23.075131       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:19:33.078640       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:19:43.084000       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:19:53.093385       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:20:03.108877       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:20:13.162572       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:20:23.180112       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:20:33.190127       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:20:43.196308       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:20:53.199721       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:21:03.212657       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:21:13.220932       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:21:23.230864       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:21:33.243161       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:21:43.291302       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:21:53.332945       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:22:03.341308       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:22:13.351750       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:22:23.360210       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:22:33.399092       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:22:43.407778       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:22:53.419868       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:23:03.434135       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:23:13.444657       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:23:23.454160       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:23:33.468309       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:23:43.476911       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:23:53.490741       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:24:03.501122       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:24:13.511690       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because deployment exceeded its progress deadline
I0619 06:24:23.523402       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because deployment exceeded its progress deadline
I0619 06:24:33.532254       1 helper.go:86] Deployment sparse-claim-j8jr not yet rolled out because replica update in-progress: 0 of 1 updated replicas are available
I0619 06:24:43.541745       1 helper.go:89] Deployment sparse-claim-j8jr rolled out successfully
I0619 06:24:43.541814       1 csp_upgrade.go:191] patched csp deployment sparse-claim-j8jr
I0619 06:24:43.562732       1 csp_upgrade.go:153] patched csp sparse-claim-j8jr
I0619 06:24:43.562790       1 csp_upgrade.go:277] Verifying the reconciliation of version for sparse-claim-j8jr { phase:Healthy }
I0619 06:24:53.593584       1 csp_upgrade.go:277] Verifying the reconciliation of version for sparse-claim-j8jr { phase:Healthy }
I0619 06:25:03.638453       1 csp_upgrade.go:380] Upgrade Successful for csp sparse-claim-j8jr
I0619 06:25:03.794877       1 spc_upgrade.go:148] Verifying the reconciliation of version for sparse-claim
I0619 06:25:13.807301       1 spc_upgrade.go:102] Upgrade Successful for spc sparse-claim
I0619 06:25:13.807373       1 cstor_spc.go:120] Successfully upgraded storagePoolClaim{sparse-claim} from 1.8.0 to 1.11.0

Any additional information for your reviewer? :
Mention if this PR is part of any design or a continuation of previous PRs

Checklist:

• Fixes #
• PR Title follows the convention of <type>(<scope>): <subject>
• Has the change log section been updated?
• Commit has unit tests
• Commit has integration tests
• (Optional) Are upgrade changes included in this PR? If not, mention the issue/PR to track:
• (Optional) If documentation changes are required, which issue on https://github.com/openebs/openebs-docs is used to track them: