Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding startingDeadlineSeconds to all cli native cronjob templates #2831

Merged
merged 2 commits into from
Dec 20, 2021

Conversation

cdchris12
Copy link
Contributor

Checklist

  • [] Affected Issues have been mentioned in the Closing issues section
  • [] Documentation has been written/updated
  • PR title is ready for changelog and subsystem label(s) applied

This PR adds the startingDeadlineSeconds value to both native cronjob templates for cli cron tasks. This prevents stale jobs from preventing the cronjob from running altogether.

Closing issues

@cdchris12 cdchris12 added 2-build-deploy Build & Deploy subsystem 0-kubernetes Vanilla kubernetes support labels Sep 8, 2021
@cdchris12 cdchris12 requested a review from Schnitzel September 8, 2021 14:24
@cdchris12 cdchris12 self-assigned this Sep 8, 2021
@tobybellwood tobybellwood added this to the v2.x milestone Oct 13, 2021
@tobybellwood tobybellwood modified the milestones: v2.x, v2.4.0 Nov 22, 2021
@tobybellwood
Copy link
Member

As I understand it, this will only retry the cronjob for 240 seconds after it's scheduled time. As cronjob runs start every ~10s, this will mean 24 failures, which should result in a "missed schedule" error (and not the exceeded retries error) - per https://stackoverflow.com/questions/51065538/what-does-kubernetes-cronjobs-startingdeadlineseconds-exactly-mean

As we have a forbid concurrency policy in place, this will cause cronjobs to try to schedule for this 4 minute period, but the retries will have finished and the job failed by the time the next schedule comes around (as long as it's over 5 minutes)

End users may be able to use the fail messages to trigger some sort of alert on long-running crons

@tobybellwood tobybellwood merged commit a10c7c5 into main Dec 20, 2021
@tobybellwood tobybellwood deleted the native_cronjob_startingDeadlineSeconds_fix branch December 20, 2021 01:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0-kubernetes Vanilla kubernetes support 2-build-deploy Build & Deploy subsystem
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants