feat: allow configuration of retry attempts for Promotion steps

[hiddeco]

We have discovered some edge cases in which a Promotion step may end up in an infinite loop. To remediate this, we can configure the number of (retry) attempts a step is allowed to make before bailing.

This could later be extended to a waiting (or polling) configuration for steps like git-wait-for-pr, to e.g., set a deadline for merging a pull request.

With this pull request, any step execution (no matter the outcome) is counted as an "attempt", while the wait-for-pr and argocd-update steps have been allowed an unlimited number of retries (by default) to achieve backwards compatible behavior.

steps:
- uses: argocd-update
  retry:
    attempts: 5
  config:
    apps:
    - name: guestbook
       sources:
       - repoURL: https://github.com/example/guestbook.git

Note: In the future, we probably want to introduce more rigorous handling of errors, to be able to distinguish "fatal" errors from transient ones.

[netlify]

✅ Deploy Preview for docs-kargo-io ready!

Name	Link
🔨 Latest commit	029aff5
🔍 Latest deploy log	https://app.netlify.com/sites/docs-kargo-io/deploys/673f92233bf90800088f322e
😎 Deploy Preview	https://deploy-preview-2940.docs.kargo.io
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[codecov]

Codecov Report

Attention: Patch coverage is 82.81787% with 50 lines in your changes missing coverage. Please review.

Project coverage is 51.27%. Comparing base (40efb5c) to head (029aff5).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
internal/directives/simple_engine_promote.go	82.38%	27 Missing and 7 partials ⚠️
internal/controller/promotions/promotions.go	0.00%	4 Missing ⚠️
internal/directives/git_pr_opener.go	40.00%	3 Missing ⚠️
internal/directives/git_pr_waiter.go	25.00%	3 Missing ⚠️
internal/directives/simple_engine_health.go	95.58%	2 Missing and 1 partial ⚠️
internal/directives/argocd_updater.go	0.00%	2 Missing ⚠️
internal/directives/argocd_revisions.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2940      +/-   ##
==========================================
+ Coverage   51.04%   51.27%   +0.23%     
==========================================
  Files         280      282       +2     
  Lines       25380    25465      +85     
==========================================
+ Hits        12955    13057     +102     
+ Misses      11719    11707      -12     
+ Partials      706      701       -5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

---

🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests
• JS Bundle Analysis - Avoid shipping oversized bundles

[krancour]

Given #2972, I wonder if we should name this PromotionStepRetry or even PromotionStepRetryPolicy as I think we will probably also end up with Promotion-level retry policies in addition to step-level.

[krancour]

Non-blocking.

[hiddeco]

Renamed it to PromotionStepRetry to reserve the policy version for e.g. some enum that describes how it should be retried.

[krancour]

🔥 This is awesome!