Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix multiple overflow errors in exponential backoff #18200

Merged
merged 8 commits into from
Aug 15, 2023
Merged

Commits on Aug 14, 2023

  1. fix multiple overflow errors in exponential backoff

    We use capped exponential backoff in several places in the code when handling
    failures. The code we've copy-and-pasted all over has a check to see if the
    backoff is greater than the limit, but this check happens after the bitshift and
    we always increment the number of attempts. This causes an overflow with a
    fairly small number of failures (ex. at one place I tested it occurs after only
    24 iterations), resulting in a negative backoff which then never recovers. The
    backoff becomes a tight loop consuming resources and/or DoS'ing a Nomad RPC
    handler or an external API such as Vault. Note this doesn't occur in places
    where we cap the number of iterations so the loop breaks (usually to return an
    error), so long as the number of iterations is reasonable.
    
    Introduce a check on the cap before the bitshift to avoid overflow in all places
    this can occur.
    
    Fixes: #18199
    tgross committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    c326e0a View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    676d15a View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2023

  1. Configuration menu
    Copy the full SHA
    379a3ac View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    bc7f360 View commit details
    Browse the repository at this point in the history
  3. remove extraneous assignment

    tgross committed Aug 15, 2023
    Configuration menu
    Copy the full SHA
    04dcb90 View commit details
    Browse the repository at this point in the history
  4. missed a helper

    tgross committed Aug 15, 2023
    Configuration menu
    Copy the full SHA
    dd00966 View commit details
    Browse the repository at this point in the history
  5. fix ineffectual assignment

    tgross committed Aug 15, 2023
    Configuration menu
    Copy the full SHA
    1f92874 View commit details
    Browse the repository at this point in the history
  6. fixes for docker driver

    tgross committed Aug 15, 2023
    Configuration menu
    Copy the full SHA
    dd46498 View commit details
    Browse the repository at this point in the history