Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix deadlock caused by syncing during destroy #2852

Merged
merged 1 commit into from
Jul 18, 2017
Merged

Fix deadlock caused by syncing during destroy #2852

merged 1 commit into from
Jul 18, 2017

Commits on Jul 18, 2017

  1. Fix deadlock caused by syncing during destroy

    When replacing an alloc the new alloc is blocked until the old alloc is
    destroyed. This could cause a deadlock:
    
    1. Destroying the old alloc includes a final sync of its status
    2. Syncing status causes a GC
    3. A GC looks for terminal allocs to cleanup
    4. The GC waits for an alloc to stop completely before GC'ing
    
    If the GC chooses the currently-being-destroyed-alloc to GC, the GC
    deadlocks. If `client.max_parallel` deadlocks happen the GC is wedged
    until the Nomad process is restarted.
    
    Performing the final sync asynchronously is an ugly hack but prevents
    the deadlock by allowing the final sync to occur after the alloc runner
    has shutdown and been destroyed.
    schmichael committed Jul 18, 2017
    Configuration menu
    Copy the full SHA
    8c4b760 View commit details
    Browse the repository at this point in the history