-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allocations should not overlap in execution #10440
Comments
Hi @prestonp ! Thanks for reporting this issue and the detailed notes. I agree that this behavior isn't ideal. Looking at the code, it doesn't seem like a simple change I can do quickly - Will raise it with the team and consider it for our backlog. |
Upon looking into it again, I opened a draft PR in #10446 . As it touches the scheduler logic, we will need to examine and test it more rigorously. If you can give it a try, we'd appreciate your feedback or experience. |
Thank you @notnoop, with my repro the new evaluation fails with a clear message, this is great!
|
I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues. |
Nomad version
Nomad v1.1.0-dev (19d58ce)
Operating system and Environment details
Linux devbox 5.8.0-23-generic #24~20.04.1-Ubuntu SMP Sat Oct 10 04:57:02 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Issue
I have a service deployed on nomad that seems to occasionally run into
bind: address already in use
. I am using a static port so this suggests that there's multiple tasks running concurrently.My expectation is that
nomad stop
would mean that tasks have fully exited, so that subsequentnomad run
s don't contend with shared system resources like ports.The service I'm working with handles graceful shutdown so sometimes there's a bit of a delay until it fully exits. Regardless, it seems that
nomad stop
doesn't block on tasks exiting and replacement tasks can be scheduled right away.Reproduction steps
Submit job
Confirm functionality
Then restart
Expected Result
I expected the second allocation to start after the first one exits.
Actual Result
Both allocations overlap in execution. The second alloc is started immediately despite the first allocation is still shutting down.
Checking the second allocation reveals a conflict:
Job
Simple Go http server that handles graceful but slow shutdown:
Build it
Set up job.hcl
The text was updated successfully, but these errors were encountered: