-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Scheduler panic at evaluation time if there is a client with max_dynamic_port
== min_dynamic_port
#17585
Comments
Hi @stswidwinski! This is definitely a scheduler bug, as setting the min and max to be the same should result in there being exactly 1 port available. I'm working up a quick PR with the fix right now. |
If the dynamic port range for a node is set so that the min is equal to the max, there's only one port available and this passes config validation. But the scheduler panics when it tries to pick a random port. Only add the randomness when there's more than one to pick from. Adds a test for the behavior but also adjusts the commentary on a couple of the existing tests that made it seem like this case was already covered if you didn't look too closely. Fixes: #17585
How would one express lack of ports available? Would one make |
That would fail config validation. I think you could do this by setting a small range and then reserve those same ports via the |
If the dynamic port range for a node is set so that the min is equal to the max, there's only one port available and this passes config validation. But the scheduler panics when it tries to pick a random port. Only add the randomness when there's more than one to pick from. Adds a test for the behavior but also adjusts the commentary on a couple of the existing tests that made it seem like this case was already covered if you didn't look too closely. Fixes: #17585
Fixed in #17619 which will ship in the upcoming 1.6.0 beta (plus backports when those ship with the GA) |
Nomad version
Tip
Operating system and Environment details
Unix. Doesn't matter.
Issue
The scheduler panics during evaluation if there exists a client whose
max_dynamic_port
is the same asmin_dynamic_port
. The documentation does not specify whether these bounds are exclusive or inclusive, but the implementation of therand
function ingo
does.The answer is: "don't misconfigure your clients" (and I agree), but it would be nice if the client itself validated the configuration and crashes if the configuration were to produce unexpected result. A corollary of the above is that it seems hard to disable dynamic port allocation.
Reproduction steps
The cluster will start without any issues:
Now, start a job with dynamically allocates a port:
The result is a scheduler panic:
Expected Result
No scheduler panic.
Actual Result
Scheduler panic.
Job file (if appropriate)
See above.
The text was updated successfully, but these errors were encountered: