-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature request: detailed allocation GC docs and logging of allocation state before GC #3421
Comments
Good timing! I'm working on the GC as we speak and the logging will be improved.
|
Great to hear! Better logging and more explicit documentation on how GC works will be very helpful. No manual call to |
#3445 includes some logging improvements to the GC but does not directly address the items you list. |
Hey @schmichael I'm pretty curious about this too. I'm trying to make Nomad stop being so aggressive about cleaning up old allocations, but not having much luck. I've tried setting
As for the debug logs, I'm not sure what I'm looking for, but I don't see anything that tells me why it decided to force this collection.
|
Hi @holtwilkins, Nomad servers periodically GC dead jobs and force GC their allocations on their client nodes. You can configure that interval on servers with the |
Cool, I'll try bumping that one too. I guess even more reason it would be nice to have better GC logging in Nomad :). |
I recently had some allocations fail and later be replaced with new ones. Judging by the client logs below, allocations were GC'd less than 2h after allocation failure. I can't tell by the logs when the Nomad servers garbage collected the allocations.
By the time I noticed that the allocation had failed, any debugging information was long gone. When googling for documentation about Nomad allocation GC, I have only been able to find the GC API call documentation and various github comments, so I've had to piece together an understanding of how alloc GC works.
Two things I would like to see:
Let me know what you think!
The text was updated successfully, but these errors were encountered: