-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Single daemonset vs one per deployment #5
Comments
Thinking about the numbers: Single daemonsetGiven the scenario where we have: Total deployment pods = 150 With a single DaemonSet for our pre-fetch we then use ✨ 31 ✨ pods per node Daemonset per deploymentGiven the same scenario as above but with a DaemonSet per deployment we'd end up with ✨ 80 ✨ pods per node ImplicationsWith the scenario above the latter option could end up having significant cost implications (due to using up more limited "resource") For example, if our scenario was deployed onto AWS EKS with all 5 nodes using But our second option would lead to pod exhaustion, preventing new pods from being scheduled. We can't solve this by increasing the number of nodes in our cluster as the number of deployments we're looking to prefetch (50) is already above the max pod limit of each node (35). To use this option we'd need to change instance type to one that supports more IPs, the cheapest that supports enough IPs is a I think I've talked myself out of this feature but would be good to know if there are any valid use cases that would benefit from suporting multiple daemonsets. |
Here's an idea. Make it configurable, then the operator decides? |
FWIW, I think that creating DaemonSets as a side effect of creating Deployments would cause operational complexity. I would never go with that unless I had a very specific need for it and for now I can not imagine a use case. |
Look at implementing an option to pick between:
The former (single DS) is "cleaner" as there's less noise when looking up resources but a single image failing to pull will mark the whole pod as having a problem.
The latter could potentially end up with many daemonsets and therefore many pods running on every node. This could lead to pod exhaustion (or in the case of AWS EKS, IP exhaustion).
From what I've been able to find out (but not confirm) there is no limit on the number of containers a pod can contain. The limit comes from the number of containers the node can suport as a whole. There is a limit to the number of pods though (usually 100 by default but that's not always the case - e.g. AWS EKS).
Tests using Kind have been able to succesfully create pods with at least
100300 containers.The text was updated successfully, but these errors were encountered: