networking: Inject implicit constraints on CNI plugins when using bridge mode #15473
Conversation
shoenig
force-pushed
the
f-cni-plugins-constraints
branch
from
057a868
to
b64914f
Compare
|
Looks good to me. |
There was a problem hiding this comment.
Neat! It could be helpful to have a comment next to nomadCNIConfigTemplate with a reminder to add or remove these implicit constraints if the template changes.
This also makes me thing about mixed version clusters 🤔
If you happen to have clients with different nomadCNIConfigTemplate, these constraints may not necessarily be true since they are applied in the servers. But that's a very edge case.
It does make me wonder if we could generate these constraints dynamically from nomadCNIConfigTemplate 😅
|
Ooh the upgrade path, yeah that gets interesting. When we have added implicit constraints in the past it was for stuff like Consul version, where clients had long already been fingerprinted, or native service discovery, which required new clients anyway. This is a case where we need to maintain compatibility with old clients not yet being fingerprinted. Not sure we can merge this until like 1.5.x, or if we backport #15473 first and wait a while. |
|
Kinda missed the boat on 1.7; pushing to 1.8 |
…dge mode This PR adds a job mutator which injects constraints on the job taskgroups that make use of bridge networking. Creating a bridge network makes use of the CNI plugins: bridge, firewall, host-local, loopback, and portmap. Starting with Nomad 1.5 these plugins are fingerprinted on each node, and as such we can ensure jobs are correctly scheduled only on nodes where they are available, when needed.
tgross
force-pushed
the
f-cni-plugins-constraints
branch
from
b64914f
to
0be5eab
Compare
tgross
force-pushed
the
f-cni-plugins-constraints
branch
from
0be5eab
to
f33980f
Compare
|
I've rebased this PR on main, added handling of the early exit we introduced for NUMA, and added an upgrade note. |
There was a problem hiding this comment.
LGTM! Maybe worth adding a note to the bridge config that these constraints may need to be updated if the bridge spec is modified.
nomad/client/allocrunner/networking_bridge_linux.go
Lines 150 to 152 in c3e7b13
Done! |
…dge mode (#15473) This PR adds a job mutator which injects constraints on the job taskgroups that make use of bridge networking. Creating a bridge network makes use of the CNI plugins: bridge, firewall, host-local, loopback, and portmap. Starting with Nomad 1.5 these plugins are fingerprinted on each node, and as such we can ensure jobs are correctly scheduled only on nodes where they are available, when needed.
This PR adds a job mutator which injects constraints on the job taskgroups
that make use of bridge networking. Creating a bridge network makes use of the
CNI plugins: bridge, firewall, host-local, loopback, and portmap. Starting
with Nomad 1.5 these plugins are fingerprinted on each node, and as such we
can ensure jobs are correctly scheduled only on nodes where they are available,
when needed.
Targets v1.5.0
Closes #14022