Proposal: drop-in directories

[squeed]

As mentioned in #927, the the reality is that network configuration is not managed top-down by an administrator in real-world clusters. Rather, it is maintained by one or more third parties, all of which may need to react according to the existing configuration and/or modify it.

This makes composability awkward, to say the least.

An extremely common use-case is wanting to insert a chained plugin in to an existing configuration. That existing configuration is managed by yet another third (fourth?) party. Currently, this can only be done by reading, editing, and re-writing the configuration file. This is fragile, since the file is nominally "owned" by one network provider, but edited by another.

This proposal is for CNI dropin directories, which libcni would read from to scan for additional chained plugins.

Example

Imagine there is a configuration file, 10-awesome.conflist with the following contents:

{
    "cniVersion": "1.0",
    "name": "my-awesome-network",
    "plugins": [
        { "type": "awesome" },
        { "type": "chained-1" }
    ]
}

Then, libcni would look for a folder named my-awesome-network. It would parse any .conf files contained in them. Imagine it finds 11-chained.conf with the following contents:

{
    "type": "chained-2"
}

This would cause libcni to insert "chained-2" to the end of list of chained plugins when parsing 10-awesome.conflist.

[bleggett]

Currently, this can only be done by reading, editing, and re-writing the configuration file. This is fragile, since the file is nominally "owned" by one network provider, but edited by another.

Yes, and if there are multiple node agents installing plugins that are mutually unaware of each other, they essentially are contending over the same node-level config file, which can and does lead to unmitigatable TOCTOU bugs which render the entire node unusable.

It seems simplest and safest to effectively mimic Unix init system design here, which encountered and solved the same problem years ago using the mechanism proposed by @squeed

We can't have a plugin infrastructure that relies on multiple noncooperating processes mutating a single, critical node-level config file with no locking.

[squeed]

I really like the idea of drop-in directories, but this isn't going to happen for v1.1

[bleggett]

Took a stab at a quick spec update to add this in #1052 (looking ahead to 1.2+) - mostly to clarify what a spec-based approach would look like.