From 152274d544d742c120149f023c33b0898be06538 Mon Sep 17 00:00:00 2001 From: carlory Date: Thu, 15 Jun 2023 03:05:35 +0800 Subject: [PATCH] update spec doc for groupcontroller service --- spec.md | 151 ++++++++++++++++++++++++++++++++------------------------ 1 file changed, 87 insertions(+), 64 deletions(-) diff --git a/spec.md b/spec.md index f16e8a47..635aaed4 100644 --- a/spec.md +++ b/spec.md @@ -80,85 +80,97 @@ A CO SHOULD be equipped to handle both centralized and headless plugins, as well Several of these possibilities are illustrated in the following figures. ``` - CO "Master" Host -+-------------------------------------------+ -| | -| +------------+ +------------+ | -| | CO | gRPC | Controller | | -| | +-----------> Plugin | | -| +------------+ +------------+ | -| | -+-------------------------------------------+ - - CO "Node" Host(s) -+-------------------------------------------+ -| | -| +------------+ +------------+ | -| | CO | gRPC | Node | | -| | +-----------> Plugin | | -| +------------+ +------------+ | -| | -+-------------------------------------------+ + CO "Master" Host ++------------------------------------------------+ +| | +| +------------+ +-----------------+ | +| | CO | gRPC | Controller | | +| | +--+--------> Plugin | | +| +------------+ | +-----------------+ | +| | | +| | +-----------------+ | +| | | GroupController | | +| +--------> Plugin | | +| +-----------------+ | +| | ++------------------------------------------------+ + + CO "Node" Host(s) ++------------------------------------------------+ +| | +| +------------+ +-----------------+ | +| | CO | gRPC | Node | | +| | +-----------> Plugin | | +| +------------+ +-----------------+ | +| | ++------------------------------------------------+ Figure 1: The Plugin runs on all nodes in the cluster: a centralized -Controller Plugin is available on the CO master host and the Node -Plugin is available on all of the CO Nodes. +Controller Plugin and GroupController Plugin (Optional) are available +on the CO master host and the Node Plugin is available on all of the +CO Nodes. ``` ``` - CO "Node" Host(s) -+-------------------------------------------+ -| | -| +------------+ +------------+ | -| | CO | gRPC | Controller | | -| | +--+--------> Plugin | | -| +------------+ | +------------+ | -| | | -| | | -| | +------------+ | -| | | Node | | -| +--------> Plugin | | -| +------------+ | -| | -+-------------------------------------------+ + CO "Node" Host(s) ++------------------------------------------------+ +| | +| +-----------------+ | +| | Node | | +| +--------> Plugin | | +| | +-----------------+ | +| | | +| +------------+ | +-----------------+ | +| | CO | |gRPC | Controller | | +| | +-----------> Plugin | | +| +------------+ | +-----------------+ | +| | | +| | +-----------------+ | +| | | GroupController | | +| +--------> Plugin | | +| +-----------------+ | +| | ++------------------------------------------------+ Figure 2: Headless Plugin deployment, only the CO Node hosts run -Plugins. Separate, split-component Plugins supply the Controller -Service and the Node Service respectively. +Plugins. Separate, split-component Plugins supply the Node Service, +the Controller Service and the GroupController Service (Optional) +respectively. ``` ``` - CO "Node" Host(s) -+-------------------------------------------+ -| | -| +------------+ +------------+ | -| | CO | gRPC | Controller | | -| | +-----------> Node | | -| +------------+ | Plugin | | -| +------------+ | -| | -+-------------------------------------------+ + CO "Node" Host(s) + +------------------------------------------------+ + | | + | +-----------------+ | + | +------------+ | Node | | + | | CO | gRPC | Controller | | + | | +-----------> GroupController | | + | +------------+ | Plugin | | + | +-----------------+ | + | | + +------------------------------------------------+ Figure 3: Headless Plugin deployment, only the CO Node hosts run -Plugins. A unified Plugin component supplies both the Controller -Service and Node Service. +Plugins. A unified Plugin component supplies both the Node Service, +the Controller Service and the GroupController Service (Optional). ``` ``` - CO "Node" Host(s) -+-------------------------------------------+ -| | -| +------------+ +------------+ | -| | CO | gRPC | Node | | -| | +-----------> Plugin | | -| +------------+ +------------+ | -| | -+-------------------------------------------+ + CO "Node" Host(s) ++------------------------------------------------+ +| | +| +------------+ +-----------------+ | +| | CO | gRPC | Node | | +| | +-----------> Plugin | | +| +------------+ +-----------------+ | +| | ++------------------------------------------------+ Figure 4: Headless Plugin deployment, only the CO Node hosts run Plugins. A Node-only Plugin component supplies only the Node Service. -Its GetPluginCapabilities RPC does not report the CONTROLLER_SERVICE -capability. +Its GetPluginCapabilities RPC does not report both the CONTROLLER_SERVICE +capability and the GROUP_CONTROLLER_SERVICE capability. ``` ### Volume Lifecycle @@ -268,7 +280,13 @@ Each SP MUST provide: * **Node Plugin**: A gRPC endpoint serving CSI RPCs that MUST be run on the Node whereupon an SP-provisioned volume will be published. * **Controller Plugin**: A gRPC endpoint serving CSI RPCs that MAY be run anywhere. -* In some circumstances a single gRPC endpoint MAY serve all CSI RPCs (see Figure 3 in [Architecture](#architecture)). + +Each SP MAY provide: + +- **GroupController Plugin**: A gRPC endpoint serving CSI RPCs that MAY be run anywhere. + + +In some circumstances a single gRPC endpoint MAY serve all CSI RPCs (see Figure 3 in [Architecture](#architecture)). ```protobuf syntax = "proto3"; @@ -322,10 +340,11 @@ extend google.protobuf.ServiceOptions { } ``` -There are three sets of RPCs: +There are four sets of RPCs: -* **Identity Service**: Both the Node Plugin and the Controller Plugin MUST implement this sets of RPCs. +* **Identity Service**: Both the Node Plugin, the Controller Plugin and the GroupController Plugin MUST implement this sets of RPCs. * **Controller Service**: The Controller Plugin MUST implement this sets of RPCs. +* **GroupController Service**: The GroupController Plugin MUST implement this sets of RPCs. * **Node Service**: The Node Plugin MUST implement this sets of RPCs. ```protobuf @@ -3066,6 +3085,10 @@ The CO MUST implement the specified error recovery behavior when it encounters t | Snapshot list mismatch | 3 INVALID_ARGUMENT | Besides the general cases, this code SHOULD also be used to indicate when plugin supporting CREATE_DELETE_GET_VOLUME_GROUP_SNAPSHOT detects a mismatch in the `snapshot_ids`. | If a mismatch is detected in the `snapshot_ids`, caller SHOULD use different `snapshot_ids`. | | Volume group snapshot does not exist | 5 NOT_FOUND | Indicates that a volume group snapshot corresponding to the specified `group_snapshot_id` does not exist. | Caller MUST verify that the `group_snapshot_id` is correct and that the volume group snapshot is accessible and has not been deleted before retrying with exponential back off. | +#### RPC Interactions + +TBD + ## Protocol ### Connectivity