[multikueue] KEP add subcomponents and integration details.

keps/693-multikueue/README.md

@@ -11,7 +11,13 @@
     - [Story 2](#story-2)
   - [Risks and Mitigations](#risks-and-mitigations)
 - [Design Details](#design-details)
-    - [Follow ups ideas](#follow-ups-ideas)
+  - [Subcomponents](#subcomponents)
+    - [MultiKueueCluster Controller](#multikueuecluster-controller)
+    - [AdmissionCheck Controller](#admissioncheck-controller)
+    - [Workload Controller](#workload-controller)
+    - [Garbage Collector](#garbage-collector)
+  - [Jobs abstraction](#jobs-abstraction)
+  - [Follow ups ideas](#follow-ups-ideas)
   - [Test Plan](#test-plan)
     - [Unit Tests](#unit-tests)
     - [Integration tests](#integration-tests)
@@ -188,16 +194,28 @@ type MultiKueueClusterStatus {
    Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"`
 }
 ```
+### Subcomponents
 
-MultiKueue controller will monitor all cluster definitions and maintain 
+#### MultiKueueCluster Controller
+
+Will monitor all cluster definitions and maintain 
 the Kube clients for all of them. Any connectivity problems will be reported both in
-MultiKueueCluster status as well as AdmissionCheckStatus and Events. MultiKueue controller 
+MultiKueueCluster status and Events. MultiKueue controller 
 will make sure that whenever the kubeconfig is refreshed, the appropriate 
-clients will also be recreated.
+clients will also be recreated and attempt to reconnect when the connection to the
+target cluster is lost.
 
 Creation of kubeconfig files is outside of the MultiKueue scope, and is cloud
 provider/environment dependant.
 
+#### AdmissionCheck Controller
+
+Will monitor the AdmissionChecks associated with MultiKueue (by ControllerName) and maintain
+their `Active` status condition based on the validity of their MultiKueueConfig and `Active`
+state of the MultiKueueClusters in use.
+
+#### Workload Controller
+
 MultiKueue controller, when pushing workloads to the worker clusters, will use the same 
 namespace and local queue names as were used in the management cluster. It is user's 
 responsibility to set up the appropriate namespaces and local queues.
@@ -223,7 +241,59 @@ If there is enough of global quota, the unknown admitted workloads would be re-a
 the management cluster. If not, some workloads will be preempted to meet the global quota.
 In case of duplicates, all but one of them will be removed.
 
-#### Follow ups ideas
+#### Garbage Collector
+
+Will monitor the Workloads in the remote clusters to detect and remove
+those that ware created by MultiKueue but were not deleted by the Workload Controller 
+in the normal flow, due to a temporary connectivity outage to the remote cluster.
+
+### Jobs abstraction
+
+In order to work with different types of jobs the MultiKueue Controller will use two abstraction
+interfaces:
+
+```go
+// MultiKueueAdapter interface needed for MultiKueue job delegation.
+type MultiKueueAdapter interface {
+	// Creates the Job object in the worker cluster using remote client, if not already created.
+	// Copy the status from the remote job if already exists.
+	SyncJob(ctx context.Context, localClient client.Client, remoteClient client.Client, key types.NamespacedName, workloadName, origin string) error
+	// Deletes the Job in the worker cluster.
+	DeleteRemoteObject(ctx context.Context, remoteClient client.Client, key types.NamespacedName) error
+	// IsJobManagedByKueue returns:
+	// - a bool indicating if the job object identified by key is managed by kueue and can be delegated.
+	// - a reason indicating why the job is not managed by Kueue
+	// - any API error encountered during the check
+	IsJobManagedByKueue(ctx context.Context, localClient client.Client, key types.NamespacedName, controllerName string) (bool, string, error)
+	// KeepAdmissionCheckPending returns true if the state of the multikueue admission check should be
+	// kept Pending while the job runs in a worker. This might be needed to keep the managers job
+	// suspended and not start the execution locally.
+	KeepAdmissionCheckPending() bool
+	// GVK returns GVK (Group Version Kind) for the job.
+	GVK() schema.GroupVersionKind
+}
+```
+Used by the [Workload Controller](#workload-controller) to interact with the owner of the reconciled workloads.
+
+
+```
+
+// MultiKueueWatcher optional interface that can be implemented by a MultiKueueAdapter
+// to receive job related watch events from the worker cluster.
+// If not implemented, MultiKueue will only receive events related to the job's workload.
+type MultiKueueWatcher interface {
+	// returns an empty list of objects
+	GetEmptyList() client.ObjectList
+	// returns the key of the workload of interest
+	// - the object name for workloads
+	// - the prebuilt workload for job types
+	GetWorkloadKey(runtime.Object) (types.NamespacedName, error)
+}
+```
+Used by the [MultiKueueCluster Controller](#multikueuecluster-controller) to start watching job updates in the remote clusters and convert them
+remote events into local workload reconcile events.
+
+### Follow ups ideas
 
 * Handle large number of clusters via selectors.
 * Provide plugin mechanism to control how the workloads are distributed across worker clusters.