updated choreo delete and ref downloads

DESIGN-HIERARCHICAL-LOADER.md

@@ -1,97 +1,34 @@
 # hierarchical locader
 
-1. load upstream refs in a hierarchical way
-    - crd refs
-    - child choreoinstance refs
-
-2. we load the apis from all of them from the beginning
-    - TODO: need to check conflict (different upstream refs use the same api object with a different reference)
-    - We load the libraries in the same way as the CRDs
-
-
-3. running the instances
-    - we can run the instances in sequence as per priority field -> first focus
-    - we can run them simulteniously
-
-
-current approach:
-
-- assumption is that the library and input data is rendered to input directory
-- at server start: for all root instances and child instances
-    - load API
-- at runner start: 
-    - get api(s)
-    - load input data
-    - look at inventory/garbage collect (non choreo resources)
-    - load reconcilers and libs from the apis
-    - start the runner
-
-new approach:
-
-1. build the parent child relationships 
-2. load the apis + reconcilers + libraries in memory per child root
-    -> we do this at the runner since things can change along the path
-3. snapshot should keep track of this context
-
-
-root (priority 100)
-data for child 
-+- childroot1 (priority 10)
-  +- child crd
-  +- child crd
-+- childroot2 (priority 20)
-  +- child crd
-  +- child crd
-+- child crd
-+- child crd
-
-root (priority 100)
-+- child crd
-+- child crd
+1. load the upstream ref in a hierarchical way
+    -> we create child choreo instances
 
+we allow for :
+- root -> childroot (crd/reconcilers/data) -> childcrd
+- root -> childcrd (crd/lib)
+- root -> data/reconcilers/libs/crds
 
+kubenet
++- childroot1 topologyref priority 10
+  +- childinstance (crd)
+  +- childinstance (crd)
+  +- reconcilers
+  +- data (templates)
++- reconcilers
++- in
++- refs
 
-examples
+2. init apis globally (we assume no api overlap right now or inconsistencies)
 
-topology
-- data for childroot1 (topology)
-+- childroot1 topologyref
-  +- child crd
-  +- child crd
-  data templates
+3. libraries are loaded and stored per rootinstance or childrootinstance
 
-sequence
-- load apis, libraries and reconcilers per choreoInstance hierarchy
-- run childroot topologyref
-    - load data 
-    - run reconcilers when data is available
-- run root
-    - load data
-    - run topology child reconcilers since data is attached to the child reconcilers
+4. reconcilers are loaded and stored per rootinstance or childrootinstance
 
-kubenet
-- data for childroot1 (topology)
-+- childroot1 topologyref priority 10
-  +- child crd
-  +- child crd
-  data templates
-- data for kubenet (ipindex, network design)
-+- childroot1 kubenet
-  +- child crd
-  +- child crd
+5. data is loaded globally
 
-sequence
-- load apis, libraries and reconcilers per choreoInstance hierarchy
-- run childroot topologyref
-    - load data 
-    - run reconcilers when data is available
-- run childroot kubenet
-    - load data 
-    - don't reconcilers since no data is available
-- run root
-    - load data
-    - run kubenet child reconcilers since data is attached to the kubenet child reconciler
+6. run garbage collection
 
+## Running
 
-run data globallay
+we run the instances in hierarchy
 

TODO.md

@@ -0,0 +1,33 @@
+# TODO
+
+## choreo
+
+- update choreo docs
+- garbage collector: do we ignore the version -> currently we do a special trick in kuid to change the ownereference to v1alpha1 in the backend apis( as, vlan, genid, etc)
+- fetch repo assumes main branch -> need to check the actual used branch
+- snapshots are stored in memory. Is this the right approach ?
+- snapshots add the detailed result.
+- do we need to add reconcilers, libraries to the api or not ?
+    - right now we dont
+- how to handle secrets? Vault ?
+- k8s API versus grpc API ??
+- project scaffold
+
+## kubenet
+- config generation
+- config-diff integration
+
+## choreo controller
+
+- Variant controller
+- Approval controller
+
+## kuid
+
+- rework the generic backend in the same way as IPAM -> allows for real claims
+- proto generation
+- reconcilers (non IPAM)
+
+## main
+
+- align storage backend kuid and chorio

apis/choreo/v1alpha1/reconciler_types.go

@@ -27,20 +27,22 @@ import (
 type ReconcilerSpec struct {
 	// ConditionType defines the condition used by this reconciler to reflect the status of its operation
 	ConditionType *string `json:"conditionType,omitempty" protobuf:"bytes,1,opt,name=conditionType"`
+	// SpecUpdate indicates the reconciler is updating the spec with additional data
+	SpecUpdate *bool `json:"specUpdate,omitempty" protobuf:"bytes,2,opt,name=specUpdate"`
 	// For defines the resource and business logic of the reconciler for this Reconciler.
-	For ReconcilerResource `json:"for" protobuf:"bytes,2,opt,name=for"`
+	For ReconcilerResource `json:"for" protobuf:"bytes,3,opt,name=for"`
 	// Owns define the child resources this Reconciler generates as part of its business logic.
 	// The For resource of this Reconciler owns the derived child resources.
 	// The OwnerReferences are set by the internal reconciler logic. Changes to any of these resources
 	// will trigger the Reconciler reconciler
-	Owns []*ReconcilerResource `json:"owns,omitempty" protobuf:"bytes,3,rep,name=owns"`
+	Owns []*ReconcilerResource `json:"owns,omitempty" protobuf:"bytes,4,rep,name=owns"`
 	// Watches defines the resources on which the main reconciler can be retriggered. The pipeline/business logic
 	// determines if the reconciler is to be retriggered.
-	Watches []*ReconcilerResource `json:"watches,omitempty" protobuf:"bytes,4,opt,name=watches"`
+	Watches []*ReconcilerResource `json:"watches,omitempty" protobuf:"bytes,5,opt,name=watches"`
 	// Type defines the software technology this library contains
-	Type *SoftwardTechnologyType `json:"type,omitempty" protobuf:"bytes,5,opt,name=type"`
+	Type *SoftwardTechnologyType `json:"type,omitempty" protobuf:"bytes,6,opt,name=type"`
 	// Code supporting the reconciler
-	Code map[string]string `json:"code,omitempty" protobuf:"bytes,6,rep,name=code"`
+	Code map[string]string `json:"code,omitempty" protobuf:"bytes,7,rep,name=code"`
 }
 
 type ReconcilerResource struct {

artifacts/choreo.kform.dev_reconcilers.yaml

@@ -181,6 +181,10 @@ spec:
                   - version
                   type: object
                 type: array
+              specUpdate:
+                description: SpecUpdate indicates the reconciler is updating the spec
+                  with additional data
+                type: boolean
               type:
                 description: Type defines the software technology this library contains
                 type: string