☔ Konnect managed entities

[lahabana]

Problem Statement

Users would like to manage Konnect entities directly from Kubernetes resources.
This issue is the main tracker to cover having CRD routes for things like: Service, Consumers, Routes.

The goal is to have a semi-automatic way to add support for entities in platform-api. This is necessary to minimise future work when new Konnect entities are added.

Proposed Solution

Pending Product brief and Engineering brief.

Some open questions:

• How does relationship work?
• What's in status/condition?
• Can you mix entities managed by CRDs and non.

Additional Information

Some useful reads:

• Discovery doc

Implementation uses https://github.com/Kong/sdk-konnect-go/ to interact with Konnect which is generated from https://github.com/Kong/platform-api.

Acceptance Criteria

Design

Beta Give feedback

1. Spike: Konnect managed entities design #382
   area/spike release/required +2
   

Implementation - August demo scope

Beta Give feedback

1. Konnect entities: reorganize API groups, add konnect API group #442
   area/konnect +1
   
2. Konnect entities: implement a generic reconciler scaffolding #437
   area/konnect +1
   
3. Konnect entities: KonnectAuthConfiguration reconciler #431
   area/crds area/konnect +2
   
4. Konnect entities: KonnectControlPlane CRD #432
   area/crds area/konnect +2
   
5. Konnect entities: KongService #433
   area/crds area/konnect +2
   
6. Konnect entities: KongRoute #435
   area/crds area/konnect +2
   
7. Konnect entities: KongConsumer #436
   area/crds area/konnect +2
   
8. Konnect entities: KongConsumerGroup #514
   area/konnect +1
   
9. Konnect entities: KongPluginBinding #438
   area/crds area/konnect +2
   
10. Konnect entities: add Konnect mocks for integration testing #447
    area/konnect +1
    
11. Konnect entities: add integration tests against an actual Konnect org #481
    area/konnect area/tests +2
    
12. Konnect entities: add k8s metadata to Konnect entities #480
    area/konnect +1
    
13. Konnect entities: KongPlugin reconciler #530
    area/konnect +1
    
14. Konnect entities: Consumer's group assignment #519
    area/konnect +1
    

Implementation - GA scope (October-ish)

Beta Give feedback

1. Konnect entities: Gateway Control Plane Group - Control Plane Group Memberships #449
   area/konnect +1
   
2. Konnect entities: KongUpstream #575
   area/crds area/konnect +2
   
3. Konnect entities: KongTarget #576
   area/crds area/konnect +2
   
4. Konnect entities: Vault #577
   area/konnect +1
   
5. Konnect entities: Certificate #581
   area/crds area/konnect +2
   
6. Konnect entities: CA Certificate #582
   area/crds area/konnect +2
   
7. Konnect entities: Key #584
   area/crds area/konnect +2
   
8. Konnect entities: Key Set #585
   area/crds area/konnect +2
   
9. Konnect entities: SNI #598
   area/crds area/konnect +2
   
10. Konnect entities: Data Plane Client Certificates #608
    area/konnect +1
    
11. Konnect entities: Credentials - Basic Auth #617
    area/crds area/konnect +2
    
12. Konnect entities: Credentials - Key Auth #619
    area/crds area/konnect +2
    
13. Konnect entities: Credentials - JWT #620
    area/crds area/konnect +2
    
14. Konnect entities: Credentials - HMAC Auth #621
    area/crds area/konnect +2
    
15. Konnect entities: Credentials - ACL #622
    area/crds area/konnect +2
    
16. Konnect entities: make Konnect entities sync period configurable #451
    area/konnect +1
    
17. Konnect entities: allow KongRoute to be targeted by a KongPluginBinding #525
    area/konnect +1
    
18. Konnect entities: allow KongConsumer to be targeted by a KongPluginBinding #526
    area/konnect +1
    
19. Konnect entities: allow KongConsumerGroup to be targeted by a KongPluginBinding #527
    area/konnect +1
    
20. Konnect entities: enqueue KongPluginBinding when a KongPlugin changes #528
    area/konnect +1
    
21. Konnect entities: enqueue KongPluginBinding when a KongClusterPlugin changes #529
    area/konnect +1
    
22. Konnect entities: watch relevant entities in the KongPluginBinding reconciler #552
    area/konnect +1
    
23. Konnect entities: Create KongPluginBinding when kongRoutes, KongConsumers, and KongConsumerGroups get annotated #583
    area/konnect +1
    
24. Make KongRoute's ServiceRef optional to allow serviceless routes kubernetes-configuration#99
    +0
    
25. Konnect entities: implement serviceless KongRoute #665
    area/konnect +1
    
26. Konnect entities: index KongServices in the KongPlugin reconciler #596
    area/konnect +1

Post GA

Beta Give feedback

1. Consider plugin bindings combinations created by annotating KongService, KongRoute, KongConsumer and KongConsumerGroup objects #660
   area/konnect +1
2. Konnect entities: Secrets to Credentials reconciler #618
   +0
   
3. Konnect entities: existing entity adoption #460
   area/konnect +1
4. Konnect entities: global plugins #440
   area/konnect +1
5. Konnect entities: KongClusterPlugin reconciler #524
   area/konnect +1
6. API Products
7. Portal
8. Konnect entities: implement migration from serviceless KongRoute (control plane bound) to service bound and vice versa. #668
   area/konnect +1

Documentation

Beta Give feedback

1. Konnect entities: add official documentation how KGO can manage Konnect entities #439
   area/konnect documentation +2
2. Konnect entities: add documentation on how plugins are mapped into Konnect and tutorials with the new resource KongPluginBinding #568
   area/konnect +1

Related issues

Beta Give feedback

1. Making /organizations/me public for use in public konnect-sdk-go: https://konghq.atlassian.net/browse/TPS-2267, https://github.com/Kong/platform-api/pull/704
2. https://github.com/Kong/khcp/pull/8011