Add methods to support CRUD operations on CSE Kubernetes clusters

[adambarreiro]

Context

While the generic Runtime Defined Entity approach to create, read, update and delete Kubernetes clusters is completely valid and usable, it causes some challenges to the final users, as the process can be cumbersome at certain stages, and prone to errors.

The idea of this PR is to provide a set of abstractions that simplify the mentioned operations. These abstractions are accomplished by introducing several new public types and methods on these types.

Description

This PR adds some new types:

• CseKubernetesCluster to manage Container Service Extension Kubernetes clusters.
• CseClusterSettings, CseControlPlaneSettings, CseWorkerPoolSettings, CseDefaultStorageClassSettings to configure a Container Service Extension Kubernetes cluster.
• CseClusterEvent to define an event inside CseKubernetesCluster
• CseClusterUpdateInput, CseControlPlaneUpdateInput, CseWorkerPoolUpdateInput to update an existing Container Service Extension Kubernetes cluster.
• Some other internal/private auxiliary types.

This PR adds some new methods:

• The Org type has two new methods CseCreateKubernetesCluster and CseCreateKubernetesClusterAsync to create Kubernetes clusters in a VCD appliance with Container Service Extension installed.
• The VCDClient type has a new method CseGetKubernetesClusterById and Org has a new method CseGetKubernetesClustersByName, both used to retrieve a Container Service Extension Kubernetes cluster. The first one returns a unique cluster, and the Org one returns several clusters, as there can be multiple clusters with the same name.
• The new type CseKubernetesCluster has several methods:
  • A method GetKubeconfig to retrieve the kubeconfig of a provisioned Container Service
    Extension Kubernetes cluster
  • A method Refresh to refresh the information and properties of an existing Container Service Extension Kubernetes cluster.
  • Methods to update a Container Service Extension Kubernetes cluster: UpdateWorkerPools,
    AddWorkerPools, UpdateControlPlane, UpgradeCluster,
    SetNodeHealthCheck and SetAutoRepairOnErrors. These are like "syntactic sugar" of the more generic/wide Update method that can do several updates in one place.
  • A method GetSupportedUpgrades to retrieve all the valid TKGm OVAs that a given Container
    Service Extension Kubernetes cluster can use to be upgraded
  • A method Delete to delete a cluster

File layout

• cse.go contains the main public CRUD operations described above. The goal of this file is to be concise, to have a clear documentation and provide an instant view of the available operations to the consumers of the SDK.
• cse_util.go contains private methods used by the CRUD operations, that may be too complex or reused several times. These are all private and users should not worry about them.
• cse_internal.go contains private methods that transform the "public facing types" (described above) to "internal private ones", and manage/consume Go templates with them.
• cse_yaml.go contains private methods that manipulate the CAPI YAML that describes a cluster. Essentially, they are used on updates, as this operation is the one that manipulates YAML.
• cse_type.go contains all the types used by all the CSE methods, public and private ones.

Extra changes

• Improves the previous logic to retrieve a created RDE. When you create a RDE, the response is empty and there is a new Task in the headers. The previous one didn't take into consideration this Task, and queried all RDEs to filter them by name, which can be error-prone, to recover the created RDE and return it to the user. The improved way recovers this Task, and with the ID that is set inside it recovers the unique RDE represented by it. This way is way more precise and correct.
• Refactors several methods that were only public and took a public entity such as VCDClient, to have also a friendly private version that can take any generic Client object.
• Adds missing fields to existing types, that were needed.

Testing

Testing CSE methods

To run the CSE tests, one must set the TEST_VCD_CSE environment variable to any non-empty value. The reason is that CSE requires a very unique environment, and the tests take a long time to be executed.

The tests also require a special govcd_test_config.yaml file that is almost empty and contains the new cse section:

provider:
  user: administrator
  password: *****
  url: https://my-vcd.com/api
  sysOrg: System
vcd:
  org: tenant_org
  nsxt:
    vdc: tenant_vdc
logging:
  enabled: true
  logFileName: go-vcloud-director.log
  verboseCleanup: true
cse:
  version: "4.2.0"
  solutionsOrg: "solutions_org"
  storageProfile: "*"
  tenantOrg: "tenant_org"
  tenantVdc: "tenant_vdc"
  routedNetwork: "tenant_net_routed"
  edgeGateway: "tenant_edgegateway"
  ovaCatalog: "tkgm_catalog"
  ovaName: "ubuntu-2004-kube-v1.25.7+vmware.2-tkg.1-8a74b9f12e488c54605b3537acb683bc"

The one can run:

GOVCD_SKIP_VAPP_CREATION=1;TEST_VCD_CSE=1 go test -tags cse -check.f 'Test_Cse' -check.vv -timeout=0 | tee $HOME/test-cse.log

Testing other minor changes

To test other minor changes, like refactors and RDE minor improvements, the usual procedure can be used.

[lvirbalas]

Huge! Looking forward to the Terraform part!