Support settings for RuntimeExtensions

[fabriziopandini]

User Story

As a RuntimeExtensions author, I would like to offer settings allowing cluster administrators to tune my RuntimeExtensions behavior when installed on a management cluster

As a RuntimeExtensions author, I would like to offer settings allowing cluster class authors to tune my RuntimeExtensions behavior when used in a ClusterClass

Detailed Description

RuntimeExtension is a powerful mechanism for extending Cluster API and we expect RuntimeExtensions to become more and more complex over time. This will lead RuntimeExtensions authors to expose settings allowing users to customize RuntimeExtensions behavior on different clusters or when used in different ClusterClass.

Eg. I have Runtime extension with a couple of lifecycle hooks, and on specific management clusters I want to disable some of them or simply tune the RequeAfter interval

Eg. I have Runtime extension with many external patches, and on specific ClusterClasses I want to disable some of them

RuntimeExtension extension can implement those use cases in many ways, exposing flags, checking for setting in a config map etc. etc. but all those approaches won't be integrated with Cluster API (so the users will be required to change RuntimeExtension deployment or to act on different objects).

With this issue we are proposing to introduce a setting fields both in

apiVersion: runtime.cluster.x-k8s.io/v1alpha1
kind: ExtensionConfig
metadata:
  name: test-extension
spec:
  # settings allows to define key value pairs
  # to be passed to the all the calls to RuntimeExtensions provided
  # by the this component.
  settings:
      disableBeforeCreateClusterHook: "true"
      retryAfterInterval: "20s"

And in

apiVersion: cluster.x-k8s.io/v1beta1
kind: ClusterClass
metadata:
  name: quick-start-runtimesdk
spec:
  ...
  variables:
      <<- for the user / they surface on the Cluster
  
  patches:
  - name: lbImageRepository
    external:
      # settings allows to define key value pairs to be passed to the topology mutation hook call;
      # settings provided in this section will be merged with settings from the ExtensionConfig, with settings
      # defined here taking precedence in case of conflicts.
      # NOTE: settins are visible to the ClusterClass authors only and cannot be changed on a Cluster bases
      # like variables.
      settings:
          disableHttpProxyPatch: "true"
      generateExtension: generate-patches.k8s-upgrade-with-runtimesdk
      validateExtension: validate-topology.k8s-upgrade-with-runtimesdk

/kind feature
/area runtime-sdk

[sbueringer]

Sounds good to me!

/triage accepted

[sbueringer]

/assign

[ykakarap]

/unassign @sbueringer

/assign