-
Notifications
You must be signed in to change notification settings - Fork 77
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feat: support Kubernetes multi-cluster configuration #1192
Comments
@hoangndst This is the issue tracking the support for |
|
@hoangndst Thanks for your comment! Could you please provide a specific example to further illustrate your requirements? From my understanding, you may want the workspace to support multiple Kubernetes clusters and users can declare some shared infrastructure resources in the workspace for multiple Projects or Stacks, such as a global database and S3, or even the EC2 instances, which may need to be created when setting up the workspace environment. However, we currently consider the workspace as a Landing Zone, which typically corresponds to a single Kubernetes cluster. For different K8s clusters, we suggest users create different workspaces to manage them. And since Kusion is application-centric, we have primarily focused on managing the resources at the application level. Thus we haven't supported the global workspace-level resources in workspace. But we have reserved a Meanwhile, we are currently working on supporting the import of existing Terraform resources. Applications can reuse existing resources by adding
The issue you mentioned about managing global infrastructure resources shared between multiple applications in workspace is still under discussion. If possible, we hope you can provide a specific scenario so that we can work together to design a solution : ) cc @SparkYuan |
After discussing with @SparkYuan and @ffforest we have tentatively formulated the following design to support Kubernetes and Terraform runtime configurations in workspace. We will use the Considering the design concept of Kusion, a
# Example K8s runtime configs in workspace.
context:
kubeconfig_path: /Users/kusion-test/.kube/config
kubeconfig_content: ref://kubeconfig/kubeconfig-content If both of the In addition to Kubernetes runtime configurations, # Example TF provider runtime configs in workspace.
context:
AWS_ACCESS_KEY_ID: AK**********
AWS_SECRET_ACCESS_KEY: ref://aws-sk/secret-access-key
AWS_REGION: us-east-1 The priority order for runtime configuration in workspace
# Use `curl` to get `kubeconfig` content.
context:
kubeconfig_context: https://location-of-the-kubeconfig As for obtaining @hoangndst Can this meet your requirements? Welcome to review and provide your ideas and suggestions. Looking forward to hearing your thoughts : ) |
With this method, the problem of k8s resources will be solved. But what about resource managers? In order for Kusion becomes real New DesignStorage:
Feature:
@liu-hm19 @SparkYuan Looking forward to hearing your comments :D |
The definition of The Spec represents the operational intentions that you aim to deliver using Kusion. It contains all resources that would be operated in one kusion command such as 'kusion apply'. The State is a record of the result of an operation. It is a mapping between resources managed by Kusion and the actual infra resources. It is used as a data source in the 3-way merge/diff during operations like 'apply' and 'preview'. We will add one document to our website to explain the concept of Release next week. Here is the issue to track it. Regarding the question you mentioned above, Kusion is an app-centric system designed to improve the efficiency of overall app development. However, certain resources may not belong to a specific application or may need to exist before any application, such as a k8s cluster. We have created a new Project and defined an internal schema to initialize all these shared resources. Applications in other Projects can reuse existing resources by adding the IaaS resource ID in the Spec. This feature will be released in the next version. The internal schema is closely tied to our internal business, so it hasn't been open-sourced yet. The requirement you proposed does exist and we plan to solve it. We also have some discussions within our team. If you have any ideas, we are open to discuss and any inputs are appreciated. Besides, Kusion also supports resource dependencies like you referred above. Details can be found here |
What would you like to be added?
Please support the declaration of Kubernetes runtime configurations in Workspace to manage multi-clusters with different workspace configurations.
The configuration may includes:
KubeConfig
KubeConfig
content, e.g. S3 endpointKubeConfig
content itselfWhy is this needed?
Background
Kusion needs to deploy the application resources to a specific Kubernetes cluster, thus it should allow users to specify the information of the targeted cluster. Currently, users can specify the cluster by configuring the
KUBECONFIG
environment variable, which stay consistent withkubectl
. The related codes can be found hereTarget
In order to better integrate Kusion in the CI/CD pipeline and support the multi-cluster scenarios more flexibly, we hope to add Kubernetes runtime configurations in Kusion Workspace, for example, it can include the following configs:
KubeConfig
KubeConfig
content, e.g. S3 endpointKubeConfig
content itselfSo that each workspace will correspond to a separate Kubernetes cluster, and users don't need to re-set the
KUBECONFIG
environment variable every time before deploying to a different Kubernetes cluster.Main Concern
Actually, we once supported declaring Kubernetes and Terraform runtime configurations in Workspace in previous versions, but later we removed it for the following reasons:
KubeConfig
can usually be considered as a workspace-level configuration, but the config of Terraform Provider is very like to be at the resource level, which means that the resources of different modules in the same workspace may still differCurrently, we also support specifying the runtime information in the
Extensions
field ofResource
inSpec
. Now, we need to consider the actual requirements and situations comprehensively to implement this feature.The text was updated successfully, but these errors were encountered: