Add FluxCD Application Management

[chengleqi]

What type of PR is this?

/kind feature
/kind design

What this PR does / why we need it:

• Extend the gitops.kubesphere.io/v1alpha1/Application CRD to Support FluxApp.
• Add fluxcd/application-controller to maintain the fluxcd Kustomization and HelmRelease.
• Update the config/rbac/role.yaml
• Add sample files in config/samples/gitops directory.

Which issue(s) this PR fixes:

Fixes #717 , #718

Special notes for reviewers:

Please check the following list before waiting reviewers:

• Already committed the CRD files to the Helm Chart if you created some new CRDs
• Already added the permission for the new API
• Already added the RBAC markers for the new controllers

Does this PR introduce a user-facing change??

None

[codecov]

Codecov Report

Merging #743 (49dcc3f) into master (9367e7a) will increase coverage by 2.55%.
The diff coverage is 84.31%.

@@            Coverage Diff             @@
##           master     #743      +/-   ##
==========================================
+ Coverage   51.09%   53.65%   +2.55%     
==========================================
  Files         171      181      +10     
  Lines       10141    10668     +527     
==========================================
+ Hits         5182     5724     +542     
+ Misses       4666     4627      -39     
- Partials      293      317      +24     

Flag	Coverage Δ
unittests	53.65% <84.31%> (+2.55%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
pkg/api/gitops/v1alpha1/application.go	100.00% <ø> (ø)
controllers/fluxcd/application-controller.go	84.10% <84.10%> (ø)
pkg/apis/apis.go	100.00% <100.00%> (ø)
pkg/client/devops/jenkins/credential.go	0.00% <0.00%> (-30.47%)	⬇️
pkg/kapis/devops/v1alpha2/register.go	95.29% <0.00%> (-0.16%)	⬇️
pkg/models/devops/common.go	100.00% <0.00%> (ø)
pkg/client/devops/jenkins/job.go	0.00% <0.00%> (ø)
pkg/client/devops/jenkins/build.go	0.00% <0.00%> (ø)
pkg/client/devops/jenkins/pipeline.go	5.52% <0.00%> (ø)
.../client/devops/jenkins/pipeline_model_converter.go	0.00% <0.00%> (ø)
... and 12 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[chengleqi]

I encountered a strange problem when I wrote unit test due to the version of controller-runtime.

I use code like this r.Client.List(..., &unstructured.UnstructuredList{}, ...), It run correctly when the Client is RestClient, but It'll run into error when the client is fake client in the unit test scenario.

I found the root cause is this issue, but the PR which fix this issue was merged only in the 0.9.x version of controller-runtime, I tried to update the controller-runtime v0.6.3 => v0.9.7 but It caused too many changes.

So I Added code manually like this in the unit test:

schema.AddKnownTypeWithName(apischema.GroupVersionKind{
        Group:   "helm.toolkit.fluxcd.io",
        Version: "v2beta1",
        Kind:    "HelmReleaseList",
}, &unstructured.UnstructuredList{})

And the fluxcd project actually provide the module which include the HelmRelease type for convenience. But It also need higher version of controller-runtime.

Is it possible to update the controller-runtime version when it arrive version 1.0 or LTS ?

[chengleqi]

When I delete the gitops.kubesphere.io/Application I want to delete all HelmReleases belong to it. I can implement it just set the all HelmReleases' OwnerReferences to the Application. And I also can set the Finalizer on Application to deal with it.

Is there any different between the OwnerReferences solution and the Finalizer solution in this situation?

[LinuxSuRen]

Is there any different between the OwnerReferences solution and the Finalizer solution in this situation?

I'm not sure if I understand it correctly. But it might be helpful for you.

IMO, the OwnerReferences is a built-in controller for resource collection. It could help you to delete Kubernetes CR (custom resource) instead of others (Jenkins jobs .e.g.). And for this case, the child resource does not have its own life cycle.

The Finalizer could let you delete any resources that you want.

See also https://kubernetes.io/blog/2021/05/14/using-finalizers-to-control-deletion/

[chengleqi]

Is there any different between the OwnerReferences solution and the Finalizer solution in this situation?

I'm not sure if I understand it correctly. But it might be helpful for you.

IMO, the OwnerReferences is a built-in controller for resource collection. It could help you to delete Kubernetes CR (custom resource) instead of others (Jenkins jobs .e.g.). And for this case, the child resource does not have its own life cycle.

The Finalizer could let you delete any resources that you want.

See also https://kubernetes.io/blog/2021/05/14/using-finalizers-to-control-deletion/

Thanks a lot, this is very helpful.

[chengleqi]

/test ?

[ks-ci-bot]

@chengleqi: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test ?

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[lxm]

@chengleqi: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

@kubesphere/sig-devops @LinuxSuRen

[ks-ci-bot]

@lxm: changing LGTM is restricted to collaborators

In response to this:

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[lxm]

@kubesphere/sig-devops

[LinuxSuRen]

/test ?

You can retrigger the actions once you close and reopen this PR.

[lxm]

while controller-runtime version has been updated to v0.12.3 with #788.
some previously change requests in the PR can be performed, such as data struct import method

[LinuxSuRen]

Please try not to use force action. I need to review all files one by one if you push the code with the force option.

[LinuxSuRen]

/lgtm

[ks-ci-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To complete the pull request process, please ask for approval from linuxsuren after the PR has been reviewed.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[lxm]

lgtm @kubesphere/sig-devops @LinuxSuRen

[LinuxSuRen]

/lgtm