Introducing sidecarInjection to ApplicationLayer resource #3460

radixo · 2024-08-14T13:12:10Z

Description

Those changes are introduced to make the operator capable to install and control the new Sidecar for L7 features.

Changes:

Create the SidecarInjection property into the ApplicationLayer resource, which will make the operator to understand when to install or remove the sidecar injection dependencies.
Install the MutatingWebhookConfiguration into the cluster for sidecar injection when the property is "Enabled"
Install the L7AdmissionControler web server inside ApiServer pods, as a new container, to control the creation of new pods that has the label "applicationlayer.projectcalico.org/sidecar" = "true", when the sidecar injection property is "Enabled" as well.

Design doc of the new feature as follows:
https://docs.google.com/document/d/1C5VKry_HLTLZq6VmpwaznpSjwYm4aYqeEpeN54S2fas/edit?usp=sharing

P.S. The introduced feature doesn't break any of the current L7 Application Layer features (WAF, Logging and ALP).

For PR author

Tests for change.
If changing pkg/apis/, run make gen-files
If changing versions, run make gen-versions

For PR reviewers

A note for code reviewers - all pull requests must have the following:

Milestone set according to targeted release.
Appropriate labels:
- kind/bug if this is a bugfix.
- kind/enhancement if this is a a new feature.
- enterprise if this PR applies to Calico Enterprise only.

caseydavenport

First quick pass - mostly nits, but a couple of API level questions that will help me evaluate some of the other bits of code.

pkg/render/apiserver.go

pkg/render/applicationlayer/applicationlayer.go

pkg/render/apiserver.go

api/v1/applicationlayer_types.go

pkg/controller/applicationlayer/applicationlayer_controller.go

Co-authored-by: Casey Davenport <caseydavenport@users.noreply.github.com>

caseydavenport

OK, second round done. Will keep looking through this.

Overall I don't see any major bugs in the code, but I want to take a few more looks as I continue to get a better feel for the feature set and desired behaviors.

caseydavenport · 2024-09-10T15:21:11Z

pkg/render/apiserver.go

@@ -65,6 +67,11 @@ const (
 	tigeraAPIServerTLSSecretName                    = "tigera-apiserver-certs"
 	APIServerSecretsRBACName                        = "tigera-extension-apiserver-secrets-access"
 	MultiTenantManagedClustersAccessClusterRoleName = "tigera-managed-cluster-access"
+	L7AdmissionControllerContainerName              = "calico-l7admssctrl"
+	L7AdmissionControllerEnvoyImage                 = "envoyproxy/envoy:v1.31-latest"


I think we need to hook this image up through the "components" tooling like we do for our other images, rather than hard coding it here. i.e., it should be configurable via enterprise_versions.yml

Additionally, I suspect that using latest tagged images isn't going to fly here, as it means the version that is installed can vary over time.

@Behnam-Shobiri could you chime in on whether we need to have build ownership / registry ownership of this image from an enterprise CVE perspective?

I think we probably need to host a specific version of this in our own registry, though.

pkg/render/applicationlayer/applicationlayer.go

pkg/render/apiserver.go

caseydavenport · 2024-09-10T17:34:12Z

pkg/render/applicationlayer/applicationlayer.go

+	collectorImage        string
+	dikastesImage         string
+	dikastesEnabled       bool
+	envoyEnabled          bool


I assume envoyEnabled means perHostEnvoyEnabled?

I think we should refactor these names to be explicit about whether they are referring to per-host, sidecar, or "any" version of the component for clarity.

pkg/render/applicationlayer/applicationlayer.go

api/v1/applicationlayer_types.go

pkg/render/apiserver.go

caseydavenport · 2024-09-10T17:47:49Z

pkg/render/applicationlayer/applicationlayer.go

 			commandArgs = append(
 				commandArgs,
-				"--waf-enabled",
 				"--waf-log-file", filepath.Join(CalicologsVolumePath, "waf", "waf.log"),


should --waf-log-file go under the if c.config.WAFEnabled section below? What aobut --waf-ruleset-file?

At face value these both seem like arguments that are only relevant when WAF is enabled.

The --waf-enabled is telling dikastes that the per-host enforcement is enabled for WAF, needed for internal decisions on requests that can come even from sidecars or TPROXY iptables rules. So the --waf-log-file and --waf-ruleset-file are the configs that are required by both.

Perfect.

Honestly, the renaming of WAFEnabled -> PerHostWAFEnabled makes this SO much easier to read for me now - this makes sense.

The --waf-enabled is telling dikastes that the per-host enforcement is enabled for WA

It is probably a good idea to rename this argument on Dikastes (or add a new one and deprecate this one)....

--per-host-waf-enabled

^ Would be more accurate.

pkg/controller/applicationlayer/applicationlayer_controller.go

caseydavenport

Only minor comments this time - otherwise this LGTM.

Thanks much @radixo !

caseydavenport · 2024-09-11T21:27:49Z

api/v1/apiserver_types.go

@@ -78,7 +78,7 @@ func init() {
 type APIServerDeploymentContainer struct {
 	// Name is an enum which identifies the API server Deployment container by name.
 	// Supported values are: calico-apiserver, tigera-queryserver
-	// +kubebuilder:validation:Enum=calico-apiserver;tigera-queryserver
+	// +kubebuilder:validation:Enum=calico-apiserver;tigera-queryserver;calico-l7admssctrl


Given this name shows up on a user-facing API, can we change the container name to avoid abbreviations?

calico-l7-admission-controller

caseydavenport · 2024-09-11T21:31:00Z

pkg/render/applicationlayer/applicationlayer.go

 			commandArgs = append(
 				commandArgs,
-				"--waf-enabled",
 				"--waf-log-file", filepath.Join(CalicologsVolumePath, "waf", "waf.log"),


Perfect.

Honestly, the renaming of WAFEnabled -> PerHostWAFEnabled makes this SO much easier to read for me now - this makes sense.

caseydavenport · 2024-09-11T21:32:01Z

pkg/render/applicationlayer/applicationlayer.go

 			commandArgs = append(
 				commandArgs,
-				"--waf-enabled",
 				"--waf-log-file", filepath.Join(CalicologsVolumePath, "waf", "waf.log"),


The --waf-enabled is telling dikastes that the per-host enforcement is enabled for WA

It is probably a good idea to rename this argument on Dikastes (or add a new one and deprecate this one)....

--per-host-waf-enabled

^ Would be more accurate.

radixo · 2024-09-12T08:20:56Z

Thanks a million @caseydavenport

radixo requested a review from a team as a code owner August 14, 2024 13:12

marvin-tigera added this to the v1.36.0 milestone Aug 14, 2024

marvin-tigera added docs-pr-required release-note-required labels Aug 14, 2024

radixo force-pushed the sidecar-injection branch 8 times, most recently from dd29447 to d00989a Compare August 19, 2024 16:11

radixo changed the title ~~[WIP] Introducing sidecarInjection to ApplicationLayer resource~~ Introducing sidecarInjection to ApplicationLayer resource Sep 3, 2024

radixo force-pushed the sidecar-injection branch 10 times, most recently from 5dbc3d7 to b6f5a90 Compare September 4, 2024 16:43

Introducing sidecarInjection to ApplicationLayer resource

0a3cfb0

radixo force-pushed the sidecar-injection branch from b6f5a90 to 0a3cfb0 Compare September 4, 2024 17:24

radixo added 2 commits September 9, 2024 02:06

Bug fixes after tests

89abfc3

UT fixes

e048439

caseydavenport requested changes Sep 9, 2024

View reviewed changes

radixo and others added 2 commits September 10, 2024 13:37

Fixes after first @caseydevenport review

ed81c92

Update pkg/render/apiserver.go (forgot on previeous commit)

053f1f9

Co-authored-by: Casey Davenport <caseydavenport@users.noreply.github.com>

radixo added 2 commits September 10, 2024 14:03

Fixes after @caseydavenport first review

9ac26b6

Executed gen-files

1385b35

caseydavenport requested changes Sep 10, 2024

View reviewed changes

radixo added 3 commits September 11, 2024 20:51

Fixes after @caseydavenport's review

e08fd2b

Fix gen files

7a1f6e0

More gen files fixes

d432a78

caseydavenport approved these changes Sep 11, 2024

View reviewed changes

radixo added 2 commits September 12, 2024 10:28

Changing Dikastes arguments

d9dd687

Clarifying calico-l7-admission-controller container name

29ed278

caseydavenport merged commit bb1e6d8 into tigera:master Sep 12, 2024
5 checks passed

radixo deleted the sidecar-injection branch September 12, 2024 14:34

electricjesus mentioned this pull request Sep 12, 2024

revert dikastes flags rename that featured 'per-host' #3492

Open

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introducing sidecarInjection to ApplicationLayer resource #3460

Introducing sidecarInjection to ApplicationLayer resource #3460

radixo commented Aug 14, 2024 •

edited

Loading

caseydavenport left a comment

caseydavenport left a comment

caseydavenport Sep 10, 2024

caseydavenport Sep 10, 2024

caseydavenport Sep 10, 2024

radixo Sep 11, 2024

caseydavenport Sep 11, 2024

caseydavenport Sep 11, 2024

caseydavenport left a comment

caseydavenport Sep 11, 2024

caseydavenport Sep 11, 2024

caseydavenport Sep 11, 2024

radixo commented Sep 12, 2024 •

edited

Loading

Introducing sidecarInjection to ApplicationLayer resource #3460

Introducing sidecarInjection to ApplicationLayer resource #3460

Conversation

radixo commented Aug 14, 2024 • edited Loading

Description

For PR author

For PR reviewers

caseydavenport left a comment

Choose a reason for hiding this comment

caseydavenport left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

caseydavenport left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

radixo commented Sep 12, 2024 • edited Loading

radixo commented Aug 14, 2024 •

edited

Loading

radixo commented Sep 12, 2024 •

edited

Loading