Ta update configs to enable mtls #3015

ItielOlenick · 2024-06-06T05:42:16Z

Description: When CertManager and secrets RBAC permissions are granted, mTLS will be used between the target allocator and the collector so that the latter can retrieve authentication secrets for endpoints that require them.

Link to Tracking Issue(s):

Second PR towards a solution for #1669

Testing: Unit tests added. E2E tests added. Tested in-cluster locally.

Documentation: Added documentation

Bumps [github.com/gin-gonic/gin](https://github.com/gin-gonic/gin) from 1.9.1 to 1.10.0. - [Release notes](https://github.com/gin-gonic/gin/releases) - [Changelog](https://github.com/gin-gonic/gin/blob/master/CHANGELOG.md) - [Commits](gin-gonic/gin@v1.9.1...v1.10.0) --- updated-dependencies: - dependency-name: github.com/gin-gonic/gin dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…elemetry#2951) Bumps the prometheus group with 1 update: [github.com/prometheus/prometheus](https://github.com/prometheus/prometheus). Updates `github.com/prometheus/prometheus` from 0.51.2 to 0.52.0 - [Release notes](https://github.com/prometheus/prometheus/releases) - [Changelog](https://github.com/prometheus/prometheus/blob/main/CHANGELOG.md) - [Commits](prometheus/prometheus@v0.51.2...v0.52.0) --- updated-dependencies: - dependency-name: github.com/prometheus/prometheus dependency-type: direct:production update-type: version-update:semver-minor dependency-group: prometheus ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* enable readiness Probe for otel operator Signed-off-by: Janario Oliveira <janario.oliveira@gmail.com> * generate CRD and controller changes Signed-off-by: Janario Oliveira <janario.oliveira@gmail.com> * Adjusted code to be similar to Liveness logic Signed-off-by: Janario Oliveira <janario.oliveira@gmail.com> * Generated manifests Signed-off-by: Janario Oliveira <janario.oliveira@gmail.com> * Add changelog Signed-off-by: Janario Oliveira <janario.oliveira@gmail.com> * Fix lint Signed-off-by: Janario Oliveira <janario.oliveira@gmail.com> * Removed readinessProbe from alpha CRD Signed-off-by: Janario Oliveira <janario.oliveira@gmail.com> * Generated manifests Signed-off-by: Janario Oliveira <janario.oliveira@gmail.com> * Fix lint Signed-off-by: Janario Oliveira <janario.oliveira@gmail.com> * Centralized probe validation Signed-off-by: Janario Oliveira <janario.oliveira@gmail.com> --------- Signed-off-by: Janario Oliveira <janario.oliveira@gmail.com> Co-authored-by: hesam.hamdarsi <hesam.hamdarsi@gmail.com>

Bumps [github.com/docker/docker](https://github.com/docker/docker) from 26.0.1+incompatible to 26.0.2+incompatible. - [Release notes](https://github.com/docker/docker/releases) - [Commits](moby/moby@v26.0.1...v26.0.2) --- updated-dependencies: - dependency-name: github.com/docker/docker dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Added new Log Enconder Config Signed-off-by: Yuri Sa <yurimsa@gmail.com> * Added new Log Enconder Config Signed-off-by: Yuri Sa <yurimsa@gmail.com> * Added new Log Enconder Config Signed-off-by: Yuri Sa <yurimsa@gmail.com> * Added new Log Enconder Config Signed-off-by: Yuri Sa <yurimsa@gmail.com> * Added new Log Enconder Config Signed-off-by: Yuri Sa <yurimsa@gmail.com> * Added new Log Enconder Config Signed-off-by: Yuri Sa <yurimsa@gmail.com> * Added new Debug doc Signed-off-by: Yuri Sa <yurimsa@gmail.com> --------- Signed-off-by: Yuri Sa <yurimsa@gmail.com>

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

* Fix Signed-off-by: Pavol Loffay <p.loffay@gmail.com> * Fix Signed-off-by: Pavol Loffay <p.loffay@gmail.com> * Fix Signed-off-by: Pavol Loffay <p.loffay@gmail.com> * Fix Signed-off-by: Pavol Loffay <p.loffay@gmail.com> * Add test Signed-off-by: Pavol Loffay <p.loffay@gmail.com> --------- Signed-off-by: Pavol Loffay <p.loffay@gmail.com>

Signed-off-by: Yuri Sa <yurimsa@gmail.com>

…ility check (open-telemetry#2964) * Verify ServiceMonitor and PodMonitor are installed in prom cr availability check * Added changelog

…try#2968) Bumps [kyverno/action-install-chainsaw](https://github.com/kyverno/action-install-chainsaw) from 0.2.0 to 0.2.1. - [Release notes](https://github.com/kyverno/action-install-chainsaw/releases) - [Commits](kyverno/action-install-chainsaw@v0.2.0...v0.2.1) --- updated-dependencies: - dependency-name: kyverno/action-install-chainsaw dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Create a separate Service Monitor when the Prometheus exporter is present Signed-off-by: Israel Blancas <iblancasa@gmail.com> * Improve changelog Signed-off-by: Israel Blancas <iblancasa@gmail.com> * Fix prometheus-cr E2E test Signed-off-by: Israel Blancas <iblancasa@gmail.com> * Remove unused target Signed-off-by: Israel Blancas <iblancasa@gmail.com> * Add docstring Signed-off-by: Israel Blancas <iblancasa@gmail.com> * Fix typo Signed-off-by: Israel Blancas <iblancasa@gmail.com> * Change the label name Signed-off-by: Israel Blancas <iblancasa@gmail.com> * Change changelog description Signed-off-by: Israel Blancas <iblancasa@gmail.com> * Recover removed labels Signed-off-by: Israel Blancas <iblancasa@gmail.com> * Add missing labels Signed-off-by: Israel Blancas <iblancasa@gmail.com> * Remove wrong labels Signed-off-by: Israel Blancas <iblancasa@gmail.com> --------- Signed-off-by: Israel Blancas <iblancasa@gmail.com>

* Prepare release 0.100.0 Signed-off-by: Vineeth Pothulapati <vineethpothulapati@outlook.com> * update the chlog * update the chlog with open-telemetry#2877 merge --------- Signed-off-by: Vineeth Pothulapati <vineethpothulapati@outlook.com>

* Refactor consistent-hashing strategy * Refactor per-node strategy * Refactor least-weighted strategy * Minor allocation strategy refactor * Add some common allocation strategy tests * Fix collector and target reassignment * Minor allocator fixes * Add changelog entry * Fix an incorrect comment

* add back webhook port * chlog

…2967)

Signed-off-by: Pavol Loffay <p.loffay@gmail.com>

* Support for kubernetes 1.30 version * Update makefile

…or, target allocator, opamp bridge (open-telemetry#2933) * set things * fix kustomize shim * restore, better chlog

…y#2984)

Bumps alpine from 3.19 to 3.20. --- updated-dependencies: - dependency-name: alpine dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…emetry#2991) Bumps alpine from 3.19 to 3.20. --- updated-dependencies: - dependency-name: alpine dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [github.com/go-logr/logr](https://github.com/go-logr/logr) from 1.4.1 to 1.4.2. - [Release notes](https://github.com/go-logr/logr/releases) - [Changelog](https://github.com/go-logr/logr/blob/master/CHANGELOG.md) - [Commits](go-logr/logr@v1.4.1...v1.4.2) --- updated-dependencies: - dependency-name: github.com/go-logr/logr dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…try#2989) Bumps [kyverno/action-install-chainsaw](https://github.com/kyverno/action-install-chainsaw) from 0.2.1 to 0.2.2. - [Release notes](https://github.com/kyverno/action-install-chainsaw/releases) - [Commits](kyverno/action-install-chainsaw@v0.2.1...v0.2.2) --- updated-dependencies: - dependency-name: kyverno/action-install-chainsaw dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps the otel group with 5 updates: | Package | From | To | | --- | --- | --- | | [go.opentelemetry.io/otel](https://github.com/open-telemetry/opentelemetry-go) | `1.26.0` | `1.27.0` | | [go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp](https://github.com/open-telemetry/opentelemetry-go) | `1.26.0` | `1.27.0` | | [go.opentelemetry.io/otel/metric](https://github.com/open-telemetry/opentelemetry-go) | `1.26.0` | `1.27.0` | | [go.opentelemetry.io/otel/sdk](https://github.com/open-telemetry/opentelemetry-go) | `1.26.0` | `1.27.0` | | [go.opentelemetry.io/otel/sdk/metric](https://github.com/open-telemetry/opentelemetry-go) | `1.26.0` | `1.27.0` | Updates `go.opentelemetry.io/otel` from 1.26.0 to 1.27.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.26.0...v1.27.0) Updates `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp` from 1.26.0 to 1.27.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.26.0...v1.27.0) Updates `go.opentelemetry.io/otel/metric` from 1.26.0 to 1.27.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.26.0...v1.27.0) Updates `go.opentelemetry.io/otel/sdk` from 1.26.0 to 1.27.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.26.0...v1.27.0) Updates `go.opentelemetry.io/otel/sdk/metric` from 1.26.0 to 1.27.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md) - [Commits](open-telemetry/opentelemetry-go@v1.26.0...v1.27.0) --- updated-dependencies: - dependency-name: go.opentelemetry.io/otel dependency-type: direct:production update-type: version-update:semver-minor dependency-group: otel - dependency-name: go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp dependency-type: direct:production update-type: version-update:semver-minor dependency-group: otel - dependency-name: go.opentelemetry.io/otel/metric dependency-type: direct:production update-type: version-update:semver-minor dependency-group: otel - dependency-name: go.opentelemetry.io/otel/sdk dependency-type: direct:production update-type: version-update:semver-minor dependency-group: otel - dependency-name: go.opentelemetry.io/otel/sdk/metric dependency-type: direct:production update-type: version-update:semver-minor dependency-group: otel ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

ItielOlenick · 2024-07-05T11:38:47Z

@swiatekm I've Added initial e2e tests.
I want to add a test that will bring up a simple app with basic auth on its metrics endpoint, to thoroughly test the entire feature. Where should I add the files? I was thinking either as another app to the e2e apps or build and add to kind during the e2e tests.

swiatekm · 2024-07-11T17:00:04Z

tests/e2e-ta-collector-mtls/ta-collector-mtls/chainsaw-test.yaml

+  - name: step-00
+    try:
+    - apply:
+        template: true
+        file: 00-install.yaml
+    - assert:
+        file: 00-assert.yaml


Looks like you need to add your other two steps here.

Waiting for #3120

swiatekm · 2024-07-11T17:08:51Z

@swiatekm I've Added initial e2e tests. I want to add a test that will bring up a simple app with basic auth on its metrics endpoint, to thoroughly test the entire feature. Where should I add the files? I was thinking either as another app to the e2e apps or build and add to kind during the e2e tests.

I don't think we need to do that - it would really just be testing the prometheus-operator packages, which is presumably handled by prometheus-operator itself. The only feature we're adding here is mTLS for the connection between the collector and target allocator, and it doesn't necessarily matter what data is sent over that connection.

I'm not strongly opposed to it, but I'd rather it happen in a separate PR, so we can more clearly see the complexity that kind of test would introduce.

ItielOlenick · 2024-07-11T17:23:49Z

@swiatekm I've Added initial e2e tests. I want to add a test that will bring up a simple app with basic auth on its metrics endpoint, to thoroughly test the entire feature. Where should I add the files? I was thinking either as another app to the e2e apps or build and add to kind during the e2e tests.

I don't think we need to do that - it would really just be testing the prometheus-operator packages, which is presumably handled by prometheus-operator itself. The only feature we're adding here is mTLS for the connection between the collector and target allocator, and it doesn't necessarily matter what data is sent over that connection.

I'm not strongly opposed to it, but I'd rather it happen in a separate PR, so we can more clearly see the complexity that kind of test would introduce.

I think it should be tested as part of the e2e. I believe we should test that an actual secret is being retrieved by the collector from the target allocator. I think of this enhancement as not only adding mTLS, but exposing the sensitive data when mTLS is used as done in #2921.

I agree it should be done in a separate PR.
Added - #3120

ItielOlenick · 2024-07-26T05:23:28Z

@swiatekm anything missing?

swiatekm · 2024-07-26T15:19:35Z

@swiatekm anything missing?

We're still waiting for the change to prometheus receiver so that e2e tests here can pass, right?

ItielOlenick · 2024-07-26T15:31:17Z

@swiatekm anything missing?

We're still waiting for the change to prometheus receiver so that e2e tests here can pass, right?

Yes, waiting for open-telemetry/opentelemetry-collector-contrib#34035 and also #3120.

swiatekm · 2024-07-30T15:07:05Z

@swiatekm anything missing?

We're still waiting for the change to prometheus receiver so that e2e tests here can pass, right?

Yes, waiting for open-telemetry/opentelemetry-collector-contrib#34035 and also #3120.

How do you want to go about merging this change, then? Can we split out a part of it that doesn't depend on these changes and merge it? I'm also ok with waiting for them, but it may be onerous for you to keep this PR rebased against the operator main branch.

ItielOlenick · 2024-07-30T15:14:51Z

@swiatekm anything missing?

We're still waiting for the change to prometheus receiver so that e2e tests here can pass, right?

Yes, waiting for open-telemetry/opentelemetry-collector-contrib#34035 and also #3120.

How do you want to go about merging this change, then? Can we split out a part of it that doesn't depend on these changes and merge it? I'm also ok with waiting for them, but it may be onerous for you to keep this PR rebased against the operator main branch.

I'm ok with splitting the test that relies on the e2e test image, but we do need open-telemetry/opentelemetry-collector-contrib#34035 as without it the collector will not be able to connect to the TA successfully over mTLS

ItielOlenick · 2024-08-19T11:59:58Z

@swiatekm Now that open-telemetry/opentelemetry-collector-contrib#34035 has been merged, can we get #3120 approved and merged?

pavolloffay · 2024-08-21T09:08:26Z

.chloggen/TA-update-configs-to-enable-mtls.yaml

+change_type: enhancement
+
+# The name of the component, or a single word describing the area of concern, (e.g. collector, target allocator, auto-instrumentation, opamp, github action)
+component: target allocator collector


I think this should be only target allocator

My thought is that since we are modifying the collector's configuration applied by the operator, it should be mentioned.

ItielOlenick and others added 30 commits May 13, 2024 21:44

Initial commit

69b4c63

Added Cert Manager CRDs & RBAC validation and management

684afbd

Added relevant resources and started adding tests

50d568a

[chore] move VineethReddy02 to emeritus (open-telemetry#2957)

dfc7882

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>

Fixed non-expected warnings on TA webhook. (open-telemetry#2962)

7fafce9

Signed-off-by: Yuri Sa <yurimsa@gmail.com>

Verify ServiceMonitor and PodMonitor are installed in prom cr availab…

53c5046

…ility check (open-telemetry#2964) * Verify ServiceMonitor and PodMonitor are installed in prom cr availability check * Added changelog

Prepare release 0.100.0 (open-telemetry#2960)

314d7c1

* Prepare release 0.100.0 Signed-off-by: Vineeth Pothulapati <vineethpothulapati@outlook.com> * update the chlog * update the chlog with open-telemetry#2877 merge --------- Signed-off-by: Vineeth Pothulapati <vineethpothulapati@outlook.com>

Bring back webhook port (open-telemetry#2973)

d1126a2

* add back webhook port * chlog

patch 0.100.1 (open-telemetry#2974)

435b900

Update the OpenTelemetry Java agent version to 2.4.0 (open-telemetry#…

2a9db5f

…2967)

simplify deletion logic (open-telemetry#2971)

eaa4e63

Update maintainers in the operator hub PR (open-telemetry#2977)

ac93c7e

Signed-off-by: Pavol Loffay <p.loffay@gmail.com>

Support for kubernetes 1.30 version (open-telemetry#2975)

1a731c9

* Support for kubernetes 1.30 version * Update makefile

[chore] Move TargetAllocator CRD to v1alpha1 (open-telemetry#2918)

cfc1dc3

[featuregate] Automatically set GOMEMLIMIT and GOMAXPROCS for collect…

cb7aaa8

…or, target allocator, opamp bridge (open-telemetry#2933) * set things * fix kustomize shim * restore, better chlog

Fix querying OpenShift user workload monitoring stack. (open-telemetr…

5326614

…y#2984)

ItielOlenick added 13 commits July 4, 2024 10:08

Merged main

0389c7c

Using TA params

c3eda01

Cleanup makefile from local changes

9c12441

Added step to create cert manager RBAC for e2e mtls tests

bea9947

Using Kustomize for patching certmanager permissions

e4ff571

Cleanup chainsaw test

612546e

Cleanup chainsaw tests

7dafe91

e2e test case verifying Collector got secret from TA over mTLS

73d0ce9

Added changelog, fixed unit tests

c392952

restored makefile

475a1e2

Renamed fg import

e293a78

Linting rules for imports

71ca0c0

Added more tests, updated the readme

09bdf3c

ItielOlenick mentioned this pull request Jul 6, 2024

TLS config in TA receiver doesn't take effect open-telemetry/opentelemetry-collector-contrib#33370

Closed

ItielOlenick marked this pull request as ready for review July 11, 2024 14:18

ItielOlenick requested review from a team as code owners July 11, 2024 14:18

swiatekm reviewed Jul 11, 2024

View reviewed changes

ItielOlenick and others added 2 commits July 11, 2024 20:55

Added steps in e2e tests for new app

fd99776

Merge branch 'main' into TA-update-configs-to-enable-mtls

deb8ff5

pavolloffay reviewed Aug 21, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ta update configs to enable mtls #3015

Ta update configs to enable mtls #3015

ItielOlenick commented Jun 6, 2024 •

edited

Loading

ItielOlenick commented Jul 5, 2024

swiatekm Jul 11, 2024

ItielOlenick Jul 11, 2024

swiatekm commented Jul 11, 2024

ItielOlenick commented Jul 11, 2024 •

edited

Loading

ItielOlenick commented Jul 26, 2024

swiatekm commented Jul 26, 2024

ItielOlenick commented Jul 26, 2024

swiatekm commented Jul 30, 2024

ItielOlenick commented Jul 30, 2024

ItielOlenick commented Aug 19, 2024

pavolloffay Aug 21, 2024

ItielOlenick Aug 21, 2024

Ta update configs to enable mtls #3015

Are you sure you want to change the base?

Ta update configs to enable mtls #3015

Conversation

ItielOlenick commented Jun 6, 2024 • edited Loading

ItielOlenick commented Jul 5, 2024

swiatekm Jul 11, 2024

Choose a reason for hiding this comment

ItielOlenick Jul 11, 2024

Choose a reason for hiding this comment

swiatekm commented Jul 11, 2024

ItielOlenick commented Jul 11, 2024 • edited Loading

ItielOlenick commented Jul 26, 2024

swiatekm commented Jul 26, 2024

ItielOlenick commented Jul 26, 2024

swiatekm commented Jul 30, 2024

ItielOlenick commented Jul 30, 2024

ItielOlenick commented Aug 19, 2024

pavolloffay Aug 21, 2024

Choose a reason for hiding this comment

ItielOlenick Aug 21, 2024

Choose a reason for hiding this comment

ItielOlenick commented Jun 6, 2024 •

edited

Loading

ItielOlenick commented Jul 11, 2024 •

edited

Loading