Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Target Allocator implementation (Part 3 - Collector Enhancement for Prometheus Config Update) #389

Merged
merged 45 commits into from
Sep 15, 2021

Conversation

rsvarma95
Copy link
Member

@rsvarma95 rsvarma95 commented Aug 10, 2021

Description

This PR addresses issue # 60 in the wg-prometheus repo. This is part 3 of a 3 part update in which enhancements are presented to the OTel Operator to support the deployment and reconcile support for a target allocator resource. This update enables the collectors to use http_sd_config option to determine their scrape targets exposed by the target allocation service. Please refer to the Design Doc, PDF version for code readability for more details

It extends Part 1 and should be merged after Part 1 is complete.

Type of change

New feature & Enhancement

Testing

Implemented go unit tests as part of the new logic responsible for replacing Prometheus config.

cc @alolita @Aneurysm9 @alexperez52

alexperez52 and others added 30 commits July 8, 2021 15:56
…etween collector and targetallocator & Minor changes"

This reverts commit 367c623 - only the folder structure change
Rahul Varma added 6 commits August 10, 2021 21:50
commit a496825
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Wed Aug 11 14:53:36 2021 -0400

    Update api/v1alpha1/opentelemetrycollector_webhook.go

    Co-authored-by: Anthony Mirabella <a9@aneurysm9.com>

commit 3d37148
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Wed Aug 11 10:34:44 2021 -0700

    Added validation in webhook to validate prometheus config

commit 33cb042
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Wed Aug 11 07:16:45 2021 -0700

    Updated docker image to quay.io image

commit a7764b9
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Tue Aug 10 20:20:32 2021 -0700

    Added version management & Minor fixes
commit 98acb2f
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Thu Aug 12 05:35:53 2021 -0700

    Updated struct name & minor changes

commit a496825
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Wed Aug 11 14:53:36 2021 -0400

    Update api/v1alpha1/opentelemetrycollector_webhook.go

    Co-authored-by: Anthony Mirabella <a9@aneurysm9.com>

commit 3d37148
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Wed Aug 11 10:34:44 2021 -0700

    Added validation in webhook to validate prometheus config

commit 33cb042
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Wed Aug 11 07:16:45 2021 -0700

    Updated docker image to quay.io image

commit a7764b9
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Tue Aug 10 20:20:32 2021 -0700

    Added version management & Minor fixes
@rsvarma95 rsvarma95 marked this pull request as ready for review August 13, 2021 22:57
@rsvarma95
Copy link
Member Author

rsvarma95 commented Aug 13, 2021

Hi @Aneurysm9, @jpkrohling and @dashpole. Please review when you have time.

pkg/collector/reconcile/config_replace.go Show resolved Hide resolved
pkg/collector/reconcile/config_replace.go Outdated Show resolved Hide resolved
pkg/collector/reconcile/configmap.go Outdated Show resolved Hide resolved
@alolita
Copy link
Member

alolita commented Aug 18, 2021

Can you please re-review @Aneurysm9 @dashpole @jpkrohling ty!

@jpkrohling
Copy link
Member

I'll review this once the part 2 is merged.

@alolita
Copy link
Member

alolita commented Aug 26, 2021

@Aneurysm9 can you please review this PR too? Related to #354

@alolita
Copy link
Member

alolita commented Sep 14, 2021

Hi @Aneurysm9 let's get this PR approved and merged by @jpkrohling please. Thx.

@rsvarma95
Copy link
Member Author

Just wanted to mention that this can be reviewed independently and does not have any dependencies on Part 2.

@jpkrohling jpkrohling merged commit 7455eed into open-telemetry:main Sep 15, 2021
shree007 pushed a commit to shree007/opentelemetry-operator that referenced this pull request Dec 12, 2021
…rometheus Config Update) (open-telemetry#389)

* load balancer deployment implementation & controller & e2e tests

* Load balancer crd update, reconcile logic and e2e tests

* Reset last commit and only uploading changes to main.go

* Updated bundle and api with new autogenerated resources

* Updated code with lint fixes

* Updated code to include minor fixes

* Update helper.go to add header

* Updated comment and bundles

* Updated lb reconcile helper function and its invocations

* Updated naming scheme and changed CR config

* Updated bundle files

* Minor changes

* Lint and default file fixes

* Added rolebinding to automate manual setting

* Update opentelemetry-operator.clusterserviceversion.yaml

* Removed role/rolebinding files & Minor changes

* Minor changes

* Added error check in configmap reconcile & spelling correction

* Updated target allocator KUTTL tests & renamed folder

* Updated folder structure to reduce code duplication between collector and targetallocator & Minor changes

* Updated kuttl tests to use namespace-scope resources

* Revert part of "Updated folder structure to reduce code duplication between collector and targetallocator & Minor changes"

This reverts commit 367c623 - only the folder structure change

* Removed separate controller for target allocation

* Added additional label for collector pod selection and removed test-step

* Minor changes

* Added collector enhancement for prometheus config update

* Update config_replace.go

* Squashed commit of the following:

commit a7764b9
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Tue Aug 10 20:20:32 2021 -0700

    Added version management & Minor fixes

* Updated port and protocol

* Lint fixes

* Removed unnecessary structs

* Squashed commit of the following:

commit a496825
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Wed Aug 11 14:53:36 2021 -0400

    Update api/v1alpha1/opentelemetrycollector_webhook.go

    Co-authored-by: Anthony Mirabella <a9@aneurysm9.com>

commit 3d37148
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Wed Aug 11 10:34:44 2021 -0700

    Added validation in webhook to validate prometheus config

commit 33cb042
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Wed Aug 11 07:16:45 2021 -0700

    Updated docker image to quay.io image

commit a7764b9
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Tue Aug 10 20:20:32 2021 -0700

    Added version management & Minor fixes

* Updated structure to reflect new changes in first part

* Squashed commit of the following:

commit 98acb2f
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Thu Aug 12 05:35:53 2021 -0700

    Updated struct name & minor changes

commit a496825
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Wed Aug 11 14:53:36 2021 -0400

    Update api/v1alpha1/opentelemetrycollector_webhook.go

    Co-authored-by: Anthony Mirabella <a9@aneurysm9.com>

commit 3d37148
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Wed Aug 11 10:34:44 2021 -0700

    Added validation in webhook to validate prometheus config

commit 33cb042
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Wed Aug 11 07:16:45 2021 -0700

    Updated docker image to quay.io image

commit a7764b9
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Tue Aug 10 20:20:32 2021 -0700

    Added version management & Minor fixes

* Merged with main branch

* Minor changes

* Updated go.sum

* Updated go.mod

Co-authored-by: Alexis Perez <perzmen@amazon.com>
Co-authored-by: Rahul Varma <rahvarm@amazon.com>
ItielOlenick pushed a commit to ItielOlenick/opentelemetry-operator that referenced this pull request May 1, 2024
…rometheus Config Update) (open-telemetry#389)

* load balancer deployment implementation & controller & e2e tests

* Load balancer crd update, reconcile logic and e2e tests

* Reset last commit and only uploading changes to main.go

* Updated bundle and api with new autogenerated resources

* Updated code with lint fixes

* Updated code to include minor fixes

* Update helper.go to add header

* Updated comment and bundles

* Updated lb reconcile helper function and its invocations

* Updated naming scheme and changed CR config

* Updated bundle files

* Minor changes

* Lint and default file fixes

* Added rolebinding to automate manual setting

* Update opentelemetry-operator.clusterserviceversion.yaml

* Removed role/rolebinding files & Minor changes

* Minor changes

* Added error check in configmap reconcile & spelling correction

* Updated target allocator KUTTL tests & renamed folder

* Updated folder structure to reduce code duplication between collector and targetallocator & Minor changes

* Updated kuttl tests to use namespace-scope resources

* Revert part of "Updated folder structure to reduce code duplication between collector and targetallocator & Minor changes"

This reverts commit 367c623 - only the folder structure change

* Removed separate controller for target allocation

* Added additional label for collector pod selection and removed test-step

* Minor changes

* Added collector enhancement for prometheus config update

* Update config_replace.go

* Squashed commit of the following:

commit a7764b9
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Tue Aug 10 20:20:32 2021 -0700

    Added version management & Minor fixes

* Updated port and protocol

* Lint fixes

* Removed unnecessary structs

* Squashed commit of the following:

commit a496825
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Wed Aug 11 14:53:36 2021 -0400

    Update api/v1alpha1/opentelemetrycollector_webhook.go

    Co-authored-by: Anthony Mirabella <a9@aneurysm9.com>

commit 3d37148
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Wed Aug 11 10:34:44 2021 -0700

    Added validation in webhook to validate prometheus config

commit 33cb042
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Wed Aug 11 07:16:45 2021 -0700

    Updated docker image to quay.io image

commit a7764b9
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Tue Aug 10 20:20:32 2021 -0700

    Added version management & Minor fixes

* Updated structure to reflect new changes in first part

* Squashed commit of the following:

commit 98acb2f
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Thu Aug 12 05:35:53 2021 -0700

    Updated struct name & minor changes

commit a496825
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Wed Aug 11 14:53:36 2021 -0400

    Update api/v1alpha1/opentelemetrycollector_webhook.go

    Co-authored-by: Anthony Mirabella <a9@aneurysm9.com>

commit 3d37148
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Wed Aug 11 10:34:44 2021 -0700

    Added validation in webhook to validate prometheus config

commit 33cb042
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Wed Aug 11 07:16:45 2021 -0700

    Updated docker image to quay.io image

commit a7764b9
Author: Rahul Varma <rahvarm@amazon.com>
Date:   Tue Aug 10 20:20:32 2021 -0700

    Added version management & Minor fixes

* Merged with main branch

* Minor changes

* Updated go.sum

* Updated go.mod

Co-authored-by: Alexis Perez <perzmen@amazon.com>
Co-authored-by: Rahul Varma <rahvarm@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants