Support NginxProxy CRD and global tracing settings #1870

sjberman · 2024-04-22T19:37:00Z

Problem: As a user of NGF
I want to set the collection point for my traces for my installation of NGF So that I can ensure all my traces are sent to the same collection platform.

Solution: Implement the NginxProxy CRD which contains the fields required to configure the collection point for tracing. This resource is attached to the GatewayClass. If the resource is not found, a condition will be set on the GatewayClass to indicate this. The GatewayClass will continue to be Accepted even if the parametersRef is invalid.

This configuration sets the http context-level otel directives. The otel module is loaded conditionally based on the existence of this configuration.

Note: tracing is not enabled by this configuration, this only sets high level options. #1828 is required to actually enable tracing on a per-route basis.

Testing: Manually verified that the otel configuration is populated in the nginx.conf when appropriate. Conditions are also properly set on the GatewayClass.

Closes #1827
Closes #1775

Checklist

Before creating a PR, run through this checklist and mark each as complete.

I have read the CONTRIBUTING doc
I have added tests that prove my fix is effective or that my feature works
I have checked that all unit tests pass after adding my changes
I have updated necessary documentation
I have rebased my branch onto main
I will ensure my PR is targeting the main branch and pulling from my branch from my own fork

Release notes

If this PR introduces a change that affects users and needs to be mentioned in the release notes,
please add a brief note that summarizes the change.

NginxProxy CRD added to configure global settings (such as tracing endpoint) at the GatewayClass level.

sjberman · 2024-04-22T20:00:51Z

Local OSS build works, so not sure about the pipeline failure...

pleshakov · 2024-04-22T21:03:48Z

@sjberman
looks like OTel module is not available for linux/ppc64le

sjberman · 2024-04-22T21:28:38Z

@pleshakov Or 390x it seems. I guess we could ping the core team about building the module for those architectures, but who knows when that would actually happen...

We may just have to build it ourselves.

pleshakov · 2024-04-22T21:30:49Z

We may just have to build it ourselves.

or drop those platforms...

sjberman · 2024-04-23T14:03:26Z

@lucacome What do you think? Do we know if these architectures are popular enough to keep support for them right now?

We could open an issue on the otel-module repo to support these, and we remove support until that point.

lucacome · 2024-04-23T22:30:00Z

It looks like it was a choice to only support amd64 and arm64 at least for the Docker image oxpa/docker-official-images@e23a9b5

I've decided to make a separate variant based on the main image instead
of extending it because the module build-depends on a fairly large chunk
of C++ code from multiple projects, which takes around 10 minutes to
compile and link on an 8-core amd64 machine. This is why it's currently
limited to amd64 and arm64v8, which nginx.org provides builds for.
Users can build them on less popular architectures as the instructions
are still provided in the dockerfiles.

Maybe they're going to add the other platforms in the future, we should probably talk to @thresheek

I'm not sure how popular these architectures are...do we have anything in our telemetry data?

build/Dockerfile.nginx

thresheek · 2024-04-23T22:37:13Z

It looks like it was a choice to only support amd64 and arm64 at least for the Docker image oxpa/docker-official-images@e23a9b5

Maybe they're going to add the other platforms in the future, we should probably talk to @thresheek

I'm not sure how popular these architectures are...do we have anything in our telemetry data?

Indeed, this was an explicit choice not to overload docker library machines with compiling code on less-than-popular architectures...

I mean we can technically extend this list, but in reality I'd like to see real users/customers demand for otel on s390x or ppc64...

thresheek · 2024-04-23T22:39:12Z

Oh and we definitely didnt build any alpine ppc64le/s390x packages on nginx.org, too

codecov · 2024-04-24T15:06:17Z

Codecov Report

Attention: Patch coverage is 99.24812% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 86.07%. Comparing base (c7fd089) to head (30df54e).

Files	Patch %	Lines
internal/mode/static/manager.go	50.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1870      +/-   ##
==========================================
- Coverage   86.12%   86.07%   -0.06%     
==========================================
  Files          83       85       +2     
  Lines        5586     5492      -94     
  Branches       50        0      -50     
==========================================
- Hits         4811     4727      -84     
+ Misses        726      716      -10     
  Partials       49       49

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

.goreleaser.yml

.github/workflows/ci.yml

deploy/helm-chart/README.md

internal/mode/static/nginx/conf/nginx.conf

internal/mode/static/state/change_processor.go

deploy/helm-chart/README.md

conformance/provisioner/static-deployment.yaml

deploy/helm-chart/templates/rbac.yaml

internal/mode/static/state/dataplane/configuration.go

internal/mode/static/state/dataplane/types.go

internal/mode/static/state/graph/gatewayclass.go

deploy/helm-chart/README.md

docs/proposals/gateway-settings.md

internal/mode/static/nginx/conf/nginx.conf

internal/mode/static/state/dataplane/types.go

internal/mode/static/state/graph/gatewayclass.go

charts/nginx-gateway-fabric/templates/deployment.yaml

internal/mode/static/nginx/config/validation/http_validator.go

internal/mode/static/state/dataplane/configuration_test.go

internal/mode/static/state/dataplane/types.go

internal/mode/static/state/graph/graph.go

internal/mode/static/state/change_processor_test.go

internal/mode/static/state/graph/gatewayclass.go

ADubhlaoich

LGTM docs-wise but the docs changes are minimal, so I will not give an approval.

kate-osborn

LGTM

internal/mode/static/nginx/config/validation/generic.go

internal/mode/static/nginx/config/telemetry_template.go

Problem: As a user of NGF I want to set the collection point for my traces for my installation of NGF So that I can ensure all my traces are sent to the same collection platform. Solution: Implement the NginxProxy CRD which contains the fields required to configure the collection point for tracing. This resource is attached to the GatewayClass. If the resource is not found, a condition will be set on the GatewayClass to indicate this. The GatewayClass will continue to be Accepted even if the parametersRef is invalid. This configuration sets the `http` context-level otel directives. The otel module is loaded conditionally based on the existence of this configuration. Note: tracing is not enabled by this configuration, this only sets high level options. #1828 is required to actually enable tracing on a per-route basis.

sjberman requested review from a team as code owners April 22, 2024 19:37

nginx-bot bot added the release-notes label Apr 22, 2024

github-actions bot added documentation Improvements or additions to documentation enhancement New feature or request helm-chart Relates to helm chart labels Apr 22, 2024

lucacome reviewed Apr 23, 2024

View reviewed changes

build/Dockerfile.nginx Outdated Show resolved Hide resolved

sjberman force-pushed the enh/nginx-proxy-crd branch from 343c671 to 2731246 Compare April 24, 2024 20:35

lucacome reviewed Apr 24, 2024

View reviewed changes

.goreleaser.yml Outdated Show resolved Hide resolved

pleshakov reviewed Apr 25, 2024

View reviewed changes

sjberman mentioned this pull request Apr 25, 2024

Client Settings Policy Attachment #1878

Closed

5 tasks

kate-osborn requested changes Apr 25, 2024

View reviewed changes

sjberman force-pushed the enh/nginx-proxy-crd branch from 2731246 to 99719fe Compare April 26, 2024 15:57

sjberman requested review from pleshakov and kate-osborn April 26, 2024 19:43

kate-osborn requested changes Apr 26, 2024

View reviewed changes

pleshakov reviewed Apr 26, 2024

View reviewed changes

internal/mode/static/state/change_processor_test.go Show resolved Hide resolved

internal/mode/static/state/graph/gatewayclass.go Show resolved Hide resolved

ADubhlaoich reviewed Apr 29, 2024

View reviewed changes

sjberman force-pushed the enh/nginx-proxy-crd branch from 2e10400 to 8397536 Compare April 29, 2024 15:01

sjberman requested review from kate-osborn and pleshakov April 29, 2024 15:25

kate-osborn approved these changes Apr 29, 2024

View reviewed changes

sjberman force-pushed the enh/nginx-proxy-crd branch from ba197c0 to 69f42f0 Compare April 29, 2024 18:31

pleshakov approved these changes Apr 29, 2024

View reviewed changes

internal/mode/static/nginx/config/validation/generic.go Outdated Show resolved Hide resolved

internal/mode/static/nginx/config/telemetry_template.go Outdated Show resolved Hide resolved

sjberman added 15 commits April 29, 2024 15:51

Remove unsupported architectures

a0e24b7

Quote string variables in conf

9c9f72a

Code review; add validation and helm template

ca72108

Revert arch removal

b994dc2

Fix nil check

219bccf

Code review

e7d953c

Remove validator

585375f

Revert reference check

1fbe4a6

More specific validators

7453b0b

Rebase and add test

5ef075d

Fix validation calls

43e79d8

Fix failing unit test

8081878

Fix test after rebase

879fa7b

Code review

30df54e

sjberman force-pushed the enh/nginx-proxy-crd branch from 6c9afbf to 30df54e Compare April 29, 2024 21:56

sjberman enabled auto-merge (squash) April 29, 2024 21:56

sjberman merged commit 7c3da8d into main Apr 29, 2024
40 checks passed

sjberman deleted the enh/nginx-proxy-crd branch April 29, 2024 22:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support NginxProxy CRD and global tracing settings #1870

Support NginxProxy CRD and global tracing settings #1870

sjberman commented Apr 22, 2024

sjberman commented Apr 22, 2024

pleshakov commented Apr 22, 2024

sjberman commented Apr 22, 2024 •

edited

Loading

pleshakov commented Apr 22, 2024

sjberman commented Apr 23, 2024

lucacome commented Apr 23, 2024

thresheek commented Apr 23, 2024

thresheek commented Apr 23, 2024 •

edited

Loading

codecov bot commented Apr 24, 2024 •

edited

Loading

ADubhlaoich left a comment

kate-osborn left a comment

Support NginxProxy CRD and global tracing settings #1870

Support NginxProxy CRD and global tracing settings #1870

Conversation

sjberman commented Apr 22, 2024

Checklist

Release notes

sjberman commented Apr 22, 2024

pleshakov commented Apr 22, 2024

sjberman commented Apr 22, 2024 • edited Loading

pleshakov commented Apr 22, 2024

sjberman commented Apr 23, 2024

lucacome commented Apr 23, 2024

thresheek commented Apr 23, 2024

thresheek commented Apr 23, 2024 • edited Loading

codecov bot commented Apr 24, 2024 • edited Loading

Codecov Report

ADubhlaoich left a comment

Choose a reason for hiding this comment

kate-osborn left a comment

Choose a reason for hiding this comment

sjberman commented Apr 22, 2024 •

edited

Loading

thresheek commented Apr 23, 2024 •

edited

Loading

codecov bot commented Apr 24, 2024 •

edited

Loading