Add broker OIDC e2e tests #3685

creydr · 2024-02-09T10:09:30Z

Proposed Changes

🎁 Add OIDC e2e tests for Broker

The call in reconciler-tests.sh to run the OIDC tests was added in #3675 already:

eventing-kafka-broker/test/reconciler-tests.sh

Lines 57 to 61 in d6a383f

    
           echo "Running E2E Reconciler Tests with OIDC authentication enabled" 
        
           kubectl apply -Rf "$(dirname "$0")/config-oidc-authentication" 
        
           go_test_e2e -timeout=1h ./test/e2e_new -run OIDC || fail_test

codecov · 2024-02-09T10:13:09Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (d6a383f) 62.12% compared to head (77c9089) 62.04%.
Report is 4 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #3685      +/-   ##
============================================
- Coverage     62.12%   62.04%   -0.08%     
+ Complexity      859      858       -1     
============================================
  Files           189      189              
  Lines         12875    12901      +26     
  Branches        275      275              
============================================
+ Hits           7998     8004       +6     
- Misses         4245     4265      +20     
  Partials        632      632

Flag	Coverage Δ
java-unittests	`74.47% <ø> (-0.13%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

creydr · 2024-02-09T10:34:18Z

/retest-required

creydr · 2024-02-09T10:59:50Z

/retest-required

CI seems to have a bad time ATM...

pierDipi

/lgtm
/approve

If tests pass

creydr · 2024-02-09T11:57:42Z

/test upgrade-tests

creydr · 2024-02-09T12:08:41Z

/test reconciler-tests

creydr · 2024-02-09T12:24:19Z

/test reconciler-tests-namespaced-broker

creydr · 2024-02-09T12:38:36Z

/test upgrade-tests

creydr · 2024-02-09T13:08:48Z

/test reconciler-tests

creydr · 2024-02-09T13:40:31Z

/test reconciler-tests-namespaced-broker

creydr · 2024-02-09T14:40:17Z

/test reconciler-tests

creydr · 2024-02-09T14:40:37Z

/test reconciler-tests-namespaced-broker

creydr · 2024-02-09T15:44:21Z

/test reconciler-tests-namespaced-broker

creydr · 2024-02-09T15:44:35Z

/test reconciler-tests

creydr · 2024-02-09T16:57:41Z

/test reconciler-tests

creydr · 2024-02-09T19:14:48Z

/retest-required

creydr · 2024-02-09T21:13:30Z

/retest-required

creydr · 2024-02-09T23:47:16Z

/retest-required

…sink

creydr · 2024-02-10T13:33:45Z

OIDC tests were failing, because the kafka broker template did not contain a field for CACerts & audience spec in its DLS config. Added it in 77c9089.

@pierDipi @Leo6Leo @Cali0707 could you recheck? :)

Cali0707

/lgtm

knative-prow · 2024-02-10T15:19:16Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Cali0707, creydr, pierDipi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [Cali0707,creydr,pierDipi]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

* Add broker OIDC e2e tests * Fix broker template to allow TLS & OIDC configuration on dead letter sink

* Update contract with fields for OIDC information (knative-extensions#3632) * Update contract to include OIDC information * Run hack/update-codegen.sh * Move OIDC SA to egress * Expose OIDC audience of KafkaChannel in its status (knative-extensions#3622) * Provision .status.address.audience and .status.addresses[*].audience in KafkaChannel * Add kafka Channel e2e test to check if audience is provisioned * Run goimport * Update deps * Auto generate Triggers OIDC identity service account and expose in its status (knative-extensions#3604) * Support auto generation of Triggers identity service account and expose in AuthStatus Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * make ServiceAccountLister public Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * add oidc unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * integrate oidc unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * add some logic to reconcile triggers, if the features config map gets updated Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * revert vendor/knative.dev/pkg/webhook/resourcesemantics/defaulting/controller.go Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix unit test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix Verify Deps and Codegen test Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * fix unit test and reconcile triggers, in case of the features configmap changes Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> --------- Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> * Create KafkaSources OIDC service account and expose in its status (knative-extensions#3660) * Create KafkaSources OIDC service account and expose in its status * Run goimport * Provision contract with OIDC information (knative-extensions#3646) * Provision contract with OIDC information * Add DLS audience in KafkaChannel CRD * Update KafkaSource to expose its sinks audience in status * Update Trigger test to include OIDC SA in contract * Propagate KafkaSources OIDC serviceAccountName to consumer and consumergroup * Propagate triggerv2s serviceAccountName to consumergroup * Fix unit test * [data-plane] Add the caching for OIDC JWT token to tokenprovider (knative-extensions#3663) * Add the caching to tokenprovider * Add the boiler * Running the codegen * Running the codegen * Revert "Running the codegen" This reverts commit 0ccf69c. * Use constant, set buffer before token expire, remove unnecessary change * Codegen changes * Codegen changes * Update data-plane/core/src/main/java/dev/knative/eventing/kafka/broker/core/oidc/TokenProvider.java Co-authored-by: Christoph Stäbler <cstabler@redhat.com> * Fix the review comments * Run codegen * Codegen changes * Code gen again --------- Co-authored-by: Christoph Stäbler <cstabler@redhat.com> * Receiver reject requests for wrong audience (knative-extensions#3675) * Receiver: reject request for wrong audience * Switch to AuthenticationHandler * Fix "Request has already been read" issue * Change TokenVerifier to an interface * Initialize TokenVerifier in main * Add test for AuthenticationHandler * Only initialize OIDC discovery config in main and create a TokenVerifier per verticle instance. * Rerun hack/update-codegen.sh * Move TokenVerifier setup into setup() to prevent null pointer exception when vertx is null * Update KafkaChannel OIDC e2e tests, to run OIDC conformance tests so the receiver is tested too. * Run OIDC e2e tests as part of the reconciler suite * Fix KafkaChannelOIDC e2e test * Fix lint issue * Address review comments * Dispatcher authenticate requests (knative-extensions#3677) * Change TokenProvider to return future to get a token * Dispatcher add OIDC to token, when target has an audience set * Add e2e test * Support exposing the Audience of a Broker (knative-extensions#3600) * Support exposing the Audience of a Broker * fix formatting * fix formatting * test fixes * Populate broker.status.addresses[*].audience field too * Run goimports and gofmt * Fix unit test --------- Co-authored-by: Christoph Stäbler <cstabler@redhat.com> * Add broker OIDC e2e tests (knative-extensions#3685) * Add broker OIDC e2e tests * Fix broker template to allow TLS & OIDC configuration on dead letter sink * Remove unneeded check when setting broker audience (knative-extensions#3708) * Check status code of OIDC discovery response (knative-extensions#3707) * Check status code of OIDC discovery endpoint * Run update-codegen.sh * Only allow 200 status code on OIDC discovery endpoint * Run update-deps.sh * Add OIDC tests to encryption/auth test suite * run make generate-release * TokenVerifier: execute blocking calls in parallel (knative-extensions#3728) * TokenVerifier: execute blocking calls in parallel * Revert "TokenVerifier: execute blocking calls in parallel" This reverts commit f3dbde9. * Revert: removed changes in contract.pb.go * Cancel receiver pod start on invalid OIDC config only if authentication.oidc is enabled (knative-extensions#3761) * Cancel pod start on invalid OIDC config only if authentication.oidc is enabled * Update namespaced broker to copy features configmap too. * Add unit test for FeaturesConfig class (knative-extensions#3771) * Add unit test for FeaturesConfig class * Update data-plane/core/src/test/java/dev/knative/eventing/kafka/broker/core/features/FeaturesConfigTest.java Co-authored-by: Calum Murray <cmurray@redhat.com> --------- Co-authored-by: Calum Murray <cmurray@redhat.com> * Run make generate-release again --------- Signed-off-by: pingjiang <xiangpingjiang1998@gmail.com> Co-authored-by: cola <45722758+xiangpingjiang@users.noreply.github.com> Co-authored-by: Leo Li <leoli@redhat.com> Co-authored-by: Gunish Matta <33680363+gunishmatta@users.noreply.github.com> Co-authored-by: Partha Ghosh <112557191+parth721@users.noreply.github.com> Co-authored-by: Calum Murray <cmurray@redhat.com>

Add broker OIDC e2e tests

8c2da33

knative-prow bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. area/test labels Feb 9, 2024

knative-prow bot requested review from lionelvillard and matzew February 9, 2024 10:09

knative-prow bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 9, 2024

pierDipi approved these changes Feb 9, 2024

View reviewed changes

knative-prow bot assigned pierDipi Feb 9, 2024

knative-prow bot added the lgtm Indicates that a PR is ready to be merged. label Feb 9, 2024

Fix broker template to allow TLS & OIDC configuration on dead letter …

77c9089

…sink

knative-prow bot removed the lgtm Indicates that a PR is ready to be merged. label Feb 10, 2024

Cali0707 approved these changes Feb 10, 2024

View reviewed changes

knative-prow bot assigned Cali0707 Feb 10, 2024

knative-prow bot added the lgtm Indicates that a PR is ready to be merged. label Feb 10, 2024

knative-prow bot merged commit eb55b98 into knative-extensions:main Feb 10, 2024
37 checks passed

creydr added a commit to creydr/knative-eventing-kafka-broker that referenced this pull request Feb 21, 2024

Add broker OIDC e2e tests (knative-extensions#3685)

258ba56

* Add broker OIDC e2e tests * Fix broker template to allow TLS & OIDC configuration on dead letter sink

creydr added a commit to creydr/knative-eventing-kafka-broker that referenced this pull request Mar 18, 2024

Add broker OIDC e2e tests (knative-extensions#3685)

a7875e3

* Add broker OIDC e2e tests * Fix broker template to allow TLS & OIDC configuration on dead letter sink

creydr added a commit to creydr/knative-eventing-kafka-broker that referenced this pull request Apr 2, 2024

Add broker OIDC e2e tests (knative-extensions#3685)

ec3ad81

* Add broker OIDC e2e tests * Fix broker template to allow TLS & OIDC configuration on dead letter sink

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add broker OIDC e2e tests #3685

Add broker OIDC e2e tests #3685

creydr commented Feb 9, 2024

codecov bot commented Feb 9, 2024 •

edited

Loading

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024 •

edited

Loading

pierDipi left a comment

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 10, 2024

Cali0707 left a comment

knative-prow bot commented Feb 10, 2024

	echo "Running E2E Reconciler Tests with OIDC authentication enabled"

	kubectl apply -Rf "$(dirname "$0")/config-oidc-authentication"

	go_test_e2e -timeout=1h ./test/e2e_new -run OIDC \|\| fail_test

Add broker OIDC e2e tests #3685

Add broker OIDC e2e tests #3685

Conversation

creydr commented Feb 9, 2024

Proposed Changes

codecov bot commented Feb 9, 2024 • edited Loading

Codecov Report

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024 • edited Loading

pierDipi left a comment

Choose a reason for hiding this comment

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 9, 2024

creydr commented Feb 10, 2024

Cali0707 left a comment

Choose a reason for hiding this comment

knative-prow bot commented Feb 10, 2024

codecov bot commented Feb 9, 2024 •

edited

Loading

creydr commented Feb 9, 2024 •

edited

Loading