Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add functional test for product telemetry #1659

Merged
merged 4 commits into from
Mar 13, 2024
Merged

Add functional test for product telemetry #1659

merged 4 commits into from
Mar 13, 2024

Conversation

pleshakov
Copy link
Contributor

Proposed changes

Problem:
Ensure product telemetry feature is tested with a functional test

Solution:

  • Add a functional test.
  • Because it requires a NGF with a custom built, it only runs when ginkgo runs with its label.

Testing:
Ran successfully:

  • make test TAG=$(whoami) GINKGO_LABEL=telemetry PLUS_ENABLED=true
  • make test TAG=$(whoami) GINKGO_LABEL=telemetry
  • make test TAG=$(whoami) # here telemetry test was skipped, but all the rest ran successfully

ClOSES - #1640

Checklist

Before creating a PR, run through this checklist and mark each as complete.

  • I have read the CONTRIBUTING doc
  • I have added tests that prove my fix is effective or that my feature works
  • I have checked that all unit tests pass after adding my changes
  • I have updated necessary documentation
  • I have rebased my branch onto main
  • I will ensure my PR is targeting the main branch and pulling from my branch from my own fork

Release notes

If this PR introduces a change that affects users and needs to be mentioned in the release notes,
please add a brief note that summarizes the change.

NONE

@pleshakov pleshakov requested a review from a team as a code owner March 8, 2024 23:06
@github-actions github-actions bot added documentation Improvements or additions to documentation tests Pull requests that update tests labels Mar 8, 2024
salonichf5
salonichf5 approved these changes Mar 9, 2024
View reviewed changes
Copy link
Contributor

@salonichf5 salonichf5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

sjberman
sjberman reviewed Mar 11, 2024
View reviewed changes
tests/Makefile Outdated Show resolved Hide resolved
tests/suite/telemetry_test.go Outdated Show resolved Hide resolved
tests/suite/telemetry_test.go Outdated Show resolved Hide resolved
tests/suite/telemetry_test.go Outdated Show resolved Hide resolved
tests/suite/telemetry_test.go Outdated Show resolved Hide resolved
@pleshakov pleshakov mentioned this pull request Mar 11, 2024
Closed
@pleshakov
Copy link
Contributor Author

@sjberman
had to squash feedback-related changes together with the rest before resolving the conflicts with the main branch

I also removed the Skip statements, to follow the updated code base in 15bab00

@pleshakov pleshakov requested a review from sjberman March 11, 2024 19:39
@sjberman
Copy link
Contributor

Depending on if this or #1634 goes in first, we'll need to make an update to ensure that this is run in the pipeline.

@pleshakov
Copy link
Contributor Author

now that #1634 is merged, I'll rebase and ensure that the added test is run in the pipeline

pleshakov and others added 3 commits March 13, 2024 09:40
@pleshakov
Add functional test for product telemetry
e332008 
Problem:
Ensure product telemetry feature is tested with a functional test

Solution:
- Add a functional test.
- Because it requires a NGF with a custom built, it needs to run with
telemetry label.

Testing:
Ran successfully:
- make test TAG=$(whoami) GINKGO_LABEL=telemetry
- make test TAG=$(whoami) # telemetry test didn't run as expected,
  the functional test succeeded

ClOSES - nginxinc#1640
@pleshakov @sjberman
Update tests/suite/telemetry_test.go
d43eedb 
Co-authored-by: Saylor Berman <s.berman@f5.com>
@pleshakov
Enable telemetry tests in pipeline
c6aa8ae
ciarams87
ciarams87 approved these changes Mar 13, 2024
View reviewed changes
Copy link
Member

@ciarams87 ciarams87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍🏼

@kate-osborn
Copy link
Contributor

@pleshakov can we use ginkgo to run the functional tests instead of go test?

@pleshakov
Copy link
Contributor Author

@kate-osborn

@pleshakov can we use ginkgo to run the functional tests instead of go test?

yep, probably

but changing to ginko would require re-testing not only telemetry but nfrs and longevity (to make sure that will not break them), so I'm a bit hesitant to do it as part of this PR. Would a separate issue work?

nginx-gateway-fabric/tests/Makefile

Lines 87 to 109 in 746abad

.PHONY: start-longevity-test
start-longevity-test: ## Start the longevity test to run for 4 days in GKE
START_LONGEVITY=true $(MAKE) nfr-test
.PHONY: stop-longevity-test
stop-longevity-test: ## Stops the longevity test and collects results
STOP_LONGEVITY=true $(MAKE) nfr-test
.PHONY: .vm-nfr-test
.vm-nfr-test: ## Runs the NFR tests on the GCP VM (called by `nfr-test`)
go test -v ./suite -ginkgo.label-filter "nfr" $(GINKGO_FLAGS) -ginkgo.v -args --gateway-api-version=$(GW_API_VERSION) \
--gateway-api-prev-version=$(GW_API_PREV_VERSION) --image-tag=$(TAG) --version-under-test=$(NGF_VERSION) \
--plus-enabled=$(PLUS_ENABLED) --ngf-image-repo=$(PREFIX) --nginx-image-repo=$(NGINX_PREFIX) \
--pull-policy=$(PULL_POLICY) --k8s-version=$(K8S_VERSION) --service-type=$(GW_SERVICE_TYPE) \
--is-gke-internal-lb=$(GW_SVC_GKE_INTERNAL)
.PHONY: test
test: ## Runs the functional tests on your default k8s cluster
go test -v ./suite -ginkgo.label-filter "functional" $(GINKGO_FLAGS) -args --gateway-api-version=$(GW_API_VERSION) \
--gateway-api-prev-version=$(GW_API_PREV_VERSION) --image-tag=$(TAG) --version-under-test=$(NGF_VERSION) \
--plus-enabled=$(PLUS_ENABLED) --ngf-image-repo=$(PREFIX) --nginx-image-repo=$(NGINX_PREFIX) \
--pull-policy=$(PULL_POLICY) --k8s-version=$(K8S_VERSION) --service-type=$(GW_SERVICE_TYPE) \
--is-gke-internal-lb=$(GW_SVC_GKE_INTERNAL)

@kate-osborn
Copy link
Contributor

@kate-osborn

@pleshakov can we use ginkgo to run the functional tests instead of go test?

yep, probably

but changing to ginko would require re-testing not only telemetry but nfrs and longevity (to make sure that will not break them), so I'm a bit hesitant to do it as part of this PR. Would a separate issue work?

nginx-gateway-fabric/tests/Makefile

Lines 87 to 109 in 746abad

.PHONY: start-longevity-test
start-longevity-test: ## Start the longevity test to run for 4 days in GKE
START_LONGEVITY=true $(MAKE) nfr-test
.PHONY: stop-longevity-test
stop-longevity-test: ## Stops the longevity test and collects results
STOP_LONGEVITY=true $(MAKE) nfr-test
.PHONY: .vm-nfr-test
.vm-nfr-test: ## Runs the NFR tests on the GCP VM (called by `nfr-test`)
go test -v ./suite -ginkgo.label-filter "nfr" $(GINKGO_FLAGS) -ginkgo.v -args --gateway-api-version=$(GW_API_VERSION) \
--gateway-api-prev-version=$(GW_API_PREV_VERSION) --image-tag=$(TAG) --version-under-test=$(NGF_VERSION) \
--plus-enabled=$(PLUS_ENABLED) --ngf-image-repo=$(PREFIX) --nginx-image-repo=$(NGINX_PREFIX) \
--pull-policy=$(PULL_POLICY) --k8s-version=$(K8S_VERSION) --service-type=$(GW_SERVICE_TYPE) \
--is-gke-internal-lb=$(GW_SVC_GKE_INTERNAL)
.PHONY: test
test: ## Runs the functional tests on your default k8s cluster
go test -v ./suite -ginkgo.label-filter "functional" $(GINKGO_FLAGS) -args --gateway-api-version=$(GW_API_VERSION) \
--gateway-api-prev-version=$(GW_API_PREV_VERSION) --image-tag=$(TAG) --version-under-test=$(NGF_VERSION) \
--plus-enabled=$(PLUS_ENABLED) --ngf-image-repo=$(PREFIX) --nginx-image-repo=$(NGINX_PREFIX) \
--pull-policy=$(PULL_POLICY) --k8s-version=$(K8S_VERSION) --service-type=$(GW_SERVICE_TYPE) \
--is-gke-internal-lb=$(GW_SVC_GKE_INTERNAL)

yep, separate issue makes sense here.

@pleshakov
Copy link
Contributor Author

@kate-osborn
#1677

@pleshakov pleshakov merged commit f059cf0 into nginxinc:main Mar 13, 2024
40 checks passed
@pleshakov pleshakov deleted the test/telemetry-functional branch March 13, 2024 16:20
amimimor pushed a commit to amimimor/nginx-gateway-fabric that referenced this pull request Apr 3, 2024
@pleshakov @sjberman @amimimor
Add functional test for product telemetry (nginxinc#1659)
f5ecd14 
* Add functional test for product telemetry

Problem:
Ensure product telemetry feature is tested with a functional test

Solution:
- Add a functional test.
- Because it requires a NGF with a custom built, it needs to run with
telemetry label.
- Enable telemetry test in the pipeline

Testing:
Ran successfully:
- make test TAG=$(whoami) GINKGO_LABEL=telemetry
- make test TAG=$(whoami) # telemetry test didn't run as expected,
  the functional test succeeded
Also, the pipeline runs telemetry tests successfully

ClOSES - nginxinc#1640

Co-authored-by: Saylor Berman <s.berman@f5.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lucacome lucacome lucacome approved these changes

@sjberman sjberman sjberman approved these changes

@ciarams87 ciarams87 ciarams87 approved these changes

@kate-osborn kate-osborn kate-osborn approved these changes

@salonichf5 salonichf5 salonichf5 approved these changes

Assignees
No one assigned
Labels
documentation Improvements or additions to documentation tests Pull requests that update tests
Projects
NGINX Gateway Fabric
Status: ✅ Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@pleshakov @sjberman @kate-osborn @lucacome @ciarams87 @salonichf5