Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Force conformance tests to fail properly #1786

Merged
merged 1 commit into from
Mar 28, 2024

Conversation

sjberman
Copy link
Contributor

Problem: If the conformance test Pod runner failed, the make process would still pass. This is because the failure code would not propagate to outside of the Pod.

Solution: After the Pod runs, check the exit code and fail it it's non-zero. Used a bash script for this because running the kubectl get pod command in the Makefile would not return an Errored Pod for some reason. Only worked in bash.

Closes #1784

Checklist

Before creating a PR, run through this checklist and mark each as complete.

  • I have read the CONTRIBUTING doc
  • I have added tests that prove my fix is effective or that my feature works
  • I have checked that all unit tests pass after adding my changes
  • I have updated necessary documentation
  • I have rebased my branch onto main
  • I will ensure my PR is targeting the main branch and pulling from my branch from my own fork

Release notes

If this PR introduces a change that affects users and needs to be mentioned in the release notes,
please add a brief note that summarizes the change.

NONE

Problem: If the conformance test Pod runner failed, the `make` process would still pass. This is because the failure code would not propagate to outside of the Pod.

Solution: After the Pod runs, check the exit code and fail it it's non-zero. Used a bash script for this because running the `kubectl get pod` command in the Makefile would not return an Errored Pod for some reason. Only worked in bash.
@sjberman sjberman requested a review from a team as a code owner March 28, 2024 14:23
@github-actions github-actions bot added the tests Pull requests that update tests label Mar 28, 2024
@sjberman sjberman added the conformance Relates to passing Gateway API conformance tests label Mar 28, 2024
@sjberman sjberman merged commit 03e24fe into nginxinc:main Mar 28, 2024
40 checks passed
@sjberman sjberman deleted the tests/fix-conformance branch March 28, 2024 22:03
amimimor pushed a commit to amimimor/nginx-gateway-fabric that referenced this pull request Apr 3, 2024
Problem: If the conformance test Pod runner failed, the `make` process would still pass. This is because the failure code would not propagate to outside of the Pod.

Solution: After the Pod runs, check the exit code and fail it it's non-zero. Used a bash script for this because running the `kubectl get pod` command in the Makefile would not return an Errored Pod for some reason. Only worked in bash.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
conformance Relates to passing Gateway API conformance tests tests Pull requests that update tests
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

make run-conformace-tests succeeds even when tests fail to run
3 participants