📖 Define testing guidelines and standards #3349
Conversation
k8s-ci-robot
added
the
size/L
|
/hold |
k8s-ci-robot
added
the
do-not-merge/hold
|
/retest |
|
Unknown CLA label state. Rechecking for CLA labels. Send feedback to sig-testing, kubernetes/test-infra and/or fejta. |
k8s-ci-robot
added
the
cncf-cla: yes
There was a problem hiding this comment.
Just added some notes. Overall I'm ok with using envtest since fakeclient is being deprecated (sigh...) and I'd prefer to not use things that are out of date and not maintained.
My concern with using envtest for unit tests is the possible increase of time it takes to run all the test suites. Are we going to have an envtest spun up per package or TestBlock?
Also, do we currently run our tests in parallel in CI? Sharing instances of envtest will definitely add more overhead to contributors to ensure they are creating and cleaning up resources correctly.
I'm working on writing a unit test using envtest and have some comments regarding cleanup of resources, etc...
#3350
docs/book/src/developer/testing.md
Outdated
| Please note that, because [envtest] uses a real kube-apiserver that is shared across many tests, the developer | ||
| should take care of ensuring each test run in isolation from the others, by: | ||
|
|
||
| - Creating objects in separated namespaces. | ||
| - Avoiding object name conflict. | ||
| - Cleaning up objects created by each test. |
There was a problem hiding this comment.
I'm guessing initially we will have an area of the codebase with tests written in the preferred style. This will probably result in creation of additional helpers that will help manage this overhead. These helpers should then be used everywhere else in order to provide a consistent test codebase.
There was a problem hiding this comment.
This makes totally sense
|
/milestone v0.3.9 |
|
/lgtm |
k8s-ci-robot
added
the
lgtm
|
Approval pending squash |
fabriziopandini
force-pushed
the
standardize-test
branch
from
81a0206
to
789abac
Compare
k8s-ci-robot
removed
the
lgtm
|
Commit squashed |
|
/lgtm |
k8s-ci-robot
added
the
lgtm
k8s-ci-robot
removed
the
do-not-merge/hold
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: vincepri The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
approved
What this PR does / why we need it:
Cluster API maintainers would like to improve consistency and maintainability of tests, in order to support the future growth of the project, avoid regressions and improve the developer experience and the overall quality of the code base.
This PR kick off this work following discussion on #3287 with the goal to reach an agreement at the community level about the direction we want to take for improving tests.
Which issue(s) this PR fixes:
Fixes #3287