-
Notifications
You must be signed in to change notification settings - Fork 211
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tests: skip update portions of the test if we don't have an update configured. #2281
tests: skip update portions of the test if we don't have an update configured. #2281
Conversation
/assign @maqiuyujoyce - should fix the issue we saw in our periodic testing |
@justinsb: GitHub didn't allow me to assign the following users: testing, -, we, periodic, saw, in, our, should, fix, the, issue. Note that only GoogleCloudPlatform members with read permissions, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/lgtm
I'm okay with not requiring update here and requiring test coverage via other approaches later.
@@ -368,7 +372,7 @@ func testUpdate(ctx context.Context, t *testing.T, testContext testrunner.TestCo | |||
// Update resource from test data | |||
updateUnstruct := testContext.UpdateUnstruct.DeepCopy() | |||
if updateUnstruct == nil { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As we've added the check above, is this check still needed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point; removed it!
…nfigured. This may enable us to stop requiring SkipUpdate in future.
17f222d
to
9431783
Compare
if updateUnstruct == nil { | ||
t.Fatalf("updateUnstruct is nil for '%v'. should SkipUpdate be set to true in resourcefixture/contexts?", testContext.ResourceFixture.Name) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems like there's a plan to move away from SkipUpdate
so disregard if this point will become moot in the future but as I onboarded on the code base, what I liked about this failure is that it gave me a strong signal for how to set up the fixtures tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't know if there's a plan per-se :-) I think the information in SkipUpdate is redundant - if we have an update.yaml then SkipUpdate must be true. So that's why I don't want to rely on it.
I do think we want tests for coverage e.g. does every kind have a create & update test, is every field in a kind touched in at least one test, are all mutable fields updated in at least one test etc. But I think those should be explicit tests (and they're more like apilinters, in that they are testing our tests, not testing the controllers itself)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We probably need a better way to ensure coverage. This change isn't really part of it.
Regarding testdata, if we do want to enforce the existence of update.yaml, we probably should do it via a unit test but not the integration test.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/lgtm
/approve
As long as we have the future plan to ensure update coverage, I'm good with it.
if updateUnstruct == nil { | ||
t.Fatalf("updateUnstruct is nil for '%v'. should SkipUpdate be set to true in resourcefixture/contexts?", testContext.ResourceFixture.Name) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We probably need a better way to ensure coverage. This change isn't really part of it.
Regarding testdata, if we do want to enforce the existence of update.yaml, we probably should do it via a unit test but not the integration test.
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: maqiuyujoyce The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
9d22e19
into
GoogleCloudPlatform:master
This may enable us to stop requiring SkipUpdate in future.