-
Notifications
You must be signed in to change notification settings - Fork 357
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MON-3934: Clean up and parallelize some e2e tests for faster runs #2397
base: master
Are you sure you want to change the base?
Conversation
machine424
commented
Jun 27, 2024
•
edited
Loading
edited
- I added CHANGELOG entry for this change.
- No user facing changes, so no entry in CHANGELOG was needed.
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: machine424 The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
@machine424: This pull request references MON-3934 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.17.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
/retest |
3871d3d
to
f010c91
Compare
f010c91
to
0355cf1
Compare
@machine424: This pull request references MON-3934 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.17.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
@@ -747,51 +752,6 @@ func TestAlertmanagerDisabling(t *testing.T) { | |||
}) | |||
} | |||
|
|||
func TestAlertManagerHasAdditionalAlertRelabelConfigs(t *testing.T) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
moved to TestMultinamespacePrometheusRule
where we're sure to get an alert with the appropriate labels.
err = framework.Poll(5*time.Second, time.Minute, func() error { | ||
// The uwm alerts port (9095) is only exposed in-cluster, so we need to use | ||
// port forwarding to access kube-rbac-proxy. | ||
host, cleanUp, err := f.ForwardPort(t, f.UserWorkloadMonitoringNs, "alertmanager-user-workload", 9095) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
inside the poll for better resiliency
} | ||
func assertCMOImageRegistryIsUsed(t *testing.T, ns string) { | ||
getRegistry := func(t *testing.T, image string) string { | ||
// This first attempt is needed; otherwise, we may blindly add a second scheme, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
make the test less error prone.
t.Cleanup(cleanUp) | ||
func verifyAlertmanagerReceivedAlerts(t *testing.T, namespace, svc string) { | ||
err := framework.Poll(time.Second, 5*time.Minute, func() error { | ||
host, cleanUp, err := f.ForwardPort(t, namespace, svc, 9093) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
inside the poll for better resiliency
} | ||
} | ||
|
||
func TestUserWorkloadMonitoringAlerting(t *testing.T) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
merged them in TestUserWorkloadMonitoring
to make use of the same setup and be able to run them in //
} | ||
} | ||
|
||
func TestUserWorkloadMonitoringOptOut(t *testing.T) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
merged them in TestUserWorkloadMonitoring
to make use of the same setup and be able to run them in //
This check will happen on a standalone namespace userWorkloadOptOutTestNs
so it can run in //
} | ||
} | ||
|
||
func TestUserWorkloadMonitoringGrpcSecrets(t *testing.T) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
merged them in TestUserWorkloadMonitoring
to make use of the same setup and be able to run them in //
@@ -436,6 +380,7 @@ func assertUserWorkloadMetrics(t *testing.T) { | |||
err = framework.Poll(5*time.Second, 5*time.Minute, func() error { | |||
body, err := f.AlertmanagerClient.GetAlertmanagerAlerts( | |||
"filter", `alertname="VersionAlert"`, | |||
"filter", fmt.Sprintf(`namespace="%s"`, userWorkloadTestNs), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we now create tow namespaces, see setupUserApplications
This also make the test better isolated and less error prone
f.ThanosQuerierClient.WaitForQueryReturn( | ||
t, time.Minute, fmt.Sprintf(`count(up{service="%s",namespace="openshift-user-workload-monitoring"} == 1)`, service), | ||
t, 5*time.Minute, fmt.Sprintf(`count(up{service="%s",namespace="openshift-user-workload-monitoring"} == 1)`, service), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This runs in parallel now, some checks that used to run at the end may need more time now.
…er runs Isolate specific tests to enable parallel execution Enhance the resilience of some tests and fix those prone to errors.
0355cf1
to
a1cd4c2
Compare
@machine424: This pull request references MON-3934 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.17.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
@@ -1488,7 +1488,7 @@ func (c *Client) CreateOrUpdateConfigMap(ctx context.Context, cm *v1.ConfigMap) | |||
return err | |||
} | |||
|
|||
func (c *Client) DeleteIfExists(ctx context.Context, nsName string) error { | |||
func (c *Client) DeleteNSIfExists(ctx context.Context, nsName string) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, this is not related to the PR :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(nit) the "IfExists" is implicit IMHO. Also the function could be moved to the framework package since it's not invoked by the operator.
func (c *Client) DeleteNSIfExists(ctx context.Context, nsName string) error { | |
func (c *Client) DeleteNamespace(ctx context.Context, nsName string) error { |
@machine424: This pull request references MON-3934 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.17.0" version, but no target version was set. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
@@ -109,15 +119,14 @@ func createPrometheusRule(t *testing.T) { | |||
} | |||
} | |||
|
|||
func verifyAlertmanagerAlertReceived(t *testing.T) { | |||
|
|||
host, cleanUp, err := f.ForwardPort(t, f.Ns, "alertmanager-operated", 9093) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was requesting the default AM instead of the additional one.
@machine424: The following test failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
@@ -1488,7 +1488,7 @@ func (c *Client) CreateOrUpdateConfigMap(ctx context.Context, cm *v1.ConfigMap) | |||
return err | |||
} | |||
|
|||
func (c *Client) DeleteIfExists(ctx context.Context, nsName string) error { | |||
func (c *Client) DeleteNSIfExists(ctx context.Context, nsName string) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(nit) the "IfExists" is implicit IMHO. Also the function could be moved to the framework package since it's not invoked by the operator.
func (c *Client) DeleteNSIfExists(ctx context.Context, nsName string) error { | |
func (c *Client) DeleteNamespace(ctx context.Context, nsName string) error { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd keep this change for another PR if you don't mind.
if err != nil { | ||
return err | ||
} | ||
t.Cleanup(cleanUp) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might leave many port forwards opened if there are retries?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The change here is huge. Can you split it to another PR?