Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix file settings service tests #115770

Merged
merged 12 commits into from
Oct 29, 2024

Conversation

n1v0lg
Copy link
Contributor

@n1v0lg n1v0lg commented Oct 28, 2024

This PR addresses some of the failure causes tracked under #115280 and #115725: the latch-await setup was rather convoluted and the move command not always correctly invoked in the right order. This PR cleans up latching by separating awaiting the first processing call (on start) from waiting on the subsequent call. Also, it makes writing the file more robust w.r.t. OS'es where atomic_move may not be available.

This should address failures around the timeout await, and the assertion failures around invoked methods tracked here:

#115725 (comment)

But will likely require another round of changes to address the failures to delete files.

Relates: #115280
Relates: #115725

@n1v0lg n1v0lg added >test Issues or PRs that are addressing/adding tests :Core/Infra/Settings Settings infrastructure and APIs test-windows Trigger CI checks on Windows auto-backport Automatically create backport pull requests when merged v8.16.0 v9.0.0 v8.17.0 labels Oct 28, 2024
@n1v0lg n1v0lg self-assigned this Oct 28, 2024
@n1v0lg n1v0lg changed the title Fix FileSettingsServiceTests Fix file settings service tests Oct 28, 2024
@n1v0lg
Copy link
Contributor Author

n1v0lg commented Oct 28, 2024

@elasticmachine update branch

@@ -223,40 +227,40 @@ public void testProcessFileChanges() throws Exception {
return null;
}).when(controller).process(any(), any(XContentParser.class), any(), any());

// we get three events: initial clusterChanged event, first write, second write
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was not a very robust setup... The two things we really care about is that processFileOnServiceStart gets called, and that processFileChanges and that we wait for these calls. I've restructured this test to use two separate latches, and do the file writes one at a time.

Files.writeString(tempFilePath, contents);
Files.move(tempFilePath, path, StandardCopyOption.ATOMIC_MOVE, StandardCopyOption.REPLACE_EXISTING);
}
private static void writeTestFile(Path path, String contents) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tweaking this to:

  • account for missing atomic_move functionality
  • make temp file removal more explicit

@n1v0lg n1v0lg marked this pull request as ready for review October 28, 2024 20:11
@n1v0lg n1v0lg requested a review from jfreden October 28, 2024 20:11
@elasticsearchmachine elasticsearchmachine added the Team:Core/Infra Meta label for core/infra team label Oct 28, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

Copy link
Contributor

@jfreden jfreden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Just some nits.


// wait for listener to be called (once for initial processing, once for subsequent update)
assertTrue(latch.await(20, TimeUnit.SECONDS));
assertTrue(changesOnStartLatch.await(20, TimeUnit.SECONDS));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Could use the safeAwait here to follow conventions in other tests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah it has 10 seconds baked in. it's obnoxious that we'd need to wait longer than that but a comment in this file makes me hesitant:

"// we need to wait a bit, on MacOS it may take up to 10 seconds for the Java watcher service to notice the file"

@n1v0lg n1v0lg added the auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) label Oct 29, 2024
@n1v0lg n1v0lg removed the test-windows Trigger CI checks on Windows label Oct 29, 2024
@n1v0lg
Copy link
Contributor Author

n1v0lg commented Oct 29, 2024

@elasticmachine update branch

@elasticsearchmachine elasticsearchmachine merged commit b868677 into elastic:main Oct 29, 2024
16 checks passed
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

The backport operation could not be completed due to the following error:

An unexpected error occurred when attempting to backport this PR.

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 115770

@n1v0lg n1v0lg deleted the fix-file-settings-test branch October 29, 2024 11:26
jfreden pushed a commit to jfreden/elasticsearch that referenced this pull request Nov 4, 2024
This PR addresses some of the failure causes tracked under
elastic#115280 and
elastic#115725: the latch-await
setup was rather convoluted and the move command not always correctly
invoked in the right order. This PR cleans up latching by separating
awaiting the first processing call (on start) from waiting on the
subsequent call. Also, it makes writing the file more robust w.r.t.
OS'es where `atomic_move` may not be available.

This should address failures around the timeout await, and the assertion
failures around invoked methods tracked here:

elastic#115725 (comment)

But will likely require another round of changes to address the failures
to delete files.

Relates: elastic#115280 
Relates: elastic#115725
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Automatically create backport pull requests when merged auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) backport pending :Core/Infra/Settings Settings infrastructure and APIs Team:Core/Infra Meta label for core/infra team >test Issues or PRs that are addressing/adding tests v8.16.0 v8.17.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants