Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Fix flaky receive/multitsdb test #7694

Merged

Conversation

cincinnat
Copy link
Contributor

@cincinnat cincinnat commented Sep 3, 2024

There is race condition in TestMultiTSDBPrune due to a dangling goroutine which can fail outside of the test function's lifetime if the database object is closed before Sync() is finished.

  • I added CHANGELOG entry for this change.
  • Change is not relevant to the end user.

Changes

Verification

Can be reliably reproduced with a simple change:

--- a/pkg/receive/multitsdb_test.go
+++ b/pkg/receive/multitsdb_test.go
@@ -502,6 +502,7 @@ func syncTSDBs(ctx context.Context, m *MultiTSDB, interval time.Duration) error
                case <-ctx.Done():
                        return nil
                case <-time.After(interval):
+                       time.Sleep(100 * time.Millisecond)
                        _, err := m.Sync(ctx)
                        if err != nil {
                                return err

There is race condition in `TestMultiTSDBPrune` due to a dangling goroutine
which can fail outside of the test function's lifetime if the database object
is closed before `Sync()` is finished.

Signed-off-by: Mikhail Nozdrachev <mikhail.nozdrachev@aiven.io>
Copy link
Contributor

@pedro-stanaka pedro-stanaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm - ty

Copy link
Contributor

@fpetkovski fpetkovski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@MichaHoffmann MichaHoffmann merged commit 1c5e7f1 into thanos-io:main Sep 3, 2024
22 checks passed
@cincinnat cincinnat deleted the fix-flaky-receive-multidb-test branch September 3, 2024 12:23
jnyi pushed a commit to jnyi/thanos that referenced this pull request Oct 17, 2024
There is race condition in `TestMultiTSDBPrune` due to a dangling goroutine
which can fail outside of the test function's lifetime if the database object
is closed before `Sync()` is finished.

Signed-off-by: Mikhail Nozdrachev <mikhail.nozdrachev@aiven.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants