Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Lucene tests sometimes stall until they timeout #2813

Closed
ScottDugas opened this issue Jul 10, 2024 · 1 comment
Closed

Lucene tests sometimes stall until they timeout #2813

ScottDugas opened this issue Jul 10, 2024 · 1 comment
Assignees
@ScottDugas

Comments

@ScottDugas
Copy link
Contributor

Specifically some people have noticed that LuceneIndexTest.fullDeleteHelper times out after 5 minutes rather than completing, but it's normal runtime is less than 500ms.

Similarly the newly added LuceneIndexMaintenanceTest.concurrentStoreTest can experience a similar issue.
@ScottDugas ScottDugas self-assigned this Jul 10, 2024
ScottDugas added a commit to ScottDugas/fdb-record-layer that referenced this issue Jul 10, 2024
@ScottDugas
Add a time limit that's less than 1 hour to LuceneIndexMaintenanceTes…
10c012e 
…t.concurrentStoreTest

Due to FoundationDB#2813 this test can, occasionally, run forever, so add a
timeout so that it doesn't.
@ScottDugas ScottDugas mentioned this issue Jul 10, 2024
Merged
ohadzeliger pushed a commit that referenced this issue Jul 10, 2024
@ScottDugas
Add a time limit that's less than 1 hour to LuceneIndexMaintenanceTes…
c279f05 
…t.concurrentStoreTest (#2816)

Due to #2813 this test can, occasionally, run forever, so add a
timeout so that it doesn't.
@ScottDugas ScottDugas mentioned this issue Jul 11, 2024
Open
ScottDugas added a commit to ScottDugas/fdb-record-layer that referenced this issue Jul 12, 2024
@ScottDugas
Fixes FoundationDB#2813: Fail fast when Batch GRV Rate Limit Exceeded
f2ae5c4 
Loading the metadata version key does not seem to throw
Batch GRV rate limit exceeded, while other operations on the same
transaciton do. Instead it stalls forever, and the tests end up
timing out at the junit level.
This change introduces a new combine that will fail when the loading
the record store state fails, instead of waiting forever for the
metadata version to be loaded.
@ScottDugas
Copy link
Contributor Author

I believe this is caused by apple/foundationdb#11500 although I'm having trouble creating a valuable reproduction that doesn't involve all of record layer.

ScottDugas added a commit that referenced this issue Jul 16, 2024
@ScottDugas
Merge pull request #2822 from ScottDugas/fix-batch-read-timeouts
32d64ec 
Fixes #2813: Fail fast when Batch GRV Rate Limit Exceeded
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ScottDugas ScottDugas

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ScottDugas