Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix TestIndexWriterOnError.testIOError failure. #13436

Merged
merged 3 commits into from
May 29, 2024

Conversation

jpountz
Copy link
Contributor

@jpountz jpountz commented May 29, 2024

Pull request #13406 inadvertly broke Lucene's handling of tragic exceptions by stopping after the first DocValuesProducer whose close() calls throws an exception, instead of keeping calling close() on further producers in the list.

This moves back to the previous behavior.

Closes #13434

Pull request apache#13406 inadvertly broke Lucene's handling of tragic exceptions by
stopping after the first `DocValuesProducer` whose `close()` calls throws an
exception, instead of keeping calling `close()` on further producers in the
list.

This moves back to the previous behavior.

Closes apache#13434
@jpountz jpountz requested a review from bruno-roustant May 29, 2024 16:24
Comment on lines +79 to +85
IOUtils.applyToAll(
dvProducersGens.stream().mapToObj(Long::valueOf).toList(),
gen -> {
RefCount<DocValuesProducer> dvp = genDVProducers.get(gen);
assert dvp != null : "gen=" + gen;
dvp.decRef();
});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My only minor concern is that this reverts the minor improvement of using primitive values. But, it fixes a much bigger issue. I am fine with reverting only this part of the optimization that caused the bug.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, good catch. This fix is required to continue using IOUtils.
I don't know why I didn't read enough the method javadoc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed @benwtrent. I optimized for making the code look as similar as before without reverting the move to LongArrayList, in order to minimize room for bugs given the upcoming release.

Copy link
Contributor

@bruno-roustant bruno-roustant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the fix

Comment on lines +79 to +85
IOUtils.applyToAll(
dvProducersGens.stream().mapToObj(Long::valueOf).toList(),
gen -> {
RefCount<DocValuesProducer> dvp = genDVProducers.get(gen);
assert dvp != null : "gen=" + gen;
dvp.decRef();
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, good catch. This fix is required to continue using IOUtils.
I don't know why I didn't read enough the method javadoc.

@jpountz jpountz merged commit 750a7c4 into apache:main May 29, 2024
3 checks passed
@jpountz jpountz deleted the fix_test_IOError branch May 29, 2024 20:08
jpountz added a commit that referenced this pull request May 29, 2024
Pull request #13406 inadvertly broke Lucene's handling of tragic exceptions by
stopping after the first `DocValuesProducer` whose `close()` calls throws an
exception, instead of keeping calling `close()` on further producers in the
list.

This moves back to the previous behavior.

Closes #13434
jpountz added a commit that referenced this pull request May 29, 2024
Pull request #13406 inadvertly broke Lucene's handling of tragic exceptions by
stopping after the first `DocValuesProducer` whose `close()` calls throws an
exception, instead of keeping calling `close()` on further producers in the
list.

This moves back to the previous behavior.

Closes #13434
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

TestIndexWriterOnVMError.testUnknownError fails with unclosed file handles
3 participants