[CI] RemoveCorruptedShardDataCommandTests.testCorruptedBothIndexAndTranslog fails after lucene upgrade

[henningandersen]

The 7.x test fails with:

No text input configured for prompt [Confirm [y/N] ]Close stacktrace
at __randomizedtesting.SeedInfo.seed([430F35E2B82EB29D:FEE5643694F8711E]:0)
at org.elasticsearch.cli.MockTerminal.readText(MockTerminal.java:60)
at org.elasticsearch.index.shard.RemoveCorruptedShardDataCommand.confirm(RemoveCorruptedShardDataCommand.java:235)
at org.elasticsearch.index.shard.RemoveCorruptedShardDataCommand.dropCorruptMarkerFiles(RemoveCorruptedShardDataCommand.java:208)

This error is caused by lucene not detecting the corruption after latest snapshot upgrade. The error reproduces locally (on 7.x branch) using:

./gradlew ':server:test' --tests "org.elasticsearch.index.shard.RemoveCorruptedShardDataCommandTests.testCorruptedBothIndexAndTranslog"   -Dtests.seed=430F35E2B82EB29D   -Dtests.security.manager=true   -Dtests.locale=mt   -Dtests.timezone=Africa/Lubumbashi   -Dcompiler.java=13

If I do:

git revert --no-commit 80e3c972100468325870319a952a46ad3ad3ed30

the problem disappears since Lucene detects the corruption.

Elasticsearch validates the last commit using Store.checkIntegrity which calls CodecUtil.checksumEntireFile(input). I am not sure if the problem is a bug in Lucene or Elasticsearch being overly ambitious here. Given that we do corrupt the file I tend to think Lucene is at fault, but the comment here could also indicate that this is simply a discrepancy in the checks?

[elasticmachine]

Pinging @elastic/es-distributed (:Distributed/Engine)

[jpountz]

This is a Lucene bug. The move of the stored fields index off-heap removed the integrity checks of the index from open-time but didn't add them back to StoredFieldsProducer#checkIntegrity. I opened https://issues.apache.org/jira/browse/LUCENE-9247.

[jpountz]

Closed via #53150.