[2.x] MultiCodecReindexIT.testReindexingMultipleCodecs test failure

[andrross]

The following deterministically fails in my tests, but only on the 2.x branch:

./gradlew ':modules:reindex:internalClusterTest' --tests "org.opensearch.index.codec.MultiCodecReindexIT.testReindexingMultipleCodecs" -Dtests.seed=1FF95E112F7A424C -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=el-CY -Dtests.timezone=Africa/Kinshasa -Druntime.java=17

Found in #9844

https://build.ci.opensearch.org/job/gradle-check/24762/consoleFull

[andrross]

@sarthakaggarwal97 Do you mind taking a look? Maybe I missed something in the backport? There were a couple conflicts I had to resolve.

[sarthakaggarwal97]

@andrross I don't think this is specific to ZStd codecs, I've seen the exception failed to obtain in-memory shard lock in clusters without Zstd as well. I will check more on this

[andrross]

Resolved by #9889