Skip to content

Commit

Permalink
cst/ducktape: Change assertion for deleted chunks
Browse files Browse the repository at this point in the history
Because chunk readers keep open the handle to a chunk file even if the
file itself is deleted, during the test_read_chunks it is possible that
the sequential consumer is entirely served by these open file handles
while all chunk files in the directory have been removed. This results
in the assertion that some chunks must have been removed during
sequential consumer to fail.

As the count of chunks deleted is not important as an assertion and the
purpose of deleting chunks randomly was to exercise failed
materialization code path which should happen during the random consumer
run, the assertion is changed to only test once for both seq and random
consumers.

Another alternative would be a more fine grained check, eg assert if
some chunks are removed only if some files are written to the cache
during a consumer run, but such checks do not buy extra value.

(cherry picked from commit 866675e)
  • Loading branch information
abhijat authored and vbotbuildovich committed Jul 16, 2024
1 parent a69d54b commit 125a63d
Showing 1 changed file with 1 addition and 7 deletions.
8 changes: 1 addition & 7 deletions tests/rptest/tests/cloud_storage_chunk_read_path_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -229,12 +229,6 @@ def test_read_chunks(self):
self._assert_not_in_cache(
fr'.*kafka/{self.topic}/.*\.log\.[0-9]+$')

# We cannot assert that there are chunks in cache because the deleting thread
# could delete them all before we run the assertion, causing it to fail.
# We can check that the thread deleted some chunks.
assert rm_chunks.deleted_chunks > 0, "Expected to delete some chunk files during rand-cons, none deleted"
rm_chunks.deleted_chunks = 0

consumer = KgoVerifierSeqConsumer(self.test_context,
self.redpanda,
self.topic,
Expand All @@ -244,7 +238,7 @@ def test_read_chunks(self):
consumer.wait(timeout_sec=120)
rm_chunks.stop()
rm_chunks.join(timeout=10)
assert rm_chunks.deleted_chunks > 0, "Expected to delete some chunk files during seq-cons, none deleted"
assert rm_chunks.deleted_chunks > 0, "Expected to delete some chunk files, none deleted"

self._assert_not_in_cache(
fr'.*kafka/{self.topic}/.*\.log\.[0-9]+$')
Expand Down

0 comments on commit 125a63d

Please sign in to comment.