CI Failure (TimeoutError('Redpanda node docker-rp-13 failed to stop in 30 seconds')) in ConsumerGroupTest.test_large_group_count

[andijcr]

https://buildkite.com/redpanda/redpanda/builds/30951#01889fbf-4d78-4127-8af1-cf2fc3ab02a5

Module: rptest.tests.consumer_group_test
Class:  ConsumerGroupTest
Method: test_large_group_count

====================================================================================================
test_id:    rptest.tests.consumer_group_test.ConsumerGroupTest.test_large_group_count
status:     FAIL
run time:   3 minutes 39.116 seconds


    TimeoutError('Redpanda node docker-rp-13 failed to stop in 30 seconds')
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 135, in run
    data = self.run_test()
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py", line 227, in run_test
    return self.test_context.function(self.test)
  File "/root/tests/rptest/utils/mode_checks.py", line 63, in f
    return func(*args, **kwargs)
  File "/root/tests/rptest/services/cluster.py", line 79, in wrapped
    r = f(self, *args, **kwargs)
  File "/root/tests/rptest/tests/consumer_group_test.py", line 488, in test_large_group_count
    self.redpanda.restart_nodes(self.redpanda.nodes)
  File "/root/tests/rptest/services/redpanda.py", line 831, in restart_nodes
    list(
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator
    yield _result_or_cancel(fs.pop())
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel
    return fut.result(timeout)
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 458, in result
    return self.__get_result()
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/root/tests/rptest/services/redpanda.py", line 832, in <lambda>
    executor.map(lambda n: self.stop_node(n, timeout=stop_timeout),
  File "/root/tests/rptest/services/redpanda.py", line 2801, in stop_node
    wait_until(
  File "/usr/local/lib/python3.10/dist-packages/ducktape/utils/util.py", line 57, in wait_until
    raise TimeoutError(err_msg() if callable(err_msg) else err_msg) from last_exception
ducktape.errors.TimeoutError: Redpanda node docker-rp-13 failed to stop in 30 seconds

[rystsov]

On Wed Jun 21 2023 we had two failures out of 5122 executions:

• https://buildkite.com/redpanda/redpanda/builds/30951 (Fri Jun 09 2023)
• https://buildkite.com/redpanda/redpanda/builds/31115 (Mon Jun 12 2023)

[rystsov]

Some of the "failed to stop in 30s" errors are correlated with f7fc026. They started after it was merged (Thu Jun 08 2023) and stopped when it was reverted (Thu Jun 15 2023).

Closing the issue since {9,12} fits the 8-15 range.