CI Failure (key symptom) in EndToEndShadowIndexingTest.test_reset_from_cloud

[vbotbuildovich]

https://buildkite.com/redpanda/redpanda/builds/50079

Module: rptest.tests.e2e_shadow_indexing_test
Class: EndToEndShadowIndexingTest
Method: test_reset_from_cloud
Arguments: {
    "cloud_storage_type": 1
}

test_id:    EndToEndShadowIndexingTest.test_reset_from_cloud
status:     FAIL
run time:   28.720 seconds

<NodeCrash docker-rp-14: ERROR 2024-06-10 20:52:37,470 [shard 0:clus] assert - Assert failure: (/var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-05782de77e74ee5f5-1/redpanda/redpanda/src/v/cluster/members_manager.cc:839) 'it != _id_by_uuid.end()' Node registration must be completed before calling
>
Traceback (most recent call last):
  File "/root/tests/rptest/services/redpanda.py", line 2708, in start
    self.for_nodes(to_start, start_one)
  File "/root/tests/rptest/services/redpanda.py", line 1478, in for_nodes
    return list(executor.map(cb, nodes))
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator
    yield _result_or_cancel(fs.pop())
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel
    return fut.result(timeout)
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 458, in result
    return self.__get_result()
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/root/tests/rptest/services/redpanda.py", line 2700, in start_one
    self.start_node(node,
  File "/root/tests/rptest/services/redpanda.py", line 3052, in start_node
    self.start_service(node, start_rp)
  File "/root/tests/rptest/services/redpanda.py", line 3128, in start_service
    start()
  File "/root/tests/rptest/services/redpanda.py", line 3043, in start_rp
    wait_until(
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/utils/util.py", line 57, in wait_until
    raise TimeoutError(err_msg() if callable(err_msg) else err_msg) from last_exception
ducktape.errors.TimeoutError: Redpanda service docker-rp-15 failed to start within 20 sec

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 182, in _do_run
    self.setup_test()
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 267, in setup_test
    self.test.setup()
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/test.py", line 91, in setup
    self.setUp()
  File "/root/tests/rptest/tests/e2e_shadow_indexing_test.py", line 107, in setUp
    self.redpanda.start()
  File "/root/tests/rptest/services/redpanda.py", line 2715, in start
    self.raise_on_crash()
  File "/root/tests/rptest/services/redpanda.py", line 3426, in raise_on_crash
    raise NodeCrash(crashes)
rptest.services.utils.NodeCrash: <NodeCrash docker-rp-14: ERROR 2024-06-10 20:52:37,470 [shard 0:clus] assert - Assert failure: (/var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-05782de77e74ee5f5-1/redpanda/redpanda/src/v/cluster/members_manager.cc:839) 'it != _id_by_uuid.end()' Node registration must be completed before calling
>
Test requested 4 nodes, used only 3```


JIRA Link: [CORE-3705](https://redpandadata.atlassian.net/browse/CORE-3705)

[vbotbuildovich]

*https://buildkite.com/redpanda/redpanda/builds/50079

[mmaslankaprv]

fixed with: #19790