Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI Failure (key symptom) in ScramfulEosTest.test_tx_init_passes_1 #22584

Closed
vbotbuildovich opened this issue Jul 26, 2024 · 3 comments
Closed
Labels
auto-triaged used to know which issues have been opened from a CI job ci-failure ci-rca/infra CI Root Cause Analysis - Infrastructure Issue

Comments

@vbotbuildovich
Copy link
Collaborator

vbotbuildovich commented Jul 26, 2024

https://buildkite.com/redpanda/vtools/builds/15928
https://buildkite.com/redpanda/vtools/builds/15944
https://buildkite.com/redpanda/vtools/builds/15952
https://buildkite.com/redpanda/vtools/builds/15980

Module: rptest.tests.scramful_eos_test
Class: ScramfulEosTest
Method: test_tx_init_passes_1
test_id:    ScramfulEosTest.test_tx_init_passes_1
status:     FAIL
run time:   5.644 seconds

RemoteCommandError({'ssh_config': {'host': 'ip-172-31-5-33', 'hostname': '172.31.5.33', 'user': 'root', 'port': 22, 'password': None, 'identityfile': '/home/ubuntu/.ssh/id_rsa'}, 'hostname': 'ip-172-31-5-33', 'ssh_hostname': '172.31.5.33', 'user': 'root', 'externally_routable_ip': '54.201.146.94', '_logger': <Logger rptest.tests.scramful_eos_test.ScramfulEosTest.test_tx_init_passes_1-1428 (DEBUG)>, 'os': 'linux', '_ssh_client': <paramiko.client.SSHClient object at 0x7faf09788910>, '_sftp_client': <paramiko.sftp_client.SFTPClient object at 0x7faf09810160>, '_custom_ssh_exception_checks': None}, 'host ip-172-31-5-33.us-west-2.compute.internal', 127, b'')
Traceback (most recent call last):
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 182, in _do_run
    self.setup_test()
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/runner_client.py", line 267, in setup_test
    self.test.setup()
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/tests/test.py", line 91, in setup
    self.setUp()
  File "/home/ubuntu/redpanda/tests/rptest/tests/redpanda_test.py", line 39, in setUp
    self.__redpanda.start()
  File "/home/ubuntu/redpanda/tests/rptest/services/redpanda.py", line 2719, in start
    self.for_nodes(to_start, start_one)
  File "/home/ubuntu/redpanda/tests/rptest/services/redpanda.py", line 1489, in for_nodes
    return list(executor.map(cb, nodes))
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 621, in result_iterator
    yield _result_or_cancel(fs.pop())
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 319, in _result_or_cancel
    return fut.result(timeout)
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 458, in result
    return self.__get_result()
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/ubuntu/redpanda/tests/rptest/services/redpanda.py", line 2711, in start_one
    self.start_node(node,
  File "/home/ubuntu/redpanda/tests/rptest/services/redpanda.py", line 3021, in start_node
    self.write_node_conf_file(
  File "/home/ubuntu/redpanda/tests/rptest/services/redpanda.py", line 3947, in write_node_conf_file
    fqdn = self.get_node_fqdn(node)
  File "/home/ubuntu/redpanda/tests/rptest/services/redpanda.py", line 3883, in get_node_fqdn
    fqdn = node.account.ssh_output(
  File "/opt/.ducktape-venv/lib/python3.10/site-packages/ducktape/cluster/remoteaccount.py", line 41, in wrapper
    return method(self, *args, **kwargs)
ducktape.cluster.remoteaccount.RemoteCommandError: root@ip-172-31-5-33: Command 'host ip-172-31-5-33.us-west-2.compute.internal' returned non-zero exit status 127.

JIRA Link: CORE-6706

@vbotbuildovich vbotbuildovich added auto-triaged used to know which issues have been opened from a CI job ci-failure labels Jul 26, 2024
@piyushredpanda piyushredpanda added the ci-rca/infra CI Root Cause Analysis - Infrastructure Issue label Jul 27, 2024
@vbotbuildovich
Copy link
Collaborator Author

@vbotbuildovich
Copy link
Collaborator Author

@rpdevmp
Copy link
Contributor

rpdevmp commented Jul 31, 2024

Duplicate of #21624
That GH issue has all the details of many tests failing, and what are the steps to avoid similar cases in the future, where many CI failures got opened based on similar infra errors.
Updating Github automatically and AWS with FIPS run should be fixed by https://github.com/redpanda-data/vtools/pull/3018

@rpdevmp rpdevmp closed this as completed Jul 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-triaged used to know which issues have been opened from a CI job ci-failure ci-rca/infra CI Root Cause Analysis - Infrastructure Issue
Projects
None yet
Development

No branches or pull requests

3 participants