[BUG]Exception calling application: Error reading SSH protocol banner

[www-uu]

Describe the bug
This exception occur irregularly and frequently.

To Reproduce
Steps to reproduce the behavior:

1. Distributed deployed: 1 master server, 1 distributed server.
2. exception log:

Traceback (most recent call last):
  File "/demo/demo.py", line 184, in startSession
    result, exceptions = self.coreemu.start_session(self.session)
  File "/root/Documents/core/daemon/core/api/grpc/client.py", line 249, in start_session
    response = self.stub.StartSession(request)
  File "/opt/core/venv/lib/python3.10/site-packages/grpc/_channel.py", line 1030, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/opt/core/venv/lib/python3.10/site-packages/grpc/_channel.py", line 910, in _end_unary_response_blocking
    raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.UNKNOWN
        details = "Exception calling application: Error reading SSH protocol banner"
        debug_error_string = "UNKNOWN:Error received from peer  {grpc_message:"Exception calling application: Error reading SSH protocol banner", grpc_status:2, created_time:"2024-08-21T07:39:03.009693668+00:00"}"
>

Expected behavior
This exception not occur.

Screenshots

Desktop (please complete the following information):

• OS: [e.g. Ubuntu 22.04]
• CORE Version [e.g. 9.0.2]

Additional context

[bharnden]

The current functionality for distributed uses fabric for SSH connections per node, there is a default SSH setting for 10 connections. I would assume you are potentially hitting this limit at times causing issue.

You can either limit the number of nodes/connections to that distributed server or update the SSH configuration to handle more connections.