Occasional connection failures, is it me or the server?

[sommmen]

Hiya, i have a hangfire job syncing files. Now and then it fails on ConnectAsync. It retries 3 times.

Renci.SshNet.Common.SshConnectionException: The connection was closed by the server: Too many connections (TooManyConnections).
   at Renci.SshNet.Session.WaitOnHandle(WaitHandle waitHandle, TimeSpan timeout)
   at Renci.SshNet.Session.ConnectAsync(CancellationToken cancellationToken)
   at Renci.SshNet.BaseClient.CreateAndConnectSessionAsync(CancellationToken cancellationToken)
   at Renci.SshNet.BaseClient.ConnectAsync(CancellationToken cancellationToken)
   ...

Renci.SshNet.Common.SshConnectionException: An established connection was aborted by the server.
   at Renci.SshNet.Session.WaitOnHandle(WaitHandle waitHandle, TimeSpan timeout)
   at Renci.SshNet.Session.ConnectAsync(CancellationToken cancellationToken)
   at Renci.SshNet.BaseClient.CreateAndConnectSessionAsync(CancellationToken cancellationToken)
   at Renci.SshNet.BaseClient.ConnectAsync(CancellationToken cancellationToken)
   ...

Renci.SshNet.Common.SshConnectionException: An established connection was aborted by the server.
   at Renci.SshNet.Session.WaitOnHandle(WaitHandle waitHandle, TimeSpan timeout)
   at Renci.SshNet.Session.ConnectAsync(CancellationToken cancellationToken)
   at Renci.SshNet.BaseClient.CreateAndConnectSessionAsync(CancellationToken cancellationToken)
   at Renci.SshNet.BaseClient.ConnectAsync(CancellationToken cancellationToken)
   ...

Another run looks the similar but with different issues:

Renci.SshNet.Common.SshConnectionException: The connection was closed by the server: Too many connections (TooManyConnections).
   at Renci.SshNet.Session.WaitOnHandle(WaitHandle waitHandle, TimeSpan timeout)
   at Renci.SshNet.Session.ConnectAsync(CancellationToken cancellationToken)
   at Renci.SshNet.BaseClient.CreateAndConnectSessionAsync(CancellationToken cancellationToken)
   at Renci.SshNet.BaseClient.ConnectAsync(CancellationToken cancellationToken)
   ...

Renci.SshNet.Common.SshOperationTimeoutException: Session operation has timed out
   at Renci.SshNet.Session.WaitOnHandle(WaitHandle waitHandle, TimeSpan timeout)
   at Renci.SshNet.PasswordAuthenticationMethod.Authenticate(Session session)
   at Renci.SshNet.ClientAuthentication.TryAuthenticate(ISession session, AuthenticationState authenticationState, String[] allowedAuthenticationMethods, SshAuthenticationException& authenticationException)
   at Renci.SshNet.ClientAuthentication.Authenticate(IConnectionInfoInternal connectionInfo, ISession session)
   at Renci.SshNet.ConnectionInfo.Authenticate(ISession session, IServiceFactory serviceFactory)
   at Renci.SshNet.Session.ConnectAsync(CancellationToken cancellationToken)
   at Renci.SshNet.BaseClient.CreateAndConnectSessionAsync(CancellationToken cancellationToken)
   at Renci.SshNet.BaseClient.ConnectAsync(CancellationToken cancellationToken)
   ...

Code:

using var client = new SftpClient(_settings.Host, _settings.Port, _settings.UserName, _settings.Password);
await client.ConnectAsync(cancellationToken);

 var sftpFiles = await client.ListDirectoryAsync("", cancellationToken)
     .Where(c => c.IsRegularFile)
     .ToListAsync(cancellationToken);
     
foreach (var sftpFile in sftpFiles) {
    // Downloads the file 
}

• Why would subsequent retries after TooManyConnections fail?
• Do i need to call Disconnect() or would the client disconnect itself properly in the dispose?
• Would TooManyConnections be a client or server issue? in other words is the server just busy or am i somehow creating and not properly removing too many connections?

[Rob-Hague]

"TooManyConnections" is the message received from the server here:

SSH.NET/src/Renci.SshNet/Session.cs

Line 1411 in 252c732

_exception = new SshConnectionException(string.Format(CultureInfo.InvariantCulture, "The connection was closed by the server: {0} ({1}).", message.Description, message.ReasonCode), message.ReasonCode);

I think it is up to the server whether subsequent retries would fail.

Calling Disconnect is not necessary

[sommmen]

Thx for your reply!

Oddly enough the situation seems to have improved by adding a try/catch and disconnect, but I feel like like this is just a transient issue.