Error on using exists function

[ankush-gaba]

I got this error while calling sftpClient.exists(path)

Error: lstat: lstat: No SFTP connection available at SftpClient.fmtError (/node_modules/ssh2-sftp-client/src/index.js:87:22) at SftpClient.lstat (/node_modules/ssh2-sftp-client/src/index.js:369:37) at SftpClient.exists (/node_modules/ssh2-sftp-client/src/index.js:390:31)

[theophilusx]

Step bacvk and imagine someone had just sent you this issue report. Now, what action do you feel you could take based on the details provided? Keep in mind, you have no idea what platform the code is runnning on, you don't know what version of node is being used, you havbe no example code to reproduce the issue and you have no details about context, about whether other commands work fine or anything else. You don't even know what version of the software is being used. Based on supplied details, all I can tell you is that you have and error because you tried to call sftp.exists() when you did not have a sftp connection and it failed. Not sure what you expected to happen.

[SeoYoung-C]

Hi. @theophilusx
I am getting the same error.

Operating System:

• Platform: darwin
• Arch: arm64
• Version: Darwin Kernel Version 23.6.0

Binaries:

• Node: 20.14.0
• npm: 10.8.1
• Yarn: N/A
• pnpm: N/A

Relevant Packages:

• next: 14.2.10
• react: 18.3.1
• react-dom: 18.3.1
• typescript: 5.3.3

ssh2-sftp-client : 11.0.0.

I am using the ssh2-sftp-client library within the Next.js framework's API. While I can't share the full code, my API function is very simple and only checks for the existence of files.

export async function GET(request: NextRequest) {
  try {
    const { searchParams } = new URL(request.url);
    const path = searchParams.get('path');
    const filesString = searchParams.get('files');
    const files = filesString?.split(',') || [];
    const connect = new Sftp();
    await connect.connect({
        host: XXX,
        username: XXXX,
        password: XXXX,
        port: XX
    })

    const data: string[] = [];
    for (let i = 0; i < files.length; i += 1) {
        const exist = await connect.exists(`${path}/${files[i]}`);
        if (exist) {
            data.push(files[i]);
        }
    } catch (e) {
     // ... 
    } 
}

When I run the code in this format, I get the following error:

Global error listener: read ECONNRESET
⨯ Error: end: read ECONNRESET
at Client.fn (webpack-internal:///(rsc)/./node_modules/ssh2-sftp-client/src/utils.js:69:22)

[theophilusx]

Thank you for at least providing some of the relevant information. Unfortunately, you failed to actually include the call to connect.end() end where the error arises? Can you also confirm that the exit() calls actually return the requested informationh and that it is only after you habve interacted with the remote server that you get this error? A debug trace would also be useful (add a console.log debug property to thye connect object). The first thing you need to do is write a very simple script which queries the same remote sftp server with the same list of files for exist status, but which only uses ssh2-sftp-client (no next.js, no typescript) and run it on the same server and see if you get the same error. This will confirm the issue is not something being introduced by either mext.js or typescript. If you do get the same error, provide the full script you used, a complete debug and I'll see if I can reproduce the issue. As the error indicates the remote server has reset the connection unexpectedly, any details regarding the type of sftp server and the platform it is running on would also be useful in attempting to reproduce the issue. If you have access, you should also check both the sftp server and remote host logs to see if that sheds any light on what is causing your issue.

[SeoYoung-C]

@theophilusx
Oh, Sorry. my next.js code is

export async function GET(request: NextRequest) {
  try {
    const { searchParams } = new URL(request.url);
    const path = searchParams.get('path');
    const filesString = searchParams.get('files');
    const files = filesString?.split(',') || [];
    const connect = new Sftp();
    await connect.connect({
        host: XXX,
        username: XXXX,
        password: XXXX,
        port: XX
    })

    const data: string[] = [];
    for (let i = 0; i < files.length; i += 1) {
        const exist = await connect.exists(`${path}/${files[i]}`);
        if (exist) {
            data.push(files[i]);
        }
        
        return NextResponse.json(data, { status: 200 }); 
    } catch (e) {
     // ... 
    } finally {
        try {
           if (connect?.end) await connect.end();
        } catch (e) {
           console.error('Finally Connect End', e);
        }
    } 
}

And I found that the same error occurs when testing with Node.js in JavaScript as you mentioned. We will provide the test code via a GitHub link instead. Please understand that we cannot share the SSH credentials.

my github project URL is https://github.com/SeoYoung-C/sftp-test
( Please check your email for an invitation to access this private GitHub project.)

[theophilusx]

Can you clarify which error you are getting. The first post in this issue referred to getting the error there was no sftp connection, but then the later post refers to an econnreset error. These are completely different error messages with completely different causes (the first indicates there is no current connection i.e. if there was a connection, it has been closed, The second is an error indicating the the remote sftp server has unexpectedly closed the connection, typically without being requested to do so). The first error is usually seen when someone has either failed to first open a connection (no call to connect) or tries to use the sftp object after a call to end() while the second error is usually an indication the remote sftp server has (for some unknown reason) closed the connection.

Can you also provide some details on the remote sftp server. If you are seeing econnreset errors, it indicates that the remote server is closing the connection. If this is the case, then there is little ssh2-sftp-client can do and it will need to be investigated at the remote sftp server end. Issues of this type have been seen due to either network issues, such as problems with deep packet inspecting firewalls abruptly dropping connections due to firewall policy (normally, the remote sftp server will provide an error event with reason for closing a connection. However, firewalls often just drop the connection without details for security reasons).

The other reported source of such errors has been some cloud based sftp server (noteably azure# which seem to issue an econnreset error in response to an end() request (which is incorrect protocol implementation as it is not an error to end the connection after a request, obviously).

I will attempt to run your test code against my test sftp servers in the next day or so. I don't expect it will reproduce the error as your code is pretty much the same as our test code and all the tests pass fine. However, I will run it to be sure. Recommend you run your test script against a different server (different host and different network) and see if you get same error. If you don't, then that would be more evidence the issue is with the sftp server and you will need to investigate at that end.

[SeoYoung-C]

@theophilusx

It seems that my issue and the one described in the initial post are somewhat different.

The original issue posted by ankush-gaba was about "Error: lstat: No SFTP connection available at SftpClient.fmtError," whereas my issue is "Error: end: read ECONNRESET at Client.fn." I apologize for the confusion in my previous response.

I have also created a log file for the Next.js environment in the GitHub repository: https://github.com/SeoYoung-C/sftp-test. The file contains the same logs, and the test results are identical except for the Next.js and TypeScript environment.

The "Error: end: read ECONNRESET at Client.fn" issue occurs consistently in both Next.js and Node.js environments, and does not happen in version 10.0.3. The SFTP server we are using is Akamai-based and is in a restricted access environment, so it cannot be shared.

If possible, I would appreciate it if you could run tests on your end. I will also test on another server and let you know the results.

Thank you very much for your kind assistance.

[theophilusx]

I just looked at the console.log file you included in your repo. From that it looks like your sftp server is sending an econnreset ERROR in response to an end request. It should not do that. This has nothing to do with the exists() method and I don't believe it is a bug in ssh2-sftp-client. This is an issue with the sftp server you are connecting to.

A google search indicates this is a known issue with the version of openssh server on windows. For example, see mscdex/ssh2#902

What you can do is try adding a catch for the econreset error on a try block wrapped around your call to end() and just ignore econreset error after any call to end(). It might also be worth checking to see if there is an updated openssh sftp server for windows that perhaps fixes this issue.

[theophilusx]

The "Error: end: read ECONNRESET at Client.fn" issue occurs consistently in both Next.js and Node.js environments, and does not happen in version 10.0.3. The SFTP server we are using is Akamai-based and is in a restricted access environment, so it cannot be shared.

FYI sadly, I think your a victim of an improvement and more 'correct' behaviour in ssh2-sftp-client v11. Changes were made in v11 to fix problems in v10 where legitimate errors were not being picked up. This caused two issues. Firstly, it could result in ssh2-sftp-client hanging indefinitely because an error occured which prevents the requested action from completing and because the error was not picked up, the promise is never resolved or rejected. The second issue was the client was not informed when an error occurred, which could result in loss of data or confusion as to why a subsequent request is rejected. There wsas also a 3rd unrelated issue where managing (catching) errors in some situations was impossible, resulting in scripts exiting abnormally rather than cleanly. Version 11 handles things more correctly and consistently. Unfortunately, this also reveals servers like the version of the openssh server you are connecting to, which don't behave correctly.

[SeoYoung-C]

The "Error: end: read ECONNRESET at Client.fn" issue occurs consistently in both Next.js and Node.js environments, and does not happen in version 10.0.3. The SFTP server we are using is Akamai-based and is in a restricted access environment, so it cannot be shared.
FYI sadly, I think your a victim of an improvement and more 'correct' behaviour in ssh2-sftp-client v11. Changes were made in v11 to fix problems in v10 where legitimate errors were not being picked up. This caused two issues. Firstly, it could result in ssh2-sftp-client hanging indefinitely because an error occured which prevents the requested action from completing and because the error was not picked up, the promise is never resolved or rejected. The second issue was the client was not informed when an error occurred, which could result in loss of data or confusion as to why a subsequent request is rejected. There wsas also a 3rd unrelated issue where managing (catching) errors in some situations was impossible, resulting in scripts exiting abnormally rather than cleanly. Version 11 handles things more correctly and consistently. Unfortunately, this also reveals servers like the version of the openssh server you are connecting to, which don't behave correctly.

Hello. @theophilusx .

I apologize for the late response.
Thank you for explaining the cause of the error in detail. I will check further to see if there is another way on the SFTP server side.