tar has problems in a CIFS mount path with node 20 and node 18.18.0

[simatec]

Hello all,

I hope you can help here... We use the package among other things for backups in the iobroker project and have with current node 20 versions and from node 18.18.0 problems with many users who write their backup directly with a CIFS mount on the Fritzbox NAS.
Currently, I am only aware of problems in connection with Fitzbox and CIFS.

It seems that all attempts since Node 18.18.0 have problems with "fs.createWriteStream" or with .pipe.
Locally on the system there are no problems. The error only occurs when the backup is to be written to the CIFS mount point.

Here is an excerpt of how the create of the backup is constructed.

return new Promise((resolve, reject) => {
            const f = fs.createWriteStream(name);
            f.on('finish', () => {
                this.removeTempBackupDir();
                resolve(path.normalize(name));
            });

            f.on('error', e => {
                console.error(`host.${this.hostname} Cannot pack directory ${this.tmpDir}/backup: ${e.message}`);
                reject(new IoBrokerError({ message: e.message, code: EXIT_CODES.CANNOT_GZIP_DIRECTORY }));
            });

            try {
                tar.create({ gzip: true, cwd: `${this.tmpDir}/` }, ['backup']).pipe(f);
            } catch (e) {
                console.error(`host.${this.hostname} Cannot pack directory ${this.tmpDir}/backup: ${e.message}`);
                reject(new IoBrokerError({ message: e.message, code: EXIT_CODES.CANNOT_GZIP_DIRECTORY }));
            }
        });

Are there any known problems with Node 20 and are there any solutions or maybe an update from you?

[isaacs]

I'm not aware of any tar problems specific to node 20, that's interesting. And if it's also happening with node 18.18, then it's probably not node version specific. My guess is that you're probably correct that it has to do with the network file system in some way.

What is the error that's raised? If it's a problem with the write stream, gzip, or tar, it should have a path, a code, message, etc. That would help figure out what syscall is actually failing.

Does it work if you write the tarball to a local file and then stream it to the CIFS? Something like this:

const tmpFile = '/tmp/backup.tgz'
const tmpWrite = fs.createWriteStream(tmpFile)
tar.create({ gzip: true, cwd: `${this.tmpDir}/` }, ['backup']).pipe(tmpWrite)
tmpWrite.on('close', () => {
  fs.createReadStream(tmpFile).pipe(f)
})

Obviously not a real solution, since it's a bit wasteful, but could help illuminate what the issue is.

[simatec]

Unfortunately, no errors are output.
The system load goes to 100% and it hangs.
The local tar.gz can be copied to the CIFS mount without problems.

However, at the end of the day, this is not purposeful. The file must be written directly to the mount.
gzip is also not the problem at this point. We did tests without compression and the error was identical.

[simatec]

The code snippet was only an example...
In the backup adapter of iobroker I use tar-fs, which also uses node-tar under the main and also here the problem occurs.

[Grothesk242]

It must be a change of behaviour between nodejs 18.17.1 and 18.18.0.
The issue only shows in 18.18.0 (and in all nodejs@20 releases). So there is something backported, I think.

[simatec]

When I look at the changelog for 18.18.0, I only notice one point
Something was done here with streams

[simatec]

Here are all the commits for the changes to the streams under Node 18.18.0

nodejs/node#48550
nodejs/node#48435
nodejs/node#48691

[simatec]

The problem seems to be known in Node and is probably on the todo list.

[simatec]

It is a Problem from node ...
Issue is done.

[isaacs]

@simatec Thanks for circling back! I was scratching my head about how to even go about investigating this lol

[simatec]

😉