write-file-atomic swallows errors

[vmx]

• Version: current HEAD
• Platform: Linux

Type: Enhancement

Severity: Low

Description:

When using an IPFS Repository with js-ipfs that was created by js-ipld it will fail as the Identity is missing in the config.

The problem is that it took my quite some time to find that out as the original error is swallowed by write-file-atomic. The current error message on the first run of js-ipfs is:

Initializing daemon...
/home/vmx/src/pl/js-ipfs/node_modules/async/internal/onlyOnce.js:9
        if (fn === null) throw new Error("Callback was already called.");
                         ^

Error: Callback was already called.
    at /home/vmx/src/pl/js-ipfs/node_modules/async/internal/onlyOnce.js:9:32
    at /home/vmx/src/pl/js-ipfs/node_modules/async/internal/parallel.js:36:13
    at /home/vmx/src/pl/js-ipfs/node_modules/write-file-atomic/index.js:143:7
    at FSReqWrap.oncomplete (fs.js:171:20)

A second run of jsipfs daemon reveals the real error:

Initializing daemon...
/home/vmx/src/pl/js-ipfs/src/core/components/pre-start.js:65
        const privKey = config.Identity.PrivKey
                                        ^

TypeError: Cannot read property 'PrivKey' of undefined
    at waterfall (/home/vmx/src/pl/js-ipfs/src/core/components/pre-start.js:65:41)
    at nextTask (/home/vmx/src/pl/js-ipfs/node_modules/async/waterfall.js:16:14)
    at next (/home/vmx/src/pl/js-ipfs/node_modules/async/waterfall.js:23:9)
    at /home/vmx/src/pl/js-ipfs/node_modules/async/internal/onlyOnce.js:12:16
    at waterfall (/home/vmx/src/pl/js-ipfs/src/core/components/pre-start.js:58:9)
    at nextTask (/home/vmx/src/pl/js-ipfs/node_modules/async/waterfall.js:16:14)
    at next (/home/vmx/src/pl/js-ipfs/node_modules/async/waterfall.js:23:9)
    at /home/vmx/src/pl/js-ipfs/node_modules/async/internal/onlyOnce.js:12:16
    at waterfall (/home/vmx/src/pl/js-ipfs/src/core/components/pre-start.js:38:18)
    at nextTask (/home/vmx/src/pl/js-ipfs/node_modules/async/waterfall.js:16:14)

The swallowing happens within write-file-atomic. Our callback function passed into writeFileAtomic() is throwing an error. When our callback is called, it will trigger the catch promise which will then trigger the callback again.

I bring this up as it is always bad if some library is hiding errors. The problem here is that write-file-atomic is using promises internally, but has a callback based public API. I tried to fix it, but couldn't find a clean way. I think the best way would be to fix npm/write-file-atomic#33 fixed, which proposes returning a promise, instead of passing in a callback.

Steps to reproduce the error:

$ IPFS_PATH=/tmp/doublecalllback jsipfs init --empty-repo
initializing ipfs node at /tmp/doublecalllback
generating 2048-bit RSA keypair...done
peer identity: QmVNqmWGV248kCZpJq2yXSYzTQi5the7vxs82HLfZj3tcK

$ echo '{}' > /tmp/doublecalllback/config

$ IPFS_PATH=/tmp/doublecalllback jsipfs daemon
Initializing daemon...
/home/vmx/src/pl/js-ipfs/node_modules/async/internal/onlyOnce.js:9
        if (fn === null) throw new Error("Callback was already called.");
                         ^

Error: Callback was already called.
    at /home/vmx/src/pl/js-ipfs/node_modules/async/internal/onlyOnce.js:9:32
    at /home/vmx/src/pl/js-ipfs/node_modules/async/internal/parallel.js:36:13
    at /home/vmx/src/pl/js-ipfs/node_modules/write-file-atomic/index.js:143:7
    at FSReqWrap.oncomplete (fs.js:171:20)

$ IPFS_PATH=/tmp/doublecalllback jsipfs daemon
Initializing daemon...
/home/vmx/src/pl/js-ipfs/src/core/components/pre-start.js:65
        const privKey = config.Identity.PrivKey
                                        ^

TypeError: Cannot read property 'PrivKey' of undefined
    at waterfall (/home/vmx/src/pl/js-ipfs/src/core/components/pre-start.js:65:41)
    at nextTask (/home/vmx/src/pl/js-ipfs/node_modules/async/waterfall.js:16:14)
    at next (/home/vmx/src/pl/js-ipfs/node_modules/async/waterfall.js:23:9)
    at /home/vmx/src/pl/js-ipfs/node_modules/async/internal/onlyOnce.js:12:16
    at waterfall (/home/vmx/src/pl/js-ipfs/src/core/components/pre-start.js:58:9)
    at nextTask (/home/vmx/src/pl/js-ipfs/node_modules/async/waterfall.js:16:14)
    at next (/home/vmx/src/pl/js-ipfs/node_modules/async/waterfall.js:23:9)
    at /home/vmx/src/pl/js-ipfs/node_modules/async/internal/onlyOnce.js:12:16
    at waterfall (/home/vmx/src/pl/js-ipfs/src/core/components/pre-start.js:38:18)
    at nextTask (/home/vmx/src/pl/js-ipfs/node_modules/async/waterfall.js:16:14)

[vmx]

I forget to add that if you want to see that issue directly without js-ipfs, you can use this script:

'use strict'

const writeFileAtomic = require('write-file-atomic')

writeFileAtomic('/tmp/somefile.txt', 'Hello Node', function (err) {
  console.log('vmx: outer callback was called')
  throw new Error('vmx: this is an error from within the final callback')
  console.log('It\'s saved!')
})

The output is:

$ node doublecallback.js
vmx: outer callback was called
vmx: outer callback was called
/home/vmx/src/pl/upstream/write-file-atomic/asyncexample/doublecallback.js:7
  throw new Error('vmx: this is an error from within the final callback')
  ^

Error: vmx: this is an error from within the final callback
    at /home/vmx/src/pl/upstream/write-file-atomic/asyncexample/doublecallback.js:7:9
    at /home/vmx/src/pl/upstream/write-file-atomic/write-file-atomic/index.js:142:7
    at FSReqWrap.oncomplete (fs.js:171:20)

So the callback was called twice.

[alanshaw]

@vmx can you provide details of how you're doing this:

When using an IPFS Repository with js-ipfs that was created by by js-ipfs it will fail as the Identity is missing in the config.

[vmx]

@alanshaw I'm so sorry, I had a typo at the most important part of the description. It should read "that was created by js-ipld". I've updated the original issue.

[alanshaw]

Since this issue was created we've switched out write-file-atomic and found and resolved #1860 please open an new issue if you notice this again.