Upload via WebUI (ipfs.files.write --create) produces different hash than CLI (ipfs.add)

[lidel]

(Originally reported in ipfs-inactive/support#45, today it was raised on irc)

Discrepancy between ipfs.add and ipfs.files.write --create was discussed and explained in ipfs-inactive/support#45, this issue is only about solving UX problem in WebUI.

How to Reproduce

1. Download QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR and save it as file.jpg
2. Add it back via CLI (ipfs add file.jpg
3. Open Files screen in old webui
4. Upload file.jpg
5. Right-click on it and select Copy hash

Current behavior

Copied value is QmWKsy9a3AU3Yn9SDuqUFNMnxFnsGq5KzkKgZREsr1cifv and not QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR.

In short ipfs.files.write --create (used by old webui) produces different hash than ipfs.add, for reasons notes in ipfs-inactive/support#45 (comment):

The hashes are different because ipfs files write uses different linking structure than ipfs add. ipfs files write's linking structure is optimized for random seeking and writing after initial creation and ipfs add structure is optimized for reduction of link count.

Underling data is still deduplicated as they use the same chunking, AFAIK.

ipfs.files.write --create

$ curl -s https://ipfs.io/ipfs/QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR | ipfs files write --create /files.write.jpg

$ ipfs files stat /files.write.jpg
QmWKsy9a3AU3Yn9SDuqUFNMnxFnsGq5KzkKgZREsr1cifv
Size: 119762
CumulativeSize: 119832
ChildBlocks: 1

Expected behavior

Most of users assume that hash will not change if the upload is done with "default settings of default tools". Value returned by ipfs.add sets expectation for entire ecosystem.

AFAIK an easy fix would be to switch WebUI's upload to use ipfs.add and then ipfs.files.cp:

ipfs.add --pin=false && ipfs.files.cp

(I think we don't want to pin, as presence in MFS will be enough to protect files from GC)

$ curl -s https://ipfs.io/ipfs/QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR | ipfs add
added QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR

$ ipfs files cp /ipfs/QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR /add.jpg

$ ipfs files stat /add.jpg
QmbWqxBEKC3P8tqsKc98xmWNzrzDtRLMiMPL8wBuTGsMnR
Size: 119762
CumulativeSize: 119776
ChildBlocks: 0
Type: file

[hacdias]

This is fixed on revamp branch. Can we close this? I also feel that revamp is coming closer to master in terms of features.

[lidel]

👌