Unable to reproduce release build, minified files differ

[lidel]

Problem

Build pipeline produces different output than one represented by a CID attached to release notes, which causes ipfs-companion to be hidden from addons.mozilla.org (ipfs/ipfs-companion#679) due to reproducibility issues:

2. I've built webui branch https://github.com/ipfs-shipyard/ipfs-webui/releases/tag/v2.3.3 using node 8.12.0 and npm 6.4.1, but the resulting code is different from the webui folder in the xpi. Please see the attached build https://drive.google.com/open?id=1Ba9kvMhSUZM0KMQMhP7ebYmGX8CCAUAh
   Please test your build steps on a virtual machine with a clean setup and update the instructions for us.

I compared

• build_webui provided by reviewer
• and one from QmXc9raDM1M5G5fpBnVyQ71vR4gbnskwnB9iMEzBuLgvoZ (ipfs-webui/releases/tag/v2.3.3)

Indeed, minified content differs:

[ Click to expand diff ]

$ diff --brief -r build_webui webui_QmXc9raDM1M5G5fpBnVyQ71vR4gbnskwnB9iMEzBuLgvoZ                                                                                                                                       
Files build_webui/asset-manifest.json and webui_QmXc9raDM1M5G5fpBnVyQ71vR4gbnskwnB9iMEzBuLgvoZ/asset-manifest.json differ
Files build_webui/index.html and webui_QmXc9raDM1M5G5fpBnVyQ71vR4gbnskwnB9iMEzBuLgvoZ/index.html differ
Only in build_webui: manifest.json
Only in webui_QmXc9raDM1M5G5fpBnVyQ71vR4gbnskwnB9iMEzBuLgvoZ: _previous-versions
Files build_webui/service-worker.js and webui_QmXc9raDM1M5G5fpBnVyQ71vR4gbnskwnB9iMEzBuLgvoZ/service-worker.js differ
Only in build_webui/static/css: main.ace3c083.css.map
Only in build_webui/static/js: 0.45d2ea26.chunk.js
Only in build_webui/static/js: 0.45d2ea26.chunk.js.map
Only in webui_QmXc9raDM1M5G5fpBnVyQ71vR4gbnskwnB9iMEzBuLgvoZ/static/js: 0.f150ccdd.chunk.js
Only in webui_QmXc9raDM1M5G5fpBnVyQ71vR4gbnskwnB9iMEzBuLgvoZ/static/js: 1.61c650c3.chunk.js
Only in build_webui/static/js: 1.6bbfba19.chunk.js
Only in build_webui/static/js: 1.6bbfba19.chunk.js.map
Only in build_webui/static/js: 2.bc355859.chunk.js
Only in build_webui/static/js: 2.bc355859.chunk.js.map
Only in webui_QmXc9raDM1M5G5fpBnVyQ71vR4gbnskwnB9iMEzBuLgvoZ/static/js: 2.d164b02b.chunk.js
Only in build_webui/static/js: 3.52556504.chunk.js
Only in build_webui/static/js: 3.52556504.chunk.js.map
Only in webui_QmXc9raDM1M5G5fpBnVyQ71vR4gbnskwnB9iMEzBuLgvoZ/static/js: 3.c371693f.chunk.js
Only in webui_QmXc9raDM1M5G5fpBnVyQ71vR4gbnskwnB9iMEzBuLgvoZ/static/js: 4.2f4edf06.chunk.js
Only in build_webui/static/js: 4.cdb20f5c.chunk.js
Only in build_webui/static/js: 4.cdb20f5c.chunk.js.map
Only in webui_QmXc9raDM1M5G5fpBnVyQ71vR4gbnskwnB9iMEzBuLgvoZ/static/js: 5.5fd1145b.chunk.js
Only in build_webui/static/js: 5.e171992c.chunk.js
Only in build_webui/static/js: 5.e171992c.chunk.js.map
Only in build_webui/static/js: 6.6b80439c.chunk.js
Only in build_webui/static/js: 6.6b80439c.chunk.js.map
Only in webui_QmXc9raDM1M5G5fpBnVyQ71vR4gbnskwnB9iMEzBuLgvoZ/static/js: 6.e9a35513.chunk.js
Only in webui_QmXc9raDM1M5G5fpBnVyQ71vR4gbnskwnB9iMEzBuLgvoZ/static/js: 7.9c60492f.chunk.js
Only in build_webui/static/js: 7.ee081174.chunk.js
Only in build_webui/static/js: 7.ee081174.chunk.js.map
Only in build_webui/static/js: main.2b881fc1.js
Only in build_webui/static/js: main.2b881fc1.js.map
Only in webui_QmXc9raDM1M5G5fpBnVyQ71vR4gbnskwnB9iMEzBuLgvoZ/static/js: main.bf82967b.js

Looking for a Solution

So far options are:

• (A) Recreate release build and provide step-by-step instruction for reviewer
  • Are we able to provide steps on how to reproduce minified output from QmXc9raDM1M5G5fpBnVyQ71vR4gbnskwnB9iMEzBuLgvoZ ?
  • Who built it? What was the version of node/npm? Will steps work cross-platform?
• (B) Update build pipeline to support reproducible build and make a new release
  • Is it technically feasible to provide reproducible build of ipfs-webui that works the same across all platforms?
    • We provide docker-based build pipeline for ipfs-companion, which is an overkill but works.
• (C) Remove bundled Web UI from IPFS Companion and load it from IPFS
  • We may do this anyway just to restore ipfs-companion in Firefox addon store (and add bundled Web UI when it is possible to build in reproducible manner)
• (D) ?

@olizilla @alanshaw @hugomrdias would love to hear your thoughts on this

[olizilla]

webui has never guaranteed a reproducible build. We've relied on Jenkins to create a traceable build / release process.

In the short term, I think webui should be extracted from companion and loaded from IPFS as the daemon does.

In the longer term, I'm in favour of making the webui build be reproducible.

[olizilla]

Als of note, Jenkins is being decommissioned, so now is a good time for us to revisit this!

[lidel]

We went with (C) – ipfs/ipfs-companion#680
(Removed bundled Web UI from IPFS Companion and load it from IPFS)