Add a concept of HUGOBIN and a way to install binary dependencies

[bep]

Dart Sass with Hugo is incredibly powerful, but it's also a pain to install. #8299 has been stranded because 1. Go doesn't have a way of doing this built-in and 2. Building a custom solution would be a challenge to do securely.

This proposal outlines a way to do this. This is deliberately a little vague about the details.

• We introduce a new HUGOBIN env var. The user (or installation script) would be responsible to add this folder to the PATH.
• We introduce a new binary component type which would allow mounting of files into this HUGOBIN folder.
• We need to handle archives in the above, obviously.

Since GitHub releases is separated from the Git repo, we need to somehow wrap these binaries in a module somehow:

• The simplest would obviously be to duplicate the binary archives, but that would create rather large downloads when you only need one out of potentially many platforms.,
• A more sensible solution would probably to just store version information + file hashes and then download the correct architecture when needed.

The above should

1. Have a security model that's fairly easy to reason about.
2. Have a familiar versioning scheme.
3. Allows theme authors to define "Dart Sass" as a dependency.

[jmooring]

just store version information + file hashes and then download the correct architecture when needed.

Agree. This is what the Scoop, Chocolatey, and Snap packages for Embedded Dart Sass do.

[bep]

@jmooring can I borrow your attention for a minute. I have a working prototype of this running and, as usual, I'm a little paranoid about security. After a little testing I have concluded that having the user do PATH fiddling to make this work, isn't ... great. So with what I have now I use abs paths to the "imported" binaries, and this should be enough to get Dart Sass:

[[module.imports]]
path="github.com/gohugoio/hugo-mod-bin-dartsass"

The SHA56 of the downloaded archive (stored in the Hugo Module) is checked on installation.

The one thing I'm a little worried about then is someone creating a fake Hugo Module with a fake dartsass binary. A little far fetched, perhaps, but to improve on this I have added a new whitelist to the security config: AllowBinFromModules: "^github.com/gohugoio/" -- thinking that most such binaries will be hosted by us, and we can be trusted.

What do you think?

Some links:

• https://github.com/gohugoio/hugo-mod-bin-dartsass
• https://github.com/gohugoio/hugo-mod-bin-config-from-github-release

[jmooring]

Thoughts, in no particular order...

1. Can we hardcode AllowBinFromModules: "^github.com/gohugoio/", without the ability to override via site configuration or env vars? You and I export env vars from our .bashrc files or whatever to make sure that, when we clone a project, we're not exposed when the cloned project overrides the default security config. But I think this is rare; I would be surprised if more than a handful of users have configured their systems this way.
2. Will there be any challenges setting the perms to 711, or 700 if the exec will be owned by the user?
3. What happens if I vendor the module?
4. Will the execs end up in HUGO_CACHEDIR, which for most users is volatile?

Finally, and please be don't be offended, but it seems like installing Go is a little more complicated than using a package manager, assuming they've already installed the package manager, which many have not, so maybe requiring Go is OK.

And a couple of FYI's:

• I updated the Snap, Scoop, and Choco packages to 1.58.3 last night. The Snap is live, the other two are pending approval.

• The Sass team changed the Embedded Dart Sass packaging for Linux. So manual installation is now like Windows:

  dart-sass-embedded (script) --> dart (exec) --> dart-sass-embedded.snapshot
  

  This change can break existing CI/CD deployments. I've updated the Netlify instructions in this forum post.

[jmooring]

There's also this:
https://www.npmjs.com/package/sass-embedded

npm i sass-embedded

It is platform aware. Not sure you want to go down that path, but we do have precedence with POSTCSS.

[bep]

• "Can we hardcode AllowBinFromModules?" Yes, or a variation of that. I don't like having security rules that's impossible to turn off.
• "Will there be any challenges setting the perms to 711, or 700 if the exec will be owned by the user?" I'm not sure, but I will ... investigate.
• "What happens if I vendor the module?" The module's config file will be vendored, which is what we need to install the correct binary.
• "Will the execs end up in HUGO_CACHEDIR"; in my initial tests I have put it in a separate folder, but I'm toying about putting it below HUGO_CACHEDIR -- the installation of these should be both fast and reproducible and it would be great if it could join the same --gc regime of the file cache etc.
• "I updated the Snap, Scoop, and Choco packages to 1.58.3 last night. The Snap is live, the other two are pending approval." And that is fantastic. This particular issue tries to cover one big hole in the current situation: The CI servers (Netlify, Vercel etc.). The current workarounds are extremely challenging to explain to the common Hugo user.
• "It is platform aware. Not sure you want to go down that path, but we do have precedence with POSTCSS." PostCSS isn't installable without NPM (as far as I know), which is a difference. Also, I have looked at it, and I don't understand how they have wired up their installation (I see no file hashes anywhere).

I'm not totally sure I'm able to land this issue anytime soon, but I appreciate your feedback.

[bep]

Thinking a little more on this, I'll let this be as it is for now:

• There will be permissions issue on systems where Hugo isn't normally run as an admin user (Windows; which I assume is also an issue with npm i; we could probably get by this with a hugo install command or something.)
• There is a certain threshold with installing binaries on peoples' computers that I'm having hard time get passed, even if I think that this is "as secure as it gets".

[brycewray]

The way 0.114.0 works with the new Dart Sass packaging is truly outstanding; kudos! Does this in any way suggest a path to resolving this particular issue (and/or #8299), or is it of no value in that regard?

Edit, 2023-06-23: Actually, on the other hand, @jmooring has now made an excellent argument against further worrying about this.