[hlx build] Improve static files handling

[tripodsan]

As a first approach, the static files are just copied over to .hlx/dist. while this works for local development, it does not work for production. eventually the static files need to be served to the CDN.

similar to the approach in https://github.com/adobe/project-helix/pull/266, the static files could be made available via wsk action. however, I don't think that the wsk-action should load it again from git, because the codeload usually provides too much content.

The static resources could be bundled in the wsk action that delivers them. either in a generic static-server action contains all the resources needed to the project, or alongside the respective html.js action.

Alternatively, a gh-pages approach would be better instead of abusing wsk as a static file server. For each deployment, the static files are committed into a helix-static branch, and the ref of the current state is used in the strains.yaml to configure the static server.

how it works

having a separate static-server action is better, as the fundamental idea of just delivering static assets is easily applied to other mediums (S3, etc.)

1. the parcel-plugin-htl is enhanced so that it also processes the sub-assets referenced in the htl code Optionally process sub-assets during htl parsing parcel-plugin-htl#16
2. the plugin generates the .hlx/build/html.js and also the .hlx/dist/* variants of the referenced assets.
3. the hlx deploy deploys not only the html.js action, but also a static-server action that provides a simple server for the static files or commits the static files into helix-static.

[trieloff]

How would this impact adobe/project-helix#266 ?

I would think that static assets are just the fallback case of dynamic assets (when parcel doesn't know what to do with an asset, it copies it). If we can have only one way to deliver assets (static or dynamic), it would definitely simplify things.

When it comes to serving assets, I see following options, all coming with some cons:

1. Turn the entire project into an archive and deploy it to OpenWhisk (con: you will need many actions per branch, and just got a very expensive file server)
2. Use S3 instead of OpenWhisk (con: you need to figure out a naming schema for branches and another set of credentials)
3. Use gh-pages (con: you can't have branches anymore, and you have to commit files that are compiled, blowing up your repo size)
4. Use codeload (con: you are loading stuff that's unnecessary)

Right now, 2 and 4 look like better options than 1 or 3. The biggest pain point with 3 would be the resulting multi-branch mess, as you can deploy branches, tags, or commits and you would have branch-branches, tag-branches, and commit-branches to serve the static (or compiled) assets.

[tripodsan]

(thanks @trieloff for re-iterating my suggestions :-)

I disagree, dealing with a branch similar to gh-pages is not a problem, if you automate it.
for example, in https://github.com/tripodsan/SimplyUtilities, I use a simple tool that builds the doc and "pushes" it to gh-pages.

IMO, abusing openwhisk for static file delivery is the worst of all options (either way, codeload or deployed...)

As for parcel, it should only include the (processed) files that are referenced in the content, if possible. especially for clientside CSS and JS, we want parcel to handle it correctly. so it would be good if it can parse the .htl as HTML and process the dependencies.

[tripodsan]

Let's create POCs for the 3/4 variants. maybe even pluggable:

hlx build --static-content=[s3|gh|bundled|codeload]

• s3 Uses S3 as storage
• gh Uses (this) github repository as storage
• bundled Bundles the static files together with the respective component
• codeload Uses dynamic rendering of the static files using a function in wsk.
• azure Uses Azure Blob Storage as storage

[davidnuescheler]

i think the current way of handling static files could be improved in two ways

(1) the file layout in the working directory should be the same as exposed through the URL, meaning that having a static directory that then turns into dist directory during the build process is confusion.

(2) i think static files should be able to be placed anywhere in the URL space, if i put an index.html into the root of my repo i should be able to just request it as /index.html

[davidnuescheler]

as for the deployment of static files, I think we should be careful to introduce more system components like S3 or Azure as well as additional copies in GitHub. It may very well be that we will have to resort to some of these solutions for performance/scale reasons, but for now i think the simplest solution would be to just deploy static files into Adobe I/O runtime as an action.

I fully appreciate that there may be multitude of limitations and possibly undesired side-effects with that approach, but I think we should consider exhausting those before introducing additional complexity.

[trieloff]

How would we distinguish between files that are meant to be served statically vs. files that are meant to be served dynamically?

Also, I don't think most users would appreciate if we just started serving their template files as is.