Serve assets with compression #69

exreplay · 2022-04-10T17:25:08Z

Environment

Operating System: macOS
Node Version: v16.14.0
Nitro Version: 0.2.1
Package Manager: yarn@3.1.1

Reproduction

n/a

Describe the bug

It would be a nice addition to nitro if there is be a possibility to change the response of static assets. Then it would also be possible to implement content encoding like brotli or gzip.

Additional context

There does exist a discussion in the nuxt repo. https://github.com/nuxt/framework/discussions/3472

In addition to the discussion i already found the corresponding lines, which i changed in nuxt, in the nitro package:

https://github.com/unjs/nitro/blob/main/src/runtime/static.ts#L68
https://github.com/unjs/nitro/blob/main/src/rollup/plugins/public-assets.ts#L44

Logs

No response

Update 21.07.2022

I wrote a module which can handle the compression https://github.com/exreplay/nuxt-compression. This is still no official way and messes with the internal nitro code but there is no need to use patches.

The text was updated successfully, but these errors were encountered:

exreplay · 2022-04-12T07:49:41Z

If it helps, i updated my patch file to work with the nitropack package

.yarn/patches/nitropack-npm-0.2.4-598f2e4fe6

diff --git a/dist/chunks/prerender.mjs b/dist/chunks/prerender.mjs
index 7e55bb75fefffdd26b0fe29ac08cf6c20294f559..8b1d659254b45707b8bd519e4faa43fb37cda687 100644
--- a/dist/chunks/prerender.mjs
+++ b/dist/chunks/prerender.mjs
@@ -512,15 +512,21 @@ function publicAssets(nitro) {
   }
   return virtual({
     "#nitro/virtual/public-assets-data": `export default ${JSON.stringify(assets, null, 2)};`,
+    "#nitro/virtual/public-assets-config": readFileSync(resolve(nitro.options.rootDir, nitro.options.static), 'utf8'),
     "#nitro/virtual/public-assets-node": `
 import { promises as fsp } from 'fs'
 import { resolve } from 'pathe'
 import { dirname } from 'pathe'
 import { fileURLToPath } from 'url'
 import assets from '#nitro/virtual/public-assets-data'
+import { hook } from '#nitro/virtual/public-assets-config'
 const mainDir = dirname(fileURLToPath(globalThis.entryURL))
-export function readAsset (id) {
-  return fsp.readFile(resolve(mainDir, assets[id].path)).catch(() => {})
+export async function readAsset (id, res) {
+  let assetPath = resolve(mainDir, assets[id].path);
+
+  if (hook) assetPath = await hook(assetPath, res);
+
+  return fsp.readFile(assetPath).catch(() => {})
 }`,
     "#nitro/virtual/public-assets": `
 import assets from '#nitro/virtual/public-assets-data'
diff --git a/dist/runtime/static.mjs b/dist/runtime/static.mjs
index 3efe120edc9d150f9f4cacdfbd6df03ec350a9e2..b2e8116a4a8203820f69777cde20b2a2ede1d954 100644
--- a/dist/runtime/static.mjs
+++ b/dist/runtime/static.mjs
@@ -1,6 +1,7 @@
 import { eventHandler, createError } from "h3";
 import { withoutTrailingSlash, withLeadingSlash, parseURL } from "ufo";
 import { getAsset, readAsset, isPublicAssetURL } from "#nitro/virtual/public-assets";
+import { maxAge } from "#nitro/virtual/public-assets-config";
 const METHODS = ["HEAD", "GET"];
 export default eventHandler(async (event) => {
   if (event.req.method && !METHODS.includes(event.req.method)) {
@@ -48,6 +49,9 @@ export default eventHandler(async (event) => {
   if (asset.mtime) {
     event.res.setHeader("Last-Modified", asset.mtime);
   }
-  const contents = await readAsset(id);
+  if (maxAge) {
+    event.res.setHeader("Cache-Control", `max-age=${maxAge}, immutable`);
+  }
+  const contents = await readAsset(id, event.res);
   event.res.end(contents);
 });

package.json

{
  "resolutions": {
    "nitropack": "patch:nitropack@npm:0.2.4#.yarn/patches/nitropack-npm-0.2.4-598f2e4fe6"
  }
}

nathanchase · 2022-07-18T13:27:35Z

I was just looking for a solution to serve assets (js|mjs|json|css|html, etc.) correctly via Nitro, as it seems to be one of the biggest contributing factors to a reduced Lighthouse score. I have https://github.com/nuxt-modules/compression installed and configured in nuxt.config.ts -

modules: [
    ['@nuxt-modules/compression', {
      algorithm: 'brotliCompress',
    }],
    ...
]

but it doesn't seem like Nitro is serving anything with brotli.

nathanchase · 2022-07-18T13:29:13Z

Similarly, can we set a default cache policy for assets?

exreplay · 2022-07-18T13:40:50Z

@nathanchase I don't know if you used my solution but if you want to, here are some up to date instructions with a new patch for nitro version 0.4.12.

Add the patch to your repo and use it with your package manager.

diff --git a/dist/chunks/prerender.mjs b/dist/chunks/prerender.mjs
index 5af8310adc4b4773d459cd231fbade416da65492..c23e6f3270abfcf1174fd38e057ccfb15054c37b 100644
--- a/dist/chunks/prerender.mjs
+++ b/dist/chunks/prerender.mjs
@@ -527,15 +527,19 @@ function publicAssets(nitro) {
   }
   return virtual({
     "#internal/nitro/virtual/public-assets-data": `export default ${JSON.stringify(assets, null, 2)};`,
+    "#internal/nitro/virtual/public-assets-config": readFileSync(resolve(nitro.options.rootDir, nitro.options.static), 'utf8'),
     "#internal/nitro/virtual/public-assets-node": `
 import { promises as fsp } from 'fs'
 import { resolve } from 'pathe'
 import { dirname } from 'pathe'
 import { fileURLToPath } from 'url'
 import assets from '#internal/nitro/virtual/public-assets-data'
-export function readAsset (id) {
+import { hook } from '#internal/nitro/virtual/public-assets-config'
+export async function readAsset (id, res) {
   const serverDir = dirname(fileURLToPath(import.meta.url))
-  return fsp.readFile(resolve(serverDir, assets[id].path))
+  let assetPath = resolve(serverDir, assets[id].path)
+  if (hook) assetPath = await hook(assetPath, res)
+  return fsp.readFile(assetPath);
 }`,
     "#internal/nitro/virtual/public-assets": `
 import assets from '#internal/nitro/virtual/public-assets-data'
diff --git a/dist/runtime/static.mjs b/dist/runtime/static.mjs
index 5b19f1fc9ff8a3d975766a3ba82d05291901c138..85fd2df47e3671b7de72d5141b4d3d73341588a2 100644
--- a/dist/runtime/static.mjs
+++ b/dist/runtime/static.mjs
@@ -1,6 +1,7 @@
 import { eventHandler, createError } from "h3";
 import { withoutTrailingSlash, withLeadingSlash, parseURL } from "ufo";
 import { getAsset, readAsset, isPublicAssetURL } from "#internal/nitro/virtual/public-assets";
+import { maxAge } from "#internal/nitro/virtual/public-assets-config";
 const METHODS = ["HEAD", "GET"];
 export default eventHandler(async (event) => {
   if (event.req.method && !METHODS.includes(event.req.method)) {
@@ -48,6 +49,9 @@ export default eventHandler(async (event) => {
   if (asset.mtime) {
     event.res.setHeader("Last-Modified", asset.mtime);
   }
-  const contents = await readAsset(id);
+  if (maxAge) {
+    event.res.setHeader("Cache-Control", `max-age=${maxAge}, immutable`);
+  }
+  const contents = await readAsset(id, event.res);
   event.res.end(contents);
 });

Install the vite-plugin-compression plugin.

Update your nuxt.config.ts with the plugin and an extra nitro param:

import { defineNuxtConfig } from 'nuxt';
import viteCompression from 'vite-plugin-compression';

export default defineNuxtConfig({
  nitro: {
    // eslint-disable-next-line @typescript-eslint/ban-ts-comment
    // @ts-ignore
    static: './staticConfig.mjs'
  },
  vite: {
    plugins: [viteCompression({ algorithm: 'brotliCompress' })]
  }
});

Create the staticConfig.mjs in your root directory with the following content:

import { promises, constants } from 'fs';

export const maxAge = 60 * 60 * 24 * 365;

export async function hook(assetPath, res) {
  if (assetPath.endsWith('.mjs') || assetPath.endsWith('.css')) {
    try {
      await promises.access(`${assetPath}.br`, constants.R_OK | constants.W_OK);
      assetPath = `${assetPath}.br`;
      res.setHeader('Content-Encoding', 'br');
    } catch {}
  }

  return assetPath;
}

What all this is doing, is first patch the nitro package to let you hook into the static assets and manipulate the response, which we can then use inside our staticConfig.mjs. Inside there we use the hook and search for brotli files and if we find them, serve them with the correct header. The vite-plugin-compression just generates all the brotli files during the build step.

Currently there is no other way to do that as far as i know but it works perfectly fine for me and i also use it in production like that.

nathanchase · 2022-07-18T15:35:38Z

@nathanchase I don't know if you used my solution but if you want to, here are some up to date instructions with a new patch for nitro version 0.4.12.

Add the patch to your repo and use it with your package manager.

Thanks! I'm using pnpm, so I'm not sure exactly how to apply this patch.
I have the following in my package.json:

"resolutions": {
  "nitropack": "patch:nitropack@npm:0.2.4#/patches/nitropack-npm-0.2.4-598f2e4fe6.patch"
},
"pnpm": {
  "patchedDependencies": {
    "nitropack": "patch:nitropack@npm:0.2.4#/patches/nitropack-npm-0.2.4-598f2e4fe6.patch"
  }
},

and I copy/pasted your patch to /patches/nitropack-npm-0.2.4-598f2e4fe6.patch in my repo

but I get a pnpm: ENOENT: no such file or directory on the patch... not sure where I'm going wrong. I've never had to apply a patch before, so maybe I'm missing some part of the process?

exreplay · 2022-07-19T07:48:59Z

@nathanchase Which nuxt version are you using? If you are using the latest rc.5 you need to make sure to install the latest nitropack version otherwise pnpm will fail to install. Also you don't need the resolutions part.

This is how my package.json looks like:

{
  "pnpm": {
    "patchedDependencies": {
      "nitropack@0.4.12": "patches/nitropack@0.4.12.patch"
    }
  }
}

The path to my patch file is ./patches/nitropack@0.4.12.patch.

exreplay · 2022-07-19T07:54:47Z

@nathanchase i just saw that rc.6 is out. The above is also true to that.

nathanchase · 2022-07-19T15:46:19Z

@nathanchase Which nuxt version are you using? If you are using the latest rc.5 you need to make sure to install the latest nitropack version otherwise pnpm will fail to install. Also you don't need the resolutions part.

This is how my package.json looks like:
{
  "pnpm": {
    "patchedDependencies": {
      "nitropack@0.4.12": "patches/nitropack@0.4.12.patch"
    }
  }
}
The path to my patch file is ./patches/nitropack@0.4.12.patch.

This worked! I think specifying the version of nitropack ("nitropack": "0.4.12") is what did it. Thank you for your guidance and work on this! Look forward to seeing this supported directly and with documentation in Nuxt 3!

"pnpm": {
    "patchedDependencies": {
      "nitropack@0.4.12": "patches/nitropack-npm-0.2.4-598f2e4fe6.patch"
    }
  },

nathanchase · 2022-07-19T15:52:18Z

Well, I spoke too soon. It worked great when I ran pnpm build and then pnpm preview on my local machine. As soon as I deployed to production, they're all 500. I wonder what the difference is?

exreplay · 2022-07-19T16:08:26Z

@nathanchase did you check if all the brotli files are available on your production server?

nathanchase · 2022-07-19T16:21:31Z

I'm getting this error now:

exreplay · 2022-07-19T16:43:38Z

that is weird because you said that it worked on your local machine. it looks like it tries to read a file but instead got a directory. there is probably no way you can provide a reproduction?

nathanchase · 2022-07-19T22:51:12Z

OK. I figured it out. I had to delete @nuxt-modules/compression and remove it from the modules in nuxt.config. So just having:

nitro: {
    // eslint-disable-next-line @typescript-eslint/ban-ts-comment
    // @ts-ignore
    static: './staticConfig.mjs',
  },

  vite: {
    plugins: [viteCompression({ algorithm: 'brotliCompress' })],
  },

works. Whew! Finally! Thanks again for your assistance!

exreplay · 2022-07-20T06:42:21Z

glad i could help and it works now! I hope that they soon add something like a hook and we can update the https://github.com/nuxt-modules/compression package

CatalinGheorghiu · 2022-07-21T08:26:26Z

@exreplay How would I achieve this with yarn? I am a little confused about how this needs to be added to package.json

{ "resolutions": { "nitropack": "patch:nitropack@npm:0.2.4#.yarn/patches/nitropack-npm-0.2.4-598f2e4fe6" } }

exreplay · 2022-07-21T08:48:03Z

@CatalinGheorghiu what yarn version are you using?

With yarn your package.json should look like this

{
  "resolutions": {
    "nitropack": "patch:nitropack@npm:0.4.12#.yarn/patches/nitropack-npm-0.4.12-0dce97461d"
  }
}

Catalin-G · 2022-07-21T08:57:35Z

@CatalinGheorghiu what yarn version are you using?

With yarn your package.json should look like this
{
  "resolutions": {
    "nitropack": "patch:nitropack@npm:0.4.12#.yarn/patches/nitropack-npm-0.4.12-0dce97461d"
  }
}

Yarn version: 1.22.19
I have created a patches folder as @nathanchase did and copy/pasted the content in there

exreplay · 2022-07-21T11:17:38Z

afaik the patch command is not available in yarn version 1, you need at least version 2. if you want to use the current stable yarn version in a project you can run the following command in the root of your project.

yarn set version stable

If that doesn't work, you can head to https://yarnpkg.com/getting-started/install to see how you can install it.

This should create a .yarn folder, where you can create the patch directory inside and copy the patch file into.

exreplay · 2022-07-21T14:30:28Z

I found another way to make all this work without the need for a patch and everything else. I wrote a module real quick for everyone to use. You can find the package here https://github.com/exreplay/nuxt-compression. I already tried it locally and in my production environment with success.

nathanchase · 2022-07-21T15:33:10Z

I found another way to make all this work without the need for a patch and everything else. I wrote a module real quick for everyone to use. You can find the package here https://github.com/exreplay/nuxt-compression. I already tried it locally and in my production environment with success.

Can confirm it works for me locally and in production. Great work!

nathanchase · 2022-08-20T14:53:33Z

Another bump for https://github.com/exreplay/nuxt-compression to be considered as a default, zero-config option in Nitro, to automatically enable gzip/brotli compression for all static assets and set Cache-Control/maxAge values.

pi0 · 2022-08-30T18:29:59Z

Hi there and sorry it took long to add this feature.

Generally, I would recommend you use a CDN that can natively support compression. It is much more efficient to handle best compression with long term caching.

nuxt/framework#449 adds a new option compressPublicAssets that can be optionally enabled to enable compression (feedback and improvements more than welcome 🙏🏼)

To enable this option for Nuxt 3, use edge channel (or wait for rc.9) and:

defineNuxtConfig({
 nitro: { compressPublicAssets: true }
})

fedeweb · 2022-09-03T13:32:11Z

Hi @pi0,

thanks for the release. The nitro compression is working only in build mode. In generate mode it doesn't work, am I miss something?

pi0 · 2022-09-03T13:33:23Z

@fedeweb Can you please explain more? Generate mode should behave similarly and generate .gzip files it doesn't? (A reproduction would be nice 🙏🏼 )

fedeweb · 2022-09-03T13:56:41Z

@pi0 I attached here 2 screenshots of some of the files generated with "generate" and "build"

This is my nuxt config file:

pi0 · 2022-09-03T14:06:39Z

Thanks! I can locally reproduce. Moved to nuxt/nuxt#14794

exreplay mentioned this issue Jul 18, 2022

refactor to offical nuxt 3 module structure Baroshem/compression#1

Open

pi0 changed the title ~~Change Content-Encoding for assets~~ Serve assets with compression Aug 30, 2022

pi0 self-assigned this Aug 30, 2022

pi0 added the enhancement New feature or request label Aug 30, 2022

pi0 mentioned this issue Aug 30, 2022

feat: opt-in option to compress public assets using gzip and br #449

Merged

7 tasks

pi0 closed this as completed Aug 30, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Serve assets with compression #69

Serve assets with compression #69

exreplay commented Apr 10, 2022 •

edited

Loading

exreplay commented Apr 12, 2022 •

edited

Loading

nathanchase commented Jul 18, 2022

nathanchase commented Jul 18, 2022

exreplay commented Jul 18, 2022

nathanchase commented Jul 18, 2022 •

edited

Loading

exreplay commented Jul 19, 2022

exreplay commented Jul 19, 2022

nathanchase commented Jul 19, 2022

nathanchase commented Jul 19, 2022

exreplay commented Jul 19, 2022

nathanchase commented Jul 19, 2022

exreplay commented Jul 19, 2022

nathanchase commented Jul 19, 2022 •

edited

Loading

exreplay commented Jul 20, 2022 •

edited

Loading

CatalinGheorghiu commented Jul 21, 2022

exreplay commented Jul 21, 2022

Catalin-G commented Jul 21, 2022

exreplay commented Jul 21, 2022 •

edited

Loading

exreplay commented Jul 21, 2022

nathanchase commented Jul 21, 2022

nathanchase commented Aug 20, 2022

pi0 commented Aug 30, 2022

fedeweb commented Sep 3, 2022 •

edited

Loading

pi0 commented Sep 3, 2022

fedeweb commented Sep 3, 2022

pi0 commented Sep 3, 2022

Serve assets with compression #69

Serve assets with compression #69

Comments

exreplay commented Apr 10, 2022 • edited Loading

Environment

Reproduction

Describe the bug

Additional context

Logs

Update 21.07.2022

exreplay commented Apr 12, 2022 • edited Loading

nathanchase commented Jul 18, 2022

nathanchase commented Jul 18, 2022

exreplay commented Jul 18, 2022

nathanchase commented Jul 18, 2022 • edited Loading

exreplay commented Jul 19, 2022

exreplay commented Jul 19, 2022

nathanchase commented Jul 19, 2022

nathanchase commented Jul 19, 2022

exreplay commented Jul 19, 2022

nathanchase commented Jul 19, 2022

exreplay commented Jul 19, 2022

nathanchase commented Jul 19, 2022 • edited Loading

exreplay commented Jul 20, 2022 • edited Loading

CatalinGheorghiu commented Jul 21, 2022

exreplay commented Jul 21, 2022

Catalin-G commented Jul 21, 2022

exreplay commented Jul 21, 2022 • edited Loading

exreplay commented Jul 21, 2022

nathanchase commented Jul 21, 2022

nathanchase commented Aug 20, 2022

pi0 commented Aug 30, 2022

fedeweb commented Sep 3, 2022 • edited Loading

pi0 commented Sep 3, 2022

fedeweb commented Sep 3, 2022

pi0 commented Sep 3, 2022

exreplay commented Apr 10, 2022 •

edited

Loading

exreplay commented Apr 12, 2022 •

edited

Loading

nathanchase commented Jul 18, 2022 •

edited

Loading

nathanchase commented Jul 19, 2022 •

edited

Loading

exreplay commented Jul 20, 2022 •

edited

Loading

exreplay commented Jul 21, 2022 •

edited

Loading

fedeweb commented Sep 3, 2022 •

edited

Loading