Precaching is in a bad state following a failed activation in Firefox

[rolandstarke]

Library Affected:
workbox-v3.6.3

Browser & Platform:
Firefox 63.0.3 Windows

Issue or Feature Request Description:
Basically the same as #1528, but it would be cool if the problem would be solved in the library itself.

I am using the InjectManifest plugin (its the same with GenerateSW).

After a deploy the service woker claims to have updates. When closing the browser and opening it again one js file is deleted form the cache. The index.html is still on the old version that depends on this js file. So the page can't load anymore.

Here the config (i am using https://www.npmjs.com/package/@vue/cli-plugin-pwa)

vue.config.js

process.env.VUE_APP_VERSION = (new Date()).toISOString();

module.exports = {
    devServer: {
        proxy: 'http://localhost:8000'
    },
    // ...other vue-cli plugin options...
    pwa: {
        name: 'My App',
        themeColor: '#4DBA87',
        msTileColor: '#000000',
        appleMobileWebAppCapable: 'yes',
        appleMobileWebAppStatusBarStyle: 'black',
        workboxPluginMode: 'InjectManifest',
        workboxOptions: {
            //https://developers.google.com/web/tools/workbox/modules/workbox-webpack-plugin
            swSrc: __dirname + '/public/service-worker.js',
            dontCacheBustUrlsMatching: /\.[0-9a-f]{8}\./,
            importWorkboxFrom: 'local',
        }
    }
}

__dirname + '/public/service-worker.js'

workbox.setConfig({
    debug: true
});
workbox.core.setLogLevel(workbox.core.LOG_LEVELS.debug);


workbox.core.setCacheNameDetails({prefix: "my-garden"});

/**
 * The workboxSW.precacheAndRoute() method efficiently caches and responds to
 * requests for URLs in the manifest.
 * See https://goo.gl/S9QRab
 */
self.__precacheManifest = [].concat(self.__precacheManifest || []);
//workbox.precaching.suppressWarnings();
workbox.precaching.precacheAndRoute(self.__precacheManifest, {});

workbox.routing.registerNavigationRoute("/index.html", {
  
  blacklist: [/^\/(api|admin|vendor|storage)/,/\.[a-z]{2,5}$/i],
});

The server is configured to not serve 404, but to fallback to index.html instead.

[rolandstarke]

Here pictures of the log output

1. after every page refreesh before the deploy: https://drive.google.com/open?id=1oIJYhL1l650A39La2iK41o7Dz8_1RlUr

2. the first page refresh after deploy: https://drive.google.com/open?id=1SfrKJGWd1DIXyJHMTKWIJd6WwghemcX2

3. first open after closing all tab: https://drive.google.com/open?id=1HteAVJJWA4l8FmNoVIAXypjm_wVlbbeA

Other info:

• The error in the 2. image is the issue: Firefox: fails on SW register/update cause it search previous precache-manifest file #1745
• I guess its related to Precaching temp #1342 and similar to the other bad app state issues.

[jeffposnick]

Thanks for the very detailed writeup; I'll try to reproduce the issue locally.

One question for you: you mentioned that you're seeing this in Firefox 63. Can you also reproduce that behavior in other browsers, like Chrome?

That message in this screenshot about the service worker attempting to pull in what I'd imagine is the old precache manifest via importScripts(), and getting back the text/html fallback response instead, definitely look suspicious. A theory that I have has to do with Firefox being more aggressive about checking for updates to resources pulled in to the SW scope via importScripts() vs. how Chrome behaves at the moment. That might lead to it entering into a state where it has an inaccurate precache manifest right after an update. So a quick check to see if Chrome is doing the same thing would be 👍

[rolandstarke]

Hey, yes you are right thats the old precache manifest and the fallback response from the server. Chrome, as far as i know, did not retry to load the old manifest and cached it. I can confirm with the access.log that on every page reload Chrome only requests the service-worker.js and Firefox additionally requests the precache manifest and 4 files in /workbox-v3.6.3/, (So what you said with the aggressive checking for updates in Firefox.)

Maybe the precache manifest file could be included in itself to overcome this issue. For now i just set a public max age 1 year cache header for this file to overcome this error.

....

The main issue is still there. I could not reproduce it in chrome.

I made some side by side images On the left Firefox on the right Chrome. (btw in Firefox its reproducible like 80% of the times.)

1. active service worker, everything is cached and works normal
   

2. deploy (in this case i just rebuild the app, there is a timestamp of the buildtime in the app, that is rendered in the frontent, so dist/js/app.{hash}.js and dist/js/index.html always change.)

3. reload the page
   

4. reload the page again (not sure if necessary, but in my first attempt to reproduce it a single reload did not result in a broken app)
   

5. close tabs
   

6. open page again
   

7. reload page (in Firefox the site is now broken)
   

[jeffposnick]

(CC: @josephliccini who brought up what looks to be the root cause in w3c/ServiceWorker#1372, and @wanderview whom I've been bugging about this as an FYI.)

Thanks again for all those debugging steps. I think I have a decent handle on what you're seeing in Firefox.

Based on this screenshot, what looks like is happening is that the activate event is fired on the waiting service worker, during which all of the Response objects that were in Workbox's temporary cache are supposed to be moved over to the activate cache. However, "something" happens in Firefox that causes that activation to fail. I believe those 3 blank "Error" log messages you can see at the top of the screenshot correspond to this activation failure. (I have no idea what's causing this failure, as it appears to be internal to Firefox—a catch() at the end of the promise passed to event.waitUntil() during activation doesn't get triggered.)

So, for whatever reason, the activate handler does not fully complete, and as per w3c/ServiceWorker#1372 (comment), in Firefox, the worker is discarded and a new install event is fired. That's why you see a message about precaching starting over again after those blank "Error" log messages.

The problem here is that the first thing the install handler does is clear out the temporary cache: https://github.com/GoogleChrome/workbox/blob/master/packages/workbox-precaching/controllers/PrecacheController.mjs#L167-L177

Workbox does not expect to be in a state where there's anything useful in the temporary cache at this point; it expects that if there's a new install event, it can start fresh with the current set of assets in the precache manifest.

...but for reasons that have to do with how we keep track of revision information in IndexedDB, that assumption will lead to the behavior you're seeing in Firefox if we get an unexpected install event following an incomplete activate event.

I believe that the workaround is Workbox just not to delete the temporary cache at the start of the install handler, and assume that if there's anything left over in it, it should actually be copied over during the next successful activate event (at which point it will be cleaned up).

As far as I can tell, that's a low-risk change, but it's likely something we'd make against the next branch and give folks a chance to test in the next v4 beta release.

[jeffposnick]

It might not be the best bug report in the world, but I tried to capture the steps that led to Firefox failing to activate the updated service worker at https://bugzilla.mozilla.org/show_bug.cgi?id=1514030

We still want to address this from a Workbox perspective, but at the same time, if Firefox appears to get into a bad state where it fails to activate a new service worker, that's hopefully something that can be resolved too.

[jeffposnick]

This is addressed in Workbox v4, where the only thing workbox-precaching performs in the activate handler is cleanup.