index.html cached in a bad state when service worker updates after a deployement

[AlexandreBonaventure]

Library Affected:
workbox-webpack@3.1.0

Browser & Platform:
Google Chrome v67

Issue or Feature Request Description:
Hi, we are using workbox in our company to power our PWAs. We noticed a bug that is hard to reproduce and it is affecting some of our clients sporadically. They have to clear their cache in order to get the app working again.
I stumbled upon the bug yesterday myself when reloading after a deployment, here's the steps :

1. we deployed a new version of the app (we are using webpack plugin, so a new precacheManifest as been generated at build time)
2. we reload the page in production to get the update
3. a new service worker is installing, after finishing we automatically reload the page using window.location.reload() as a callback of the statechange event (and checking event.target.state === 'installed')
4. After reloading the page is broken it won't load assets.

What happened ?
The new service worker is active and up to date, the precache manifest as well, I can see old revisioned assets are successfully deleted from the cache using Chrome devtool but when I look at the content of the cached index.html is the one from the previous version. It is referencing for assets matching the previous version revision, it can't fetch them because they are deleted from the cache by this time.

I don't know if this is a bug of workbox not updating the index.html in cache despite the new revision number, or somehow a timing issue with our server, serving the new service worker before busting some internal cache for the index.html (it is served with max-age=0 headers).

If anyone can bring some lights with this, it could be very helpful to solve this problem. Unfortunately, I can't provide any repro since it is very hard to reproduce, but if someone can give clues or workaround, it may help me to understand the issue better.
Is there any recommended practices for dealing with this particular problem.

Thank you.

ps: here a screenshot I took, on the left it is an incognito window showing the right index.html being cached. On the right, the original window showing the obsolete index.html in cache.

[AlexandreBonaventure]

My conf goes like this :

module.exports = new GenerateSW({
  swDest: 'service-worker.js',
  cacheId: 'service-worker',
  navigateFallback: '/index.html',
  navigateFallbackBlacklist: [/^\/static/],
  importScripts: ['/static/addons.sw.js']
})

[jeffposnick]

Apologies for the issues you're running into. Based on what you describe, since you're not using skipWaiting, what I would expect is that calling window.location.reload() would not be enough to move the new service worker into the activated state. The new service worker should remain in waiting state until you either call skipWaiting() or you close (not reload) all the tabs that are under the old service worker's scope. The previously cached assets should not be deleted until after the service worker activates.

Could you share the full refresh logic that you're using on the client page?

If you do manage to reproduce at some point, could you grab a screenshot of the Service Workers tab of the Applications panel in Chrome Dev Tools? I'd like to confirm whether there's a service worker stuck in waiting state.

Additionally, workbox-precaching creates two caches: a cache with -temp in its name, which is populated as part of the install handler, and a cache without -temp, which is populated during the activate handler. It would be good to see screenshots of the contents of both of those caches.

[AlexandreBonaventure]

Hi Jeff, thanks for your answer.

So, yes we are using skipWaiting, in fact the registering script is a fork from the workbox advanced recipes (the client prompt) except for our use-case we don't need any confirmation for the user, so we are reload as soon as the sw activates. Here's a gist : https://gist.github.com/AlexandreBonaventure/d765ea79b522c8bc3e49f0e060adb381

As for the service worker tab, I remember well that it was showing only one sw, the latest up-to-date version, no other service worker waiting whatsoever.
I checked as well the -temp cache and it was empty.

Looking forward to sort this out. Thanks for your time and your work

[AlexandreBonaventure]

I forgot to mention: this snippet is in the addons.js imported script

self.addEventListener('message', (event) => {
  if (!event.data) {
    return;
  }

  switch (event.data) {
    case 'skipWaiting':
      self.skipWaiting();
      break;
    default:
      // NOOP
      break;
  }
});

It's how we can trigger skipWaiting from the registering script.

[andreaschrist27]

i have same issues

[sebastienroul]

@AlexandreBonaventure ,

Same problem here - Deploying a new version is a nightmare for the support's team... :)
Did you find a workaround or solution ?

[AlexandreBonaventure]

@sebastienroul actually yes !
In fact, I was mostly using service-workers with PWA, thus making sure to configure server configuration as you can read here: https://github.com/vuejs-templates/pwa/blob/development/docs/prevent_caching.md#how-to-add-caching-headers-for-your-server

We are running a nginx server so I basically copy paste the snippet. However, this was not suiting our case because this snippet make any 404 fallback to index. This can be catastrophic, let's imagine your server fails to serve main.css loaded by your app, instead of returning a real 404 (remember if any file return with a 404, the service worker installation will just stop), the server will instead serve index.html. Your service worker is like: hum great we have a 200 let's store it locally. Next time the user refresh, your service worker is taking control, serving the css with html content. Consequence: syntax error!

In order to fix we had to tweak the config to disable the fallback for actual files (aka the dot rule) https://stackoverflow.com/questions/13812550/configuring-nginx-for-single-page-website-with-html5-push-state-urls/32137788#32137788

If you are using express I suggest you take a look at https://github.com/bripkens/connect-history-api-fallback#disabledotrule

We don't have any problem anymore since we tweak our server config

[apoorv007]

@AlexandreBonaventure ,

How did this fix the issue of a stale index.html that you mentioned in your first post?

[zheeeng]

Have this issue that staled index.html guides browser fetching inexisted assets and get vast 404 errors

[Tirumaleshwar]

index.html returned by service worker. But it is cacheed one. Service worker also old and not updating to new one Even after clear data of the app.

…

On Fri, Jul 19, 2019, 1:15 AM Zheeeng ***@***.***> wrote: Have this issue that staled index.html guides browser fetching inexisted assets and get vast 404 errors — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1528?email_source=notifications&email_token=AHMMP537FYSGWEJRPZGBACLQADB6HA5CNFSM4FEZEHCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2JSL3A#issuecomment-512959980>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHMMP57IRY4ZE3FYSXVODDLQADB6HANCNFSM4FEZEHCA> .

[sebastienroul]

@Tirumaleshwar and all, as I mentioned 6 month ago, we had the same problem... I can assure we experiment many solutions, but none was perfect - and clients complained again and again...
So I decided to fix definitively the problem with a stupid "brute force" solution, and since 3 month, we have NO problem.
The solution is very simple : the missing resource, usualy a JS file, normaly do something (create a global variable, modify the DOM or whatever) : in our case, the JS file add a div with an ID "#q-app" : Anyway : WHATYOUWANT created.

In the index.html I had a stupid interval loop for checking if the #q-app was there or not, and, if after 15sec it's not there, then :

• clear all serviceWorker,
• clean all the cache
• reload

I know it's not very elegant, but since 3 month NO MORE call from client about the "blank page"

Hope this workaround will helps as many people as possible, cause after so many nights spent to upgrade everything I was tired :)

[Tirumaleshwar]

@RouL In my case I am getting Js file but that is cached one so above solution doesn't seems to be work.

…

On Fri, Jul 19, 2019, 3:34 AM ROUL ***@***.***> wrote: @Tirumaleshwar <https://github.com/Tirumaleshwar> and all, as I mentioned 6 month ago, we had the same problem... I can assure we experiment many solutions, but none where perfect - and clients complained again and again... So I decided to fix definitively the problem with a stupid "brute force" solution, and since 3 month, we have NO problem. The solution is very simple : the missing resource, usualy a JS file, normaly do something (create a global variable, modify the DOM or whatever) : in our case, the JS file add a div with an ID "#q-app" : Anyway : WHATYOUWANT created. In the index.html I had a stupid interval loop for checking if the #q-app was there or not, and, if after 15sec it's not there, then : - clear all serviceWorker <https://developer.mozilla.org/en-US/docs/Web/API/ServiceWorkerRegistration/unregister> , - clean all the cache <https://developer.mozilla.org/fr/docs/Web/API/Cache/delete> - reload <https://developer.mozilla.org/fr/docs/Web/API/Location/reload> I know it's not very elegant, but since 3 month NO MORE call from client about the "blank page" Hope this workaround will helps as many people as possible, cause after so many nights spent to upgrade everything I was tired :) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1528?email_source=notifications&email_token=AHMMP5ZDFOPGSGPWZ5HCM33QADSFHA5CNFSM4FEZEHCKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2J5VPQ#issuecomment-513006270>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHMMP55QVJHACFHXTJN2W3LQADSFHANCNFSM4FEZEHCA> .

[sebastienroul]

@Tirumaleshwar, yes for sure, in our case, all resources loaded by index.html are packaged at the build time and them all have a unique name : by example script.23424323.js and at next build script.5675756765.js, so old index.html failed to find the old JS... and them above works.
For sure if the resource you load is already in cache, above doesn't work 💯 And them removing the worker/date/cache doesn't make sense at all... :)

[zheeeng]

Maybe we need a way to force revision index.html file

[dkhizhniakov]

I've run into the same issue. I've tried several different ways to fix this, but neither of them seems to work. The workaround from @sebastienroul seems to be the best solution so far for our project, but there definitely should be better solution.

I would appreciate any help with this.

[myers]

Here is how I put @sebastienroul advice into code. It has protection against endless reloads too.

window.serviceWorkerRegistration = null;

async function startPWA() {
  try {
    window.serviceWorkerRegistration = await navigator.serviceWorker.register("/service-worker.js");
  } catch(registrationError) {
    console.log("SW registration failed: ", registrationError);
  }
  console.log("SW registered: ", window.serviceWorkerRegistration);

  try {
    // call what ever starts your PWA here
    // to debug you might do: throw("some error");
    window.localStorage.removeItem("usedBigHammer");
  } catch(ee) {
    await fixWithBigHammer();
  }
}

async function fixWithBigHammer() {
  if (window.localStorage.getItem("usedBigHammer")) {
    window.alert("Problem loading app.  Please contact support.");
    window.localStorage.removeItem("usedBigHammer");
    return;
  }
  if (window.serviceWorkerRegistration) {
    const unregistered = await serviceWorkerRegistration.unregister();
    console.log("SW unregistered");
  }

  const cacheKeys = await window.caches.keys();
  for (key of cacheKeys) {
    console.log("Working on cache: ", key);
    const cache = await caches.open(key);
    const requestKeys = await cache.keys();
    for (request of requestKeys) {
      console.log("Deleting request: ", request);
      await cache.delete(request);
    }
  }
  console.log("Cache deleted");
  window.localStorage.setItem("usedBigHammer", true);
  console.log("Reloading");
  window.location.reload();
  return true;
}

window.addEventListener("load", startPWA);

[anhnt-mageplaza]

The main problem here is that the index.html is cached. The solution is to tell the workbox not to cache the index.html and it should have the no-cache header too. Here is our snippet

new workboxPlugin.GenerateSW({
     swDest: 'sw.js',
     clientsClaim: true,
     skipWaiting: true,
     ignoreURLParametersMatching: [/.*/],
     dontCacheBustURLsMatching: new RegExp(".+.[a-f0-9]{20}..+|index.html"),
     cleanupOutdatedCaches: true,
     exclude: [new RegExp("index.html")]
   }),

[sebastienroul]

@anhnt-mageplaza Do you mean no cache at all with this dontCacheBustURLsMatching ? Cause if so, how do you manage avaibility of index.html when user is offline ?

[data-handler]

@sebastienroul did you manage to answer your own question? Is index.html available offline even when excluding it from being cached? Seems counter-intuitive..

[waldemarennsaed]

The issue for me: Also cached the index.html. The index-file references a couple of build-artifacts, like {hash}.js or {hash}.css. Theses hashes are defined by our build-tool (using vite) and the hash changes on each new build.

Once the service-worker is registered and active, it is using the precaching feature to precache all static assets - unfortunately, also the index.html file.

While renaming the entry-file to e.g. main.html is possible, it would just break more than repair (e.g. our client-side router can't resolve paths like /my-path anymore but instead would need to resolve /main.html/my-path).

Also, we can't ship a service-worker update, since the index-file blocks the loading and registration of the new service-worker file.

The only two solution that I can think of, is redirecting the old {hash}.js (which is our breaking file) to the new {new_hash}.js file or implementing a single-purpose redirect.html file, which takes care of:

• clear the SW cache and register the new SW
• redirect to index.html afterwards

The second option would need an additional configuration on the webserver (e.g. using nginx, you would need to resolve an initial load of my-app.com/ to redirect.html.