Allow overlapping scopes

[AshleyScirra]

From my understanding, #566 relates to allowing multiple/dynamic scopes for a single SW, and #921 relates to allowing multiple SW at the same scope. However I think SW should allow overlapping scopes as a separate case.

What I mean by overlapping scopes is one site could register two SWs at different but nested scopes like this:

https://example.com/root-sw.js
https://example.com/path/path-sw.js

Note path-sw.js is within the scope of root-sw.js. I can't find much information on what is actually specified as happening in this situation, but I did see one comment saying "it's implementation defined, don't do it". I also tried it on Chrome and totally couldn't figure out what it was actually doing, but it looked like it picked one SW and just permanently used that. However I think there are valid use-cases around this.

In the above example I would like root-sw.js gets all fetch events on the origin, except for anything under /path/, which then goes to path-sw.js. For example some mappings would be:

https://example.com/index.html -> root-sw.js
https://example.com/otherpath/index.html -> root-sw.js
https://example.com/path/index.html -> path-sw.js
https://example.com/path/anotherpath/index.html -> path-sw.js

Use cases

One potential disadvantage of web apps over traditional installed desktop apps is if an update breaks or changes something, users cannot easily roll back to an older version. This can be mitigated by making old versions of a web app available as well.

To me, the obvious way to do that would be something like this:

https://myapp.com/ is the latest version and auto-updates
https://myapp.com/v1/ would be specifically version 1
https://myapp.com/v2/ would be specifically version 2, etc.

Then users just use myapp.com and if anything goes wrong they can roll back to myapp.com/v1, for example.

However adding SW-based offline support in this scenario requires that there be SWs at the following scopes:

https://myapp.com/
https://myapp.com/v1/
https://myapp.com/v2/

But note all the versioned URLs fall under the scope of the origin-level SW. Since as far as I can tell SW does not support this case, it forces the URL scheme to be changed. The next best workaround seems to be to use another path for the latest version, e.g.

https://myapp.com/latest/

and now none of the SW scopes overlap. However it seems awkward that we have to change the URL structure of our app solely because of this aspect of SW. Also it's not such a nice URL - it is easier for customers to remember simply myapp.com, whereas this way they may have to remember "myapp.com slash something". (Obviously myapp.com can redirect to myapp.com/latest, but the user doesn't necessarily know that will work, and will permanently see myapp.com/latest in their URL bar.)

Another use case might be using paths for other services, e.g.:

myapp.com/ serves a web page entry point with a SW that handles notifications and such
myapp.com/static/ serves static content with a long-term caching SW
myapp.com/api/ serves API content with a short-term caching SW

In this case again the URL structure has to be changed. Alternatively in both cases you could write one master SW that handles all versions or all cases, but that significantly increases the complexity of the SW script. It's nice to have little modular SWs that handle specific paths in specialised ways.

Spec changes

This seems relatively straightforward to specify: fetch events go to the SW with the "nearest" scope. I'd probably better define that as the SW with the longest scope that the fetch event still falls under.

This doesn't need any new API surface and still enables these use cases. It just better defines what currently appears to be murky/undocumented/undefined behavior.

I suspect there may need to be special provisions for updating SWs themselves. For example in the original example, it is unlikely the web developer wants the fetch for path-sw.js when registering that service worker to be handled by a previously registered root-sw.js (this seems likely to just cause headaches with stale cached versions). I can't think of any use case for wanting one SW to handle the fetch event for another SW, so it seems reasonable to just say they are an exception.

[delapuente]

It is defined in the Match Service Worker Registration algorithm and I think it behaves as you say:

5. Set matchingScope to the longest value in allScopes which the value of clientURLString starts with, if it exists.

[AshleyScirra]

Ah, is this already defined then? It seems to be undocumented everywhere else I've looked.

[jakearchibald]

In the above example I would like root-sw.js gets all fetch events on the origin, except for anything under /path/, which then goes to path-sw.js

At the moment, the scope of a SW decides which clients it'll control. Then, fetches from controlled clients triggers fetch events in the controlling service worker. This is how you can get fetch events for urls that are on other origins, despite those urls being clearly out of scope.

But, in terms of deciding which service worker gets control, the longest matching scope wins, as @delapuente says.

Does this cover your use cases? Do you want an img fetch for /foo/cat.jpg from page / to go via the service worker controlling the page, or the SW scoped to /foo/? If it's the latter, that would need to be a new thing, like making foreign fetch work in the same origin or something.

[AshleyScirra]

We'd like a request to /foo/cat.jpg from a page open at / to be served by the SW at /foo/, and anything else served by the SW at /. I thought this is already how it works as @delapuente described - it already goes to the longest scope it can, right?

Our use case is actually live now, at editor.construct.net. The pattern we've settled on after a few false starts is:

editor.construct.net/ - serves latest version
editor.construct.net/r1/ - first version
editor.construct.net/r2/ - second version, etc.

The / path serves an index.html with a base href pointing at the latest version, e.g. r2/. There is a SW at r2/sw.js which caches version 2 for offline use.

There's just the problem of serving the file at editor.construct.net/index.html while offline - so we put another SW at editor.construct.net/sw.js designed to cache only index.html (and a couple of other files we happen to need at that level). So then:
/sw.js servers root-level files
/r2/sw.js serves that version of the web app

As I say it's live so these URLs should all work and you can check it out. As far as I can tell it works, so I think our use case is covered already... unless it only works by accident?

[AshleyScirra]

Huh, actually it looks like with this setup, the /r2/ SW caches the whole app but in Chrome that SW never handles any fetch events - they all get handled by the one at /. So in our case the / SW is magically able to serve the entire app because it happens to also be in the cache, but then the one at /r2/ isn't handling fetch events, so we lose the ability to do extra features like lazy-caching responses.

Is this to spec? According to what @delapuente pointed out, shouldn't the fetch be handled by the SW with longest scope, i.e. at /r2/?

[jakearchibald]

It works as I described in #1085 (comment) & it's what the spec says.

Here's a demo https://cdn.rawgit.com/jakearchibald/aa15821ef593aa871b241898f926a492/raw/0fe98d28a5c2bddbabc5aafd9260ca0274e82cc6/ (see the console), and here's the code https://gist.github.com/jakearchibald/aa15821ef593aa871b241898f926a492.

• When "./" requests "./foo.html", it triggers a fetch event in navigator.serviceWorker.controller, which is "./sw.js", which has a scope of "./".
• When "./foo.html" requests "./", it triggers a fetch event in navigator.serviceWorker.controller, which is "./sw-foo.js", which has a scope of "./foo".

Longest matching scope wins control of the page. Requests go through the service worker which controls the page - navigator.serviceWorker.controller.

[AshleyScirra]

Okay, but I think our use case is slightly different. We have SWs at:

/
/r2/

The user navigates their browser to /. This page then makes a request for /r2/cat.jpg.

What we want to happen: SW at /r2/ handles fetch, based on the fetch URL
What seems to actually happen: SW at / handles fetch, based on document URL

Is this correct? If so what is the rationale for the SW based on the document rather than the fetch URL to handle the request? It seems to break this versioning use case.

[jakearchibald]

If the SW at the fetch URL was used, you wouldn't be able to cache anything on another origin, which you want to do in many cases.

[AshleyScirra]

Could there be an opt-in to this kind of behavior?

It looks like at the moment we have to write our SW to be a catch-all that handles everything, including caching multiple different versions of the app separately, as well as caching some of the root-level files separately. What we really want is a series of small, independent SWs that are only responsible for one thing, which makes it more modular and easier to write. Currently it seems we have to change the URL structure of our app to get what we want, and oops, we only just figured this out after deploying to production. (Service Worker strikes again!)

Other approaches have their own drawbacks too - for example we tried having /latest/ redirect to /r2/ (but we don't want users to bookmark specific versions), tried having /latest/ URL rewrite all its contents to the latest version (but then if you load /r2/ directly, it re-downloads an identical version of the entire app again a second time, instead of knowing it can serve from /latest/)... I'm not sure how to structure this without pitfalls. Changing the base href at / works nicely, except SW scope rules interfere.

[jakearchibald]

we only just figured this out after deploying to production. (Service Worker strikes again!)

FWIW this is similar to how appcache works. The appcache that controls the page serves the content. And I think preventing caching of any cross-origin resources would have been a huge step backwards.

I'll have a think about ways we could support this, but aside from foreign fetch (which is exclusively cross origin) this is the first time I've seen a feature request like this.

[jakearchibald]

One way could be to figure out if postMessage could be made to work here. That would involve making Request/Response transferrable. This might not be as tough as it sounds, since there's an effort underway to make streams transferrable.

[AshleyScirra]

I think preventing caching of any cross-origin resources would have been a huge step backwards.

I'd be fine with making this same-origin only, but I guess that only solves our use case.

this is the first time I've seen a feature request like this.

What's the best practice for SW caching multiple versioned URLs? Or is this not something anybody does with SW? If so what is done instead? I'd be interested to learn what others do, since we've found versioning deployments quite tricky so far. I think our key requirement is that old versions are still available at their own URL. Ideally we would then also be able to update a version in-place, such as to back-patch fixes. We've ported from a traditional downloadable desktop app where we have an archive of old releases, and this is something we didn't want to regress on when moving to the web.

[AshleyScirra]

Just thinking about the versioning scheme a bit more, I suppose the URL rewrite with a second download if you visit the other URL is probably the least bad option.

[jakearchibald]

Closing this, as I think it's been resolved through discussion. Please reopen if I'm wrong!

[AshleyScirra]

What discussion was there and what was the conclusion?

[jakearchibald]

Oh sorry, I meant discussion in this thread. I thought this was down to a misunderstanding of how appcache worked.

[AshleyScirra]

Actually, this is still important to us, and I'd still like to see it happen. I still run in to nightmarish bugs with overlapping SW scopes (https://bugs.chromium.org/p/chromium/issues/detail?id=789220, closed wontfix, but still no idea what's really going on). It'd be great if it could be better supported.

Perhaps the simplest solution is to provide an explicit way for a service worker to hand over a fetch to another SW with a longer scope. E.g. suppose the following two SWs are registered:

example.com/sw1.js
example.com/v2/sw2.js

then sw1.js could do something like this in its fetch handler:

self.addEventListener("fetch", event =>
{
    // Dispatch to sw2.js if under v2/ path
    if (event.request.url.startsWith("v2/"))
        return getV2ServiceWorker().handleFetch(event);
   
    // ... (optionally test other paths) ...

    // ... handle fetch using this SW's logic ...
});

This provides a simple way to dispatch fetches to other SWs, which helps split a large god-SW in to smaller SWs with limited responsibility. It doesn't sound like it needs to change the existing ways SWs control clients or fetches, it's just a way to use the lowest-level SW as a dispatcher to other SWs.

Can we reopen this issue or should I file a new issue?