How do we install import maps in worker/worklet contexts?

[domenic]

It's unclear how to apply import maps to a worker. There are essentially three categories of approach I can think of:

• The worker-creator specifies the import map, e.g. with new Worker(url, { type: "module", importMap: ... })
• The worker itself specifies the import map, e.g. with self.setImportMap(...) or import "map.json" assert { type: "importmap" }.
• The HTTP headers on the worker script specify the import map, e.g. Import-Map: file.json or maybe even Import-Map: { ... a bunch of JSON inlined into the header ... }.

The worker-creator specified import map is a bit strange:

• It's unlike the Window-context mechanism in that the creator of the realm controls the realm's module resolution, instead of the realm itself
• It's like the Window-context mechanism in that the script that runs in the realm does not control the realm's module resolution; the control is external to the script itself.

Basically, there is no clear translation of <script type="importmap"> into a worker setting.

Also, as pointed out below, anything where the creator controls the import map works poorly for service worker updates.

The worker itself specifying seems basically unworkable, for reasons discussed below.

And the header-based mechanism is hard to develop against and deploy.

Original post, for posterity, including references to the old "package name map" name

We have a sketch of an idea for how to supply a package name map to a worker:

new Worker(someURL, { type: "module", packageMap: ... });

This is interesting to contrast with the mechanism for window contexts proposed currently (and discussed in #1):

• It's unlike the Window-context mechanism in that the creator of the realm controls the realm's module resolution, instead of the realm itself
• It's like the Window-context mechanism in that the script that runs in the realm does not control the realm's module resolution; the control is external to the script itself.

Basically, there is no clear translation of <script type="packagemap"> into a worker setting.

If we went with this, presumably we'd do the same for SharedWorker. Service workers would probably use an option to navigator.serviceWorker.register(), and have an impact similar to the other options?

For worklets, I guess you'd do something like CSS.paintWorklet.setModuleMap({ ... }). Only callable once, of course.

In all cases, it would be nice to make it easy to inherit a package name map. With the current tentative proposal you could do

new Worker(url, {
  type: "module",
  packageMap: JSON.parse(document.querySelector('script[type="packagemap"]').textContent)
});

but this is fairly verbose and a bit wasteful (since the browser has already done all the parsing and processing of the map, and you're making it do that over again). At the same time, inheriting by default seems conceptually weird, since workers are a separate realm.

[matthewp]

Could the packagemap script have a property on it, packageMap or something so you could use: document.querySelector('#mymap').packageMap to get it? Not sure if there is precedent for properties on elements only when it's a certain type like this.

[justinfagnani]

@matthewp I'd certainly like to see a .module property for <script type=module> that's a Promise resolving to the module object. If this ends up being a general feature, maybe we just need a .content or .value property. edit: those are bad names, because they don't imply them being the post-parse/instantiate object

[domenic]

Yeah, that seems pretty reasonable. I think I like the idea of a general property, with some name or another.

[guybedford]

Could we by default just have workers inherit the package map from the parent window?

That seems to cover the 99% scenario.

Isolation doesn't seem to align with the worker boundary, as much as it does concepts of Realms.

I understand the issues for worklets and service workers but I'm' not sure they should be dominating use cases here.

[domenic]

Hmm, I don't understand. Workers don't share the parent realm so if you're saying isolation is aligned with realms then it is also aligned with the worker boundary.

[guybedford]

I was referring to the proposal to create isolated realms, but it's a somewhat misleading tangent.

The question really is what is the use case for having a different package map for the worker, that couldn't be shared with the page?

[guybedford]

Note if we first shipped with the default of having the worker share the package map, then a future proposal could always provide package map isolation. The isolation picture and use cases will be clearer then than they are now to spec something good as necessary, and realms are possibly part of that picture too.

[domenic]

In the end I am very uncomfortable with sharing module-related features cross-realm. In general cross-realm sharing of state of this sort is largely unprecedented, and I'd be especially unhappy about doing it for modules, which right now are purely tied to realms.

I am pretty certain that v0 will not have such a strange new feature as cross-realm sharing, even if we eventually want to introduce an easy way to do so.

[guybedford]

@domenic then we must guarantee that when it comes to implementations of the Realms proposal, that they do not share the package map with the main page by default.

[guybedford]

Also, note that what I'm suggesting here is a default behaviour - there is nothing to say this cannot be overridden.

As someone who has worked on build tools in JS for a while, assuming all workers are instantiated with a packagemap argument will be a tooling nightmare - the third-party worker story is really being neglected from all directions currently which is incredibly worrying already.

[guybedford]

I see this is already catered to in the Realms spec (although interesting the Realms spec then effectively hinges on a full blown module API for modules support - tc39/proposal-shadowrealm#103).

Passing the package map can work - we'd likely have build workflows that inject it into the new Worker instantiations, while npm packages leave it out. It's just annoying to have non-portable ties on this boundary effectively requiring a build step for third-party libraries with worker instantiations.

[domenic]

Yes, in general the realms proposal has a large variety of blocking issues on how it interacts with how browsers work with realms.

I understand that breaking the current model for how realms work can be convenient, especially for tools or similar. I don't think that means we should do so, especially by default.

[guybedford]

Then make it a boolean option - packageMap: 'parent' and at least provide a portable-non build workflow for the web that can deal with workers. It's the complete lack of concern for portable worker workflows that gets me here. Us build tool authors then have to fix the problems you create here.

[domenic]

That's an idea. I'd prefer we have a more holistic story for how workers relate to other realms, instead of creating a one-off feature in package name maps. For example it's not clear why a package name map would be easily inherited but not a module map, or URL, or content security policy, or...

[domenic]

Related: w3ctag/design-principles#111

[annevk]

Note that any kind of inheritance would also be "racy" for shared and service workers and therefore not an acceptable solution for them.

This feature doesn't seem needed for worklets as they can't fetch anything.

[domenic]

My understanding is that worklets can use static import statements. (And per spec dynamic ones, but IIUC the sole worklet implementer so far removed support for that JS language feature, but hasn't updated the spec.)

[annevk]

I see, I doubt the fetching model for that is well-tested. All those module fetches would come with Origin: null per how things are written today which I doubt is how it's implemented...

[littledan]

If worklets should be able to access built-in modules, we may want to work through how import maps could work, so that these modules can be polyfilled/virtualized.

[Jamesernator]

For example it's not clear why a package name map would be easily inherited but not a module map, or URL, or content security policy

As per CSP3 dedicated workers and worklets now inherit their page's CSP so this indicates some precedence.

Dedicated workers now always inherit their creator’s policy.

[rektide]

As a web developers, it's trying experiencing inconsistent old outdated capabilities in workers. Article after article highlights the importance of this feature set for keeping a responsive web app & doing computation, but support for Workers has been awful & this ticket is another intimidating document implying that working with workers is going to remain really hard for the forseeable future, & not receiving the benefit of the es2015 modules progress that the happier paths have gotten.

We still don't have module 1.0 support in workers on Chrome. Now, I'm finding that when we do get support, it's going to be for modules that won't be able to use import-maps? It's frustrating that,

• modules shipped a 1.0 that wasn't modular (cannot reuse scripts across sites because of hardcoded specifiers),
• that we still don't have platform support for them (in workers),
• that import-maps was required to make them modular,
• and now 18 months into import-maps discussion, there's no indicator how import-maps may someday be able to help workers get useful modules.

Modules feels like it was shipped prematurely, and I'd like to avoid going 1.0 with import-maps in a similar fashion, before there's a plan for how it can be used in practice across the platform.

[guybedford]

Thanks for the call to action @rektide.

FWIW es-module-shims supports import maps in web workers in Chrome today using importMap as an input, which seems very sensible to me. See https://github.com/guybedford/es-module-shims/#module-workers for more info.

In terms of spec work, I think it's just the constraints of prioritization with a serial process with limited people working on this. And I'm sure a spec PR for a worker approach wouldn't be ignored.

[domenic]

I have to say, it definitely discourages me from working on module stuff, if making incremental progress gets that kind of negative reception. If the demand is that every piece of the puzzle be perfect and full before any specs are accepted or any browsers ship modules, then I don't want to work on modules.

Fortunately I think we have a lot of web developers who gain value from what we have today, and appreciate it, instead of saying that it's "premature" to ship anything but a big-bang modules + dynamic import() + import.meta + modulepreload + workers + import maps + import maps in workers + more all at once. So I plan to ignore such sentiments.

[jakearchibald]

With service workers, the import map should be updatable between versions of the service worker. There isn't a JS call directly linked to service worker updates, so something like a header, or something inline would work better.

[masx200]

navigator.serviceWorker.register('/latest-worker.js', {  
type: 'module',

importMap: import.meta.importMap }); 


new Worker('/worker.js', { type: 'module',

 importMap: import.meta.importMap}); 

navigator.serviceWorker.register('/latest-worker.js', {  
type: 'module',

importMap: '/latest-importmap.json' }); 


new Worker('/worker.js', { type: 'module',

 importMap: '/importmap.json' }); 

navigator.serviceWorker.register('/latest-worker.js', {  
type: 'module',

importMap: {
  "imports": { ... },
  "scopes": { ... }
} }); 


new Worker('/worker.js', { type: 'module',

 importMap: {
  "imports": { ... },
  "scopes": { ... }
} }); 

[jakearchibald]

@guybedford

One approach to mitigating this concern might be to allow the service worker update() call to take an options parameter that can override the previously provided registration options for the service worker. Thus a service worker might upgrade from a script to a module, just as it might upgrade its import map.

The problem is, update() isn't the only way a service worker script updates. It's more of a forced update. Most service worker script updates do not happen via the update() method.

Since service worker updates are not initiated by script, I don't think we're looking for a script solution. The best I can think of is an HTTP header containing the map.

That doesn't mean dedicated/shared workers need to use this method too, as the constructor param seems like a good solution there. Although, it would be nice if workers and HTML documents could use the HTTP header too. Just need to decide what happens if both methods are used.

[lewisl9029]

Although, it would be nice if workers and HTML documents could use the HTTP header too.

FWIW, I wanted to note that this isn't just a nice to have for HTML documents either, since in-document import maps are fundamentally incompatible with modules loaded from modulepreload headers (see discussion here for details, TL;DR modulepreload'ed modules need to resolve import specifiers, which requires locking the import map from further modifications, and often occurs before in-document import maps are even seen by the browser).

Right now we're forced to choose between using modulepreload headers and import maps, which is obviously not ideal for long term adoption. It sounds like specifying import maps through a header could support both the modulepreload and the service worker updates use case, so I'd love to see this effort gain more traction!

[jespertheend]

@jakearchibald What makes this case different from changing "classic" to "module" or updating the scope of a service worker?
Doesn't that run into the same issues as with import maps?

[guybedford]

@jakearchibald thanks for clarifying that makes sense. Do you feel this service worker update concern is necessary to work through first before progress can be made on setting an { importMap } option explicitly for workers? Or do you think it would be ok to separately tackle these in-band and out-of-band import map configuration feature progressions? If you do think we should fully work through the discussion here, is there somewhere we can discuss further on this topic to avoid cluttering this thread?

Specifically I really would like to see progress on new Worker('mod.js', { type: 'module', importMap }) and would even be interested in working on spec text. But this work is currently blocked by this concern.

[guybedford]

Thinking about the out-of-band configuration case further, another option might be to define an importMapSrc attribute for the worker instantiation / service worker registration:

navigator.serviceWorker.register('/latest-worker.js', {
  importMapSrc: '/latest-worker-map.json'
});

The import map can then be updated along with the JS just fine, and this fully mirrors the <script type=importmap>...</script> and <script type=importmap src="...'> in-band versus out of band mechanisms respectively.

For this reason specifying both importMap and importMapSrc as being possible would still be useful.

The benefit of this over the header might be that the source and import map can be fetched in parallel for the service worker and for service worker updates, rather than as separate requests. Since import maps directly inform further loading, it seems a fairly useful property to have as well.

[jakearchibald]

@jespertheend

@jakearchibald What makes this case different from changing "classic" to "module" or updating the scope of a service worker?

Those are initiated via script whereas most service worker updates are not.

[jakearchibald]

@guybedford

Do you feel this service worker update concern is necessary to work through first before progress can be made on setting an { importMap } option explicitly for workers? Or do you think it would be ok to separately tackle these in-band and out-of-band import map configuration feature progressions?

I think that's ok.

Thinking about the out-of-band configuration case further, another option might be to define an importMapSrc attribute for the worker instantiation / service worker registration:

navigator.serviceWorker.register('/latest-worker.js', {
  importMapSrc: '/latest-worker-map.json'
});

Yeah, that's kinda neat. However, I worry that it complicates the security story. Right now, if someone gets to run script on your origin, they can extend the life of that attack by installing a service worker. However, that script needs to be same-origin, and in the same path as the scope (unless a special header is used). Service worker script requests also have special headers, so servers can pick up on unexpected service worker requests.

importMapSrc introduces a system where an attacker could 'infect' an existing service worker. It would need to be hardened security-wise, and I worry that would become pretty complicated.

The header solution doesn't have these issues, and it seems like it would also plug other feature gaps? #2 (comment)

[domenic]

Yeah, I'm coming around more to the header-based solution. Especially since it looks like we need something similar for speculation rules, based on some recent web developer feedback.

[guybedford]

However, I worry that it complicates the security story

That's certainly understandable, thanks for pointing this out. Separately import maps with a "src" attribute will likely have integrity support - it would also be good to think about the same thing in this scenario as well.

Do you feel this service worker update concern is necessary to work through first before progress can be made on setting an { importMap } option explicitly for workers?

I think that's ok.

Thanks @jakearchibald I'm fine with a header solution for service workers for an out-of-band mechanism, I just think we also need an in-band mechanism in addition, and I just hope this service worker use case doesn't preclude progress on an in-band approach.

[littledan]

I agree with @guybedford that it would be good to have some approach which can work in-band; it might be hard to configure the header approach for worker import maps, and these are essential for functioning, whereas detailed prefetch/speculation hints work fine if omitted. If we go with a header-based approach, it would be great if we had some clear ideas about the deployment workflows beforehand (web bundles may help!). I agree with Guy's comment that the ideal approach would be to inherit the import map by default.

[jakearchibald]

I just think we also need an in-band mechanism in addition

Agreed, particularly as blob URLs don't have headers. Also agree that inheriting from the parent should be easy.

and I just hope this service worker use case doesn't preclude progress on an in-band approach.

I don't see why it would.

[fuweichin]

According to the import-map-processing section:

Because they affect all imports, any import maps must be present and successfully fetched before any module resolution is done. This means that module graph fetching is blocked on import map fetching.

This means that the inline form of import maps is strongly recommended for best performance.

and given the fact that

External import maps are not yet supported.

Multiple import maps are not yet supported. https://crbug.com/927119

I prefer the first approach, and my idea is to add an option "importMap" to WorkerOptions, as defined below

dictionary WorkerOptions {
  WorkerType type = "classic";
  RequestCredentials credentials = "same-origin"; // credentials is only used if type is "module"
  DOMString name = "";
  ImportMapOptions importMap = null; // importMap is only used if type is "module"
};

dictionary ImportMapOptions {
  USVString src = ""; // a http/https url, a data url, or a blob url
  Blob srcObject = null; // option `importMap.src` takes precedence over `importMap.srcObject` if both specified
};

See also IDL of the Worker interface

Notes:

• Option importMap.src may not be supported until feature "external import maps" gets supported
• Use of importMap.srcObject is strongly recommended for best performance

Examples:

You can dynamically generate import maps,

let importMapObject = {
  imports: {
    a: './a.js'
  }
};

let worker = new Worker('path/to/worker.js', {
  type: 'module',
  importMap: {
    srcObject: new Blob([JSON.stringify(importMapObject)], {type: 'application/importmap+json'}),
  },
});

load an importmap resource manually,

let importMapObject = await fetch('path/to/importmap.json').then((res) => {
  if (!res.ok) {
    throw new Error('Error loading importmap, received status: ' + res.status);
  }
  let type = res.headers.get('content-type');
  if (!/^application\/(importmap\+)?json/.test(type)) {
    console.warn('Resource interpreted as importmap but transferred with MIME type ' + type);
  }
  return res.blob();
});

let worker = new Worker('path/to/worker.js', {
  type: 'module',
  importMap: {
    srcObject: importMapObject,
  },
});

or use embed data url as importmap source until someday <script type="importmap" src="xxx"> is supported

let worker = new Worker('path/to/worker.js', {
  type: 'module',
  importMap: {
    src: "data:application/importmap+json;base64,...",
  },
});

About multiple import maps

If you do have multiple import maps to be used, then you may load multiple import maps from multiple sources, parse them into plain JSON objects, and merge them as single Blob, just do it by your self.

[jakearchibald]

I don't think it needs to be that complicated. Just:

const worker = new Worker(src, {
  type: 'module',
  importMap: {…}
});

Then, if folks want to fetch the import map from somewhere external, they can use fetch(). If they want to inherit the import map, they can get it from something like import.meta.importMap.

For shared/service workers, where the worker creation may have been triggered by another environment, the import map is specified in a header.

[bartlomieju]

I'm a bit late to the discussion and I already see there are a few folks from the Deno community here suggesting solutions and I wanted to give my 2 cents. The Deno team certainly recognizes that this is a requested feature that could help to solve many common workloads. I also recognize that our restrictions are in some places wildly different than in browsers - we don't need to consider import maps loaded via <script> tag.

I don't think it needs to be that complicated. Just:

const worker = new Worker(src, {
  type: 'module',
  importMap: {…}
});

Then, if folks want to fetch the import map from somewhere external, they can use fetch(). If they want to inherit the import map, they can get it from something like import.meta.importMap.

For shared/service workers, where the worker creation may have been triggered by another environment, the import map is specified in a header.

I agree with Jake here - but would really love to see a complementary option to specify a URL. In Deno's case a call to fetch() API requires a permission granted on the CLI via --allow-net flag; if we had a statically analyzable solution (with importMapUrl) then the permission check wouldn't be necessary; user aknowledgles that it's a part of their program and these sources are to be trusted.

That said, I think it's more important to ship a first iteration of the support to get further discussion going and the Deno team is willing to dedicate resources to ship the first implementation on the agreed proposal in a one-month timeframe.

[AXT-AyaKoto]

(Does this discussion allow me to join? Sorry if I overlooked some rules.)
What is the current status of this issue?
I can't write it well, but I want this.

I want it to be as easy as possible.

I don't think it needs to be that complicated. Just:

const worker = new Worker(src, {
  type: 'module',
  importMap: {…}
});

I agree with Jake here - but would really love to see a complementary option to specify a URL. In Deno's case a call to fetch() API requires a permission granted on the CLI via --allow-net flag; if we had a statically analyzable solution (with importMapUrl) then the permission check wouldn't be necessary; user aknowledgles that it's a part of their program and these sources are to be trusted.

It seems to me that you can just set a rule like "We can use either one, but importMap takes precedence."
(Or is it possible to change the interpretation depending on the data type of importmap?)

[FoxLisk]

This is a huge missing feature. I just went through a bunch of trouble to get all my import map stuff working properly to work around some oddities in how some libraries reference dependencies in their import statements, and then when moving to web workers it just doesn't work at all and I need a whole different solution and in fact probably just can't use modules at all.

[khrome]

I'm working on a module which uses a pluggable terrain loader in a worker as well as buildless ESM. I worked around the existing behavior using local links, but quickly realized, when importing it into app space, that I wouldn't be able to import any dependencies because they use named modules. This was just unacceptable, so I took the performance hit to proxy an iframe as a worker so I could inject an importmap into it. So far this is working well (though much more taxing than a worker would be).

This is what I did to make that work: https://github.com/environment-safe/esm-worker/blob/master/src/index.mjs#L10-L59 (abandon all hope ye who enter). Feel free to crib the code or use the module.

Here's hoping it doesn't take another 5 years to land this as a native feature. 🤞

[JoakimCh]

Please just apply the import map to anything which can import a module (it's called an "import map" for a reason guys!!).

It's beyond ridiculous in my opinion that we are in 2024 and we're still discussing this. My worklets and workers doesn't function because of this stupidity. A terrible developer experience!

Anything but what I suggested IS a terrible developer experience and there are no GOOD arguments against it really.

6 years discussing this... It's a shame.

[jespertheend]

I don't think the discussions is what's delaying it here. I think it is because no browser vendor has the capacity to work on it.

[rektide]

To Domenic's original proposal here, just as a module has a .type on it's object exposing that attribute, it would be really cool if there was a programmatic interface to import maps attrobutes that webdevs could talk to. 👍 to a hypothetical .importMaps or what have you.

But boy oh boy am I happy to take whatever I can get here! Fine if this is implicit; I really just want anything, badly.

Is anyone successfully working around by deploying a https://github.com/joeldenning/import-map-service-worker serviceworker or some other translator?

[kungfooman]

What about relative import map paths, would they always resolve against document.baseURI (or whatever the browser internal way is to resolve relative import map paths) or should that be an option?

Simple example of relative paths:

        <script type="importmap">{"imports": {
            "esm-worker": "../src/index.js",
            "calculator": "./calculator.js"
        }}</script>