How to invalidate ES module cache

[mfidemraizer]

• 10.x:
• Linux Mint 18.3:
• runtime:

Hi!

ES modules docs state:

require.cache is not used by import. It has a separate cache.

Where's that separate cache? I need to invalidate it as it can be done in CommonJS modules for testing purposes.

How can I access that cache? Or, how can I completely invalidate the module cache?

Thank you in advance!

[devsnek]

the es module cache is designed to be immutable. we are working on hooks to allow mocking and such but raw access to the cache will never be available.

[mfidemraizer]

@devsnek

Oops! And does changing the file:// URI break the cache? I mean, I could add a random querystring parameter, since what I want to invalidate starts from a dynamic import.

In fact, I've implemented ES module mocking already but it fails to work in some scenarios because of caching:

const httpApiInitP = importWithMocks (
         '../../../../src/init.mjs',
         import.meta.url,
         [
            `../../../../../shared/src/cache/collectionOps.mjs`,
            `./mocks/collectionOpsMock.mjs`
         ]
      )

Also, does this WIP hooks are available in nightly builds? I really need an approach to mock modules or I'll need to face a very disturbing refactor in my project codebase.

[devsnek]

@mfidemraizer es modules are still in development so i wouldn't recommend migrating your project over to them just yet. Mocking/development are high on our list of things to support so don't worry about not having that ability in the future.

I could add a random querystring parameter, since what I want to invalidate starts from a dynamic import.

Yeah that's fine. We use the full URL internally (including query and hash) for the cache key.

[mfidemraizer]

@devsnek Nice (about both hooks with mocking and the thing of URLs).

My concern with randomizing URLs is I'm not breaking an entire dependency tree if I'm not mistaken: say there's a module A which mocks some import: I need to randomize URLs for all modules having the mocked module dependency too. Right?

[devsnek]

@mfidemraizer yeah nothing will depend or know about the module that got re-imported. again you will just have to wait for us to figure out how to safely expose the behaviour for this kinda stuff. you can follow along at https://github.com/nodejs/modules

[devsnek]

gonna close this -- please reopen if you feel your question hasn't been answered

[kimamula]

We use the full URL internally (including query and hash) for the cache key.

Probably I am misunderstanding something but I cannot confirm this to be true.
I have tested the caching behavior of ESModules using Node.js v11.11.0 as follows.

// esm.mjs 
console.log('Hello, ESModules!');

// import-esm.mjs
import './esm?query=1';
import './esm?query=2';

$ node --experimental-modules import-esm
(node:40370) ExperimentalWarning: The ESM module loader is experimental.
Hello, ESModules!

As shown, Hello, ESModules! is logged only once, which indicates esm.mjs is loaded only once even if the module is imported twice with different queries.

[kimamula]

Confirmed that the behavior I described above is fixed in Node.js v12.0.0.

[marxangels]

the es module cache is designed to be immutable. we are working on hooks to allow mocking and such but raw access to the cache will never be available.

Any updates for 2021? @devsnek

[rlnt]

I don't know if this is still relevant for anyone but here is my current solution.
I am still searching for a better way to do this.
What I am doing right now is cache-busting which causes a memory leak but it's better than rewriting everything.
I rely on the "type": "module" option.
https://ar.al/2021/02/22/cache-busting-in-node.js-dynamic-esm-imports/

[bcomnes]

The issue with the cache busting query string is that it doesn't reload the entire tree, so if the thing you are trying to re-import or reload imports anything else, those nested imports will not be re-read.

[bcomnes]

I found a workaround that might work for people under some circumstances. If you need to re-load an esm module, as well as its entire dependency tree, you can spawn a worker and import the module in there (https://nodejs.org/api/worker_threads.html has a good example), and it will be freshly re-imported every time you spawn a new worker. The other advantage is it avoids the memory leak that re-importing a module with a new query string in the same thread.

[frank-dspeed]

@bcomnes while that is true i wonder is that really faster then restarting the node process and how much faster

[bcomnes]

If you can get away with restarting the whole process, then that would work similarly. I’m not sure what’s more performant, I haven’t tested.

[ganeshkbhat]

I need help. How do I access the import cache like the require.cache? https://stackoverflow.com/questions/74215997/how-do-i-do-console-log-in-esm-or-es6-file-without-having-a-package-json-in-the?noredirect=1#comment131031974_74215997

[frank-dspeed]

@ganeshkbhat oh sorry that my prev answer was not clear

By ECMAScript Engine Specification your not able to Access the Module Cache as that is Managed by The Runtime not the Engine.

How ever a runtime has ways to give you access to not brake the standard NodeJS did invest into the creation of additions to its VM module to do ESM loading in userland and so you can access the cache

as also you can use loader hooks to access the cache.

in Chromium you got the webworker hooks to get access to the module cache.

hope that helps and makes some sense to you

[icetbr]

This landed on node 22: --experimental-require-module. It allows mocha --watch to work with ESM modules, which was broken feature due to require.cache.

Here is the person who found the "fix"
https://joyeecheung.github.io/blog/2024/03/18/require-esm-in-node-js/

The docs
https://nodejs.org/api/modules.html#loading-ecmascript-modules-using-require

Here is where I first saw the merge
nodejs/node#51977 (comment)

[frank-dspeed]

@icetbr its a good thing and as additonal note if that flag has any problems you can simply require the npm package with the name "esm" before your ESM code and everything works

npm install esm
node -r esm yourcode.js

Explainer the npm esm package is a real isomorphic loader that implements both specs under require and exposes module cache.