Requests aren't intercepted anymore since 2.4.4

[dbanck]

Prerequisites

• I confirm my issue is not in the opened issues
• I confirm the Frequently Asked Questions didn't contain the answer to my issue

Environment check

• I'm using the latest msw version
• I'm using Node.js version 18 or higher

Node.js version

v20.18.0

Background

Thanks for this great library. We've been using it for some time now to mock response in the vscode-terraform test suite.

During some regular dependency bumps we noticed that the mocking didn't work anymore.

Reproduction repository

https://github.com/dbanck/playground-vscode-msw-bug

Reproduction steps

• Open the reproduction repository with VSCode
• Run the extension (F5)
• Run the "hello world" command
  

Current behavior

The request isn't intercepted anymore

The request isn't captured by any of the life-cycle events that log to the extension debug channel

Axios logs a 404 error, because the request isn't mocked and a real request is sent to https://example.com/test

Expected behavior

I have traced it back to this change: v2.4.3...v2.4.4

When I downgrade msw to 2.4.3 everything is working as expected.

The request is logged by the lifecycle events

And my mocked response is returned

[jmccure]

Noticed the same updating to 2.4.4 here https://gitlab.com/gitlab-org/gitlab-vscode-extension/-/merge_requests/2188/diffs

[kettanaito]

Hi, @dbanck. Thanks for the detailed report. I will take a look at this once I have a moment.

[kettanaito]

Discovery

The first thing I noticed is that replacing axios.get with fetch makes MSW intercept and handle that request just fine. So the issue is somewhere in how axios.get performs a request. Following this hunch, I configured Axios to use fetch as the underlying request mechanism, and that also fixed the issue:

const request = axios.create({
  adapter: 'fetch',
})

request
  .get('https://example.com/test') // OK!

In fact, using the http adapter reproduces the issue again, and that's where I assume it's used as the default adapter (since XHR is not available in VS Code extension context).

Root cause

The request is not intercepted because the reference to http (or follow-redirects's reference to http, which is the default in axios) points to a different module than the one patched by server.listen(). As such, MSW cannot know about any calls to that different reference.

We have both Axios and follow-redirects test cases passing, and this leads to me presume that the issue here is CommonJS bundling (likely the .default. chain).

Swapping the import/require order of axios doesn't seem to make a difference.

[kettanaito]

Interestingly enough, when I inspect the request.agent on the Axios error, that's some different Agent than must be used with MSW:

That may explain why, despite being constructed, the MockHttpsAgent is never called during this HTTPS request with Axios. options.agent is undefined when it's passed to transport.request() in Axios:

// node_modules/axios/dist/node/axios.cjs
req = transport.request(options, function handleResponse(res) {

One more detail, https.request even before patching by MSW is already patched by what I assume to be Axios:

https.request BEFORE patching: ƒ patched(url, options, callback) {

This is troubling because calling this patched() function never taps into the underlying Agent and its createConnection() method as Node.js does normally. In other words, we are patching a patch here and this is a no-op.

I think the solution here is to force Axios out of that internal patching.

Looking closer at the patched() function, its origins is vscode module itself: microsoft/vscode#120354. vscode seems to be patching node:https and its https.request function!

Moreover, that patch seems to disregard any custom options.agent set, always forcing the options.agent to be the internal agent set by vscode:

options.agent = agent;

Unfortunately, the patched() function doesn't seem to be a part of the vscode package as I cannot find it in the source or in their GitHub repo.

How to fix this?

@dbanck, given vscode meddles with Node.js internals, I recommend opting out from using node:https and use fetch instead, which they don't patch, allowing MSW to work properly.

[kettanaito]

Here's the entirety of the patched() function introduced by vscode for reference:

patched(url, options, callback) {
            if (typeof url !== 'string' && !(url && url.searchParams)) {
                callback = options;
                options = url;
                url = null;
            }
            if (typeof options === 'function') {
                callback = options;
                options = null;
            }
            options = options || {};
            if (options.socketPath) {
                return original.apply(null, arguments);
            }
            const originalAgent = options.agent;
            if (originalAgent === true) {
                throw new Error('Unexpected agent option: true');
            }
            const isHttps = originals.globalAgent.protocol === 'https:';
            const optionsPatched = originalAgent instanceof agent_1.PacProxyAgent;
            const config = params.getProxySupport();
            const useProxySettings = !optionsPatched && (config === 'override' || config === 'fallback' || (config === 'on' && originalAgent === undefined));
            const addCertificatesV1 = !optionsPatched && params.addCertificatesV1() && isHttps && !options.ca;
            if (useProxySettings || addCertificatesV1) {
                if (url) {
                    const parsed = typeof url === 'string' ? new nodeurl.URL(url) : url;
                    const urlOptions = {
                        protocol: parsed.protocol,
                        hostname: parsed.hostname.lastIndexOf('[', 0) === 0 ? parsed.hostname.slice(1, -1) : parsed.hostname,
                        port: parsed.port,
                        path: `${parsed.pathname}${parsed.search}`
                    };
                    if (parsed.username || parsed.password) {
                        options.auth = `${parsed.username}:${parsed.password}`;
                    }
                    options = Object.assign(Object.assign({}, urlOptions), options);
                }
                else {
                    options = Object.assign({}, options);
                }
                const resolveP = (req, opts, url) => new Promise(resolve => resolveProxy({ useProxySettings, addCertificatesV1 }, req, opts, url, resolve));
                const host = options.hostname || options.host;
                const isLocalhost = !host || host === 'localhost' || host === '127.0.0.1'; // Avoiding https://github.com/microsoft/vscode/issues/120354
                const agent = (0, agent_1.default)(resolveP, {
                    originalAgent: (!useProxySettings || isLocalhost || config === 'fallback') ? originalAgent : undefined,
                    lookupProxyAuthorization: params.lookupProxyAuthorization,
                });
                agent.protocol = isHttps ? 'https:' : 'http:';
                options.agent = agent;
                return original(options, callback);
            }
            return original.apply(null, arguments);
        }

[kettanaito]

This wasn't an issue before because MSW's patch was more aggressive, overriding http.request altogether. With recent versions, we've migrated to a more permissive patch that allows more of the Node.js network code to run, which is beneficial on all accounts. As a side-effect of this change, MSW now expects the underlying code to be even more Node.js-compatible, having no patches whatsoever.