-
-
Notifications
You must be signed in to change notification settings - Fork 131
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: support Node.js 18 #283
Conversation
Note: the change includes an edit to |
const url = typeof input === 'string' ? input : input.url | ||
const url = | ||
typeof input === 'string' | ||
? input | ||
: input instanceof URL | ||
? input.href | ||
: input.url | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe this is the only user-facing change, to select .href
in the case that input
is an instance of URL
.
I don't fully grok the root cause of this - update to JSDom maybe? But I note that it fixed a type error, which was reported in my VSCode with latest TypeScript but not by any build script. It also fixed some tests, but I can't remember exactly which, since this was one of multiple bugs fixed simultaneously.
It looks like a safe change to me, though, since the new branch should only evaluate when input instanceof URL
, which (AFAICT) previously could only result in an error since there is no valid url
property on an instance of URL
. And all tests still pass in Node v14/v16, although that's not necessarily unexpected if the root cause was an update to jest-environment-jsdom
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need this change for type compliance only? Afaik Fetch supports URL
as input. Is that no longer the case for Node.js fetch?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good question. I'm looking more closely at this now. Reverting this change does not cause any tests to fail on my local (non-IPv6) system. I also checked in GitHub Actions, and no tests failed there either. So we could revert it, but let me first check if it breaks any tests in the next branch following this one. I think it could fix a real issue that was only exposed by a TypeScript error, but I don't have any regression test to prove that.
The reason it's a type error in VSCode but not any of the build scripts is that VSCode uses version 4.7.3 while this package uses version 4.3.5, and the type error is related to the latest version of the fetch
API from lib.dom.d.ts
. So the type error is likely indicating a real bug, even if there are no tests for it. Likely the bug would surface in some environments (browser vs. Node) but not others.
Specifically, the error is:
Property 'url' does not exist on type 'Request | URL'.
Property 'url' does not exist on type 'URL'.ts(2339)
as reported in VSCode:
Note that it's inferring the type of input
as RequestInfo
which it infers from the definition of globalThis.fetch
in the latest TypeScript lib.dom.d.ts
:
declare function fetch(input: RequestInfo | URL, init?: RequestInit): Promise<Response>;
visually:
Whether or not this indicates a bug at runtime depends on the fetch
API provided by the environment. If we're using a subset of that API, then we should use module augmentation to keep the TypeScript API consistent with what we expect at runtime (although admittedly it won't be an issue until we upgrade the version of TypeScript used in the build scripts).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, so reverting this change does have an effect on a downstream dependency - namely, the msw branch where I'm adding support for node-native interception. It causes tests to fail on IPv6 systems that were otherwise already fixed with this commit.
So, from a backwards compatibility perspective, it would be safer to revert this change. But from a forward compatibility perspective, we need it if we want to operate properly on RequestInfo objects in Node 18. The balance seems to weigh in favor of keeping the commit.
If you're okay with keeping this, then the PR is ready to merge
- Basically, `msw` can mostly work with node-native `fetch` powered by undici, but it does not currently create its lone export `createServer` with the `FetchInterceptor` that is necessary to intercept `globalFetch` - Clone the library locally, fix this issue, build it locally, and copy the build files into this Yarn patch Upstream, I've started the chain of PRs necessary to fix this: - mswjs/interceptors#283 (migrate to Node 18) - milesforks/msw-interceptors#5 (wip, add native fetch tests) - Last: one line change to msw that I made in our patch here
…rver This actually seems to work fine, even without any tests or changes necessary. However, for tests to pass with Node v18 and also I think to support some https cases, the PR from mswjs/interceptors should be merged: mswjs/interceptors#283
- Point `@mswjs/interceptors` to a tarball that was built from the code in this pending PR: mswjs/interceptors#283
…o fix IPv6 issue - Fix bug in IPv6 URL serialization that isn't available upstream - Source is not available, so waiting on new release Same reasoning as: mswjs/interceptors#283
…o fix IPv6 issue - Fix bug in IPv6 URL serialization that isn't available upstream - Source is not available, so waiting on new release Same reasoning as: mswjs/interceptors#283
…o fix IPv6 issue - Fix bug in IPv6 URL serialization that isn't available upstream - Source is not available, so waiting on new release Same reasoning as: mswjs/interceptors#283
- Point `@mswjs/interceptors` to a tarball that was built from the code in this pending PR: mswjs/interceptors#283
…o fix IPv6 issue - Fix bug in IPv6 URL serialization that isn't available upstream - Source is not available, so waiting on new release Same reasoning as: mswjs/interceptors#283
Btw, Node 18 pipeline on CI is passing 🎉 I believe it wasn't running because all CIs need to be approved my maintainers before running. I've encountered a flaky browser test in Node 14, rerun, and now the entire CI is green. |
I've released |
…matrix * Update development environment (`.nvmrc`) to default to Node 18 * Update CI environment (`ci.yml`) to include Node 18 in testing matrix * Upgrade to Jest 28, which is necessary because earlier versions do not export the `fetch` global (among others). In Jest 28 the system for injecting globals is changed to fix this issue. * see: See: https://jestjs.io/blog/2022/04/25/jest-28#all-nodejs-globals * also upgrade other Jest-related dependencies
8de6a91
to
09198d9
Compare
If `input` (first argument to fetch) is not a string, then it can be a URL, which does not have a `.url` property but does have a `.href` property This fixes a type error and also fixes tests that were failing in Node 18 without this fix.
…rgs` * when intercepting requests to IPv6 hosts, rewrite `baseUrl` correctly so that the host is surrounded with square brackets * apply fix to `getUrlByRequestOptions` which is called by `normalizeClientRequestArgs` * update a test that assumes `127.0.0.1` is the only valid resolution of `localhost` to also accept `::1`
…e v18 Upgrade to the latest `page-with` package, which fixes issues with IPv6 serialization in URLs caused by Node 18 resolving `localhost` to IPv6 addresses on some dual-stack systems (like GHA runners)
…ibility in Node v18 Upgrade version of `@open-draft/test-server` to fix a bug in URL serialization of IPv6 hosts, which can occur when Node v18 resolves `localhost` to an IPv6 address on systems such as GHA
Is there a planned release time? |
@Y2zz, no there's not. There's a discussion above suggesting we need additional work to make this support happen. As absolutely everything that I ship, I do so if it's work personally interesting for me and I have time for it. Anyone can help with this support, just as a few contributors above already did. Bundling the library is a standalone change and can be added and merged separately, unblocking Node 18 support. |
I'd be happy to work on it, but not sure what level of time I have over the next month - If anyone else has time sooner, I'd be happy to offer support though! |
@mattcosta7 I might have some time to look into it. Could you give some guidance on what exactly we want here? I see that in |
I don't think we need those specifically, but we probably need to consider separate imports (with shared code between them) for each of the individual interceptors, since they have code that might not evaluate in every environment, and a single file bundle won't necessarily shake things out. https://github.com/mswjs/interceptors/tree/main/src/interceptors I think we'd need to move the polyfills we want to bundle into |
@mattcosta7 first roadblack - the Edit: Even targeting ES6 results in errors such as "Transforming async generator functions to the configured target environment ("es2015") is not supported yet" (various node_modules have this problem such as |
I think we're probably ok to at least es2019 (or even esnext), but @kettanaito might have thoughts on what amount of downleveling we need there natively? I would edge towards minimizing that transpilation, and letting user applications do it if they need it? |
@mattcosta7 I've opened a draft PR for this work with some initial config based on our discussions above. Let's continue discussing the details of what exactly we want/need there. |
I'd go for @cdimitroulas, thank you for opening that pull request! I will take a look soon. |
hey all - sorry for the delay on my end. I have this all working, but I haven't rebased it for a few weeks. I diagnosed the memory leak problem as being due to the number of tests being run (each test is leaking some memory for some reason), which could be fixed by just splitting it into two runs. Otherwise, I haven't touched this since I saw some PRs in progress that mine would likely conflict with, and rebasing these PRs is pretty annoying since it touches multiple repositories I'm not sure I will get to clean this up this week, or even if it will still be necessary. But all written in my comment above still applies. In the meantime, if you want to do something really hacky you can use the tarball of
Just to emphasize, this is really hacky and I offer no guarantees about the stability of that URL. That tarball is a build of my
which is a tarball built from https://github.com/milesforks/msw-interceptors/commits/6aaf5a14562f9deffc044aa3b2872e649612f957 |
Also, I just saw the comment from @kettanaito about my fork. If you still want to help with that, I am adding you as an external contributor now (to both the interceptors and msw forks). feel free to rebase whatever you need (although i'd prefer if you push to a new branch where possible, since I'm not sure where my local copy is) |
Is it just the rebasing which needs to be fixed for this PR to get merged? I'm interested in pitching in to get this happening. |
I think we're about done with distributing this package as CJS and ESM in #313, which should be merged before this. I will rebase this branch and see if there are any loose ends. |
@kettanaito Feel free to email me if you need help with this (miles [at] splitgraph.com). I don't have the bandwidth to do all the rebasing myself atm but happy to help if you get stuck with my code |
re #283 Bundle the library, including `remix-run` and other polyfill libs, so that consumers of the lib can consume it without having to bundle/transpile it themselves. --------- Co-authored-by: Artem Zakharchenko <kettanaito@gmail.com>
Released: v0.21.0 🎉This has been released in v0.21.0! Make sure to always update to the latest version ( Predictable release automation by @ossjs/release. |
Addressed and merged this in #349. We are dropping support for Node < 18 from the next minor release. No more polyfills. |
Released: v0.22.0 🎉This has been released in v0.22.0! Make sure to always update to the latest version ( Predictable release automation by @ossjs/release. |
In this PR
This PR migrates the development environment to Node v18 (which replaces v16
as active LTS in October), and adds tests in CI for v14, v16 and v18. It fixes
compatibility issues running existing tests in a Node v18 environment and does
not break tests in earlier environments. It does not include any new user-facing features.
It does not add support for "native fetch" (if that's even missing - see #159)
but it creates the baseline for adding it, which is the next step. (I assume
this will be easy, and will hopefully only require adding new integration
tests to use the native
fetch
instead of the one fromcross-fetch
.)It's smaller than it looks. Most of the file changes are changing import
paths in 8de6a91, which patches a bug in upstream
@opendraft/test-server
by mutating and re-exporting it, hence the import path changes.
Fixes:
page-with
and@open-draft/test-server
packages? #282127.0.0.1
instead oflocalhost
, but there were still some tests failing - I forget the details. The fix in this PR doesn't change the tests and should work correctly with both IPv4 and IPv6 hosts (which will be tested as long as GitHub Actionsubuntu:latest
continues to resolvelocalhost
to an IPv6 address).Makes progress on:
need undici, but that could be added as a third-party interceptor
Related:
a now-deprecated API. Also, I don't think using undici is necessary, if we can
use global fetch interceptor, although undici could be a sensible third-party
interceptor, perhaps using its native mocks.
.nvmrc
)Upstream PR:
page-with
bug (which is fixed here via 1c4309e and 4291844) is open at feat: fix IPv6 url serialization, use node 18 kettanaito/page-with#9How to review
I suggest reviewing this commit-by-commit, because unfortunately one of the
commits (8de6a91) required updating an import
path in every test file.
If you want to run this locally, make sure you switch to Node 18. You might
also need to delete
node_modules
to fix thememfs
resolution issue(see 1c4309e), but I'm not sure.
Here's how I do that with
nvm
, while retaining my global package installations:Fix IPv6 compatibility issues
Multiple bugs shared the same root cause, which was incorrect serialization of
IPv6 hosts in URLs. This only surfaced as an issue because Node v18 now
binds to a hostname by choosing the first IP address returned by the OS,
including for the
localhost
domain name. Previously, this would only resolve to127.0.0.1
,but now on some hosts (such as GitHub Actions), it resolves to
::1
.We use this IP address, which is returned by
HttpServer.listen()
when we bindon
localhost
, to construct a URL in multiple places, mostly in tests but alsouser facing (anyone using the library and binding to
localhost
would encounterthis same bug). The bug is caused by code that incorrectly
assumes
${scheme}://${host}:${port}
serializes to a valid URL even for an IPv6host, which should be surrounded by square brackets, like
${scheme}://[${host}]:${port}
.This bug was repeated in multiple places:
interceptors
codebasetwo commits marked
fix
(eb46b83 and 22abd1c)@opendraft/test-server
codebase (dev dependency, only affects tests)page-with
and@open-draft/test-server
packages? #282the commit that changes a bunch of files, because it needs to update the imports to use
the re-exported server.
OpenDraftTestServer.ts
in that commit for the patch changes whichcould be replicated to the upstream codebase
to remove the patch (I was careful to make it atomic)
page-with
codebase (dev dependency, only affects tests)page-with
and@open-draft/test-server
packages? #282memfs
dependency needed to be upgraded. I fixed this by settinga compatible version in the
resolutions
field of ourpackage.json
so thatthe transitive dependency is forced to resolve to the latest
memfs
(1c4309e)PreviewServer
. I fixed this bypatching the singleton
server
object that is created at browser setup time (4291844).When the upstream bug is fixed, this commit can be reverted.