Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Concerns with TypeScript 4.5's Node 12+ ESM Support #46452

Closed
DanielRosenwasser opened this issue Oct 20, 2021 · 170 comments
Closed

Concerns with TypeScript 4.5's Node 12+ ESM Support #46452

DanielRosenwasser opened this issue Oct 20, 2021 · 170 comments
Labels
Discussion Issues which may not have code impact

Comments

@DanielRosenwasser
Copy link
Member

For TypeScript 4.5, we've added a new module mode called node12 to better-support running ECMAScript modules in Node.js. Conceptually, the feature is simple - for whatever Node.js does, either match it or overlay something TypeScript-specific that mirrors thes same functionality. This is what TypeScript did for our initial Node.js support (i.e. --moduleResolution node and --module commonjs); however, the feature is much more expansive, and over the past few weeks, a few of us have grown a bit concerned about the complexity.

I recently put together a list of user scenarios and possibly useful scripts for a few people on the team to run through, and we found a few sources of concerns.

  • Bugs
  • UX
  • Ecosystem
  • User Guidance

Bugs

Most complex software ships with a few bugs. Obviously, we want to avoid them, but the more complex a feature is, the harder it is to cover all the use-cases. As we get closer to our RC date, do we feel confident that what we're shipping has as few blocking bugs as possible?

I would like to say we're close, but the truth is I have no idea. It feels like we'll have to keep trying the features for a bit until we don't run into anything - but we have less than 3 weeks before the RC ships.

Here's a few surprising bugs that need to get fixed before I would feel comfortable shipping node12 in stable.

UX Concerns

In addition to bugs we found, there are just several UX concerns. Package authoring is already a source of confusion in the TypeScript ecosystem. It's too easy to accidentally shoot yourself in the foot as a package author, and it's too hard to correctly consume misconfigured packages. The node12 mode makes this a whole lot worse. Two filed examples of user confusion:

While there might be a lot of "working as intended" behavior here, the question is not about whether it works, but how it works - how do we tell users when something went wrong. I think the current implementation leaves a lot of room for some polish.

But there are some questions about this behavior, and we've had several questions about whether we can simplify it. One motivating question I have is:

When a user creates a new TypeScript project with this mode, when would they not want "type": "module"? Why? Should that be required by default?

We've discussed this a bit, and it seems a bit strange that because we want to cover the "mixed mode" case so much, every Node 12+ user will have to avoid this foot-gun.

I would like to see a world where we say "under this mode, .ts files must be covered by a "type": "module"". .cts can do their own CommonJS thing, but they need to be in a .cts file.

Another motivating question is:

Why would I use node12 today instead of nodenext?

Node 14.8 added top-level await, but Node 12 doesn't have it. I think this omission is enough of a wart that starting at Node 12 is the wrong move.

Ecosystem

The ecosystem is CONFUSING here. Here's a taste of what we've found:

  • ts-node, Webpack, and Vite don't like imports with .js extensions, but TypeScript expects them. Not all of these can be configured with a plugin.
  • ts-node, Webpack, and Vite, and Deno are fine with .ts extensions, but TypeScript doesn't allow them!
  • Many packages that ship types have started supporting export fields, but don't have a types sub-field within export (e.g. RxJS, Vue 3).
  • Many packages have started supporting export fields, but their @types package might not reflect that.

The last two can be easily fixed over time, though it would be nice to have the team pitch in and help prepare a bit here, especially because it's something that affects our tooling for JavaScript users as well (see #46339)

However, the first two are real issues with no obvious solutions that fall within our scope.

There's also other considerations like "what about import maps?" Does TypeScript ever see itself leveraging those in some way, and will package managers ever support generating them?

Guidance

With --moduleResolution node, it became clear over time that everyone should use this mode. It made sense for Node.js apps/scripts, and it made sense for front-end apps that were going to go through a bundler. Even apps that didn't actually load from node_modules could take advantage of @types in a fairly straightforward way.

Now we have an ecosystem mismatch between Node.js and bundlers. No bundler is compatible with this new TypeScript mode (and keep in mind, back-end code also occasionally uses a bundler).

Here's some questions I wouldn't know how to answer confidently:

  • Is our guidance to always use this mode for plain Node.js apps, and let tools "catch up"?
  • Should new projects that use this mode pick node12 or nodenext?
  • There's a big foot-gun with forgetting "type": "module" - should we always recommend .mts?

Next Steps

I see us having 3 options on the table:

  • A mad dash to fix everything - I think this is too hard in the next 2 weeks to pull off.
  • Keep the feature in, but make it inaccessible until we're ready - I think temporarily removing the feature would be impractical. It would probably take more work to remove it than to fix the issues I've already mentioned. So we might as well keep it in.
  • Ship with some "experimental" labeling - I think this makes the most sense, but with some caveats to what I mean by "ship". It would make sense to just ship this as "experimental", but I think we should make this feature only work in nightly releases so that people can continue to easily use it, but not depend on a stable version until it's ready.
@frank-dspeed
Copy link

I think this issue is a good example for the long needed plugin (hook) system.

The Solution to the first 2 Problems is rollup at present you can use it with plugin typescript to resolve anything correct and then inject it into the typescript program.

i am already researching how i could maintain and release a typescript-rollup version which would be typescript + rollup hooks and plugins.

Conclusion

a Plugin/Hook System is the Solution for the Resolve Problem. The Only one that is flexible and adjustable enough to cover every case.

@FossPrime
Copy link

FossPrime commented Oct 25, 2021

Conclusion

a Plugin/Hook System is the Solution for the Resolve Problem. The Only one that is flexible and adjustable enough to cover every case.

There is already a hooks system built into package.json... it's bad, but the whole point is to get rid of it as soon as dependencies merge the PR's to fix the issues.

I have the following install hook as a bandaid while upstream applies the fixes to default imports and reachable types/modules.

#!/bin/bash
set -euo pipefail

sed -i '2s/import express/import \* as express/' node_modules/\@feathersjs/express/index.d.ts
sed -i '1s/import http/import \* as http/' node_modules/\@feathersjs/socketio/index.d.ts
sed -i '2s/import io/import \* as io/' node_modules/\@feathersjs/socketio/index.d.ts
sed -i '8s/"source",/"\.\/source\/index\.js",\n"types": "\.\/index\.d\.ts",/' node_modules/chalk/package.json

I'd rather have 4.5 stable sooner, than to wait for yet another workaround.

@frank-dspeed
Copy link

@rayfoss we can take your example to again show that a plugin/hook system like the one from rollup is badly neeeded.

you mixed linux shell script into your package.json as workaround.

@Jamesernator
Copy link

Jamesernator commented Oct 25, 2021

Now we have an ecosystem mismatch between Node.js and bundlers. No bundler is compatible with this new TypeScript mode (and keep in mind, back-end code also occasionally uses a bundler).
Is our guidance to always use this mode for plain Node.js apps, and let tools "catch up"?

Yeah, so the trouble is if TypeScript doesn't encourage the use of node12/nodenext then the package author will be unable to use any packages that use Node ESM. So no matter which choice the author makes some things will be invariably broken.

This is something I've mentioned in the past on some related issue, but has it been considered not to have a distinct node and node12/nodenext mode at all, but rather just use --module node and require the presence of "type": "module" or "type": "commonjs" in package.json? (And regardless of this being present, .mts/.mjs/.cts/.cjs would work always, this would only be required for using .ts/.js in the new mode).

By having both --module nodenext and "type": "module" people are gonna be essentially double-configuring in a lot of cases anyway (most cases?), given people have to add --module nodenext to their tsconfig to enter the mode anyway, I don't see that it would be significantly worse to have them add "type": "module"/"commonjs" to their package.json instead.

@TheThing
Copy link

TheThing commented Oct 25, 2021

Conclusion

a Plugin/Hook System is the Solution for the Resolve Problem. The Only one that is flexible and adjustable enough to cover every case.

This seems like the absolute worst conclusion to ever reach. I'm really sorry for butting in on this issue as not a huge avid typescript users but the one thing I like about typescript is precisely it doesn't require 9001 packages like soo many other builders and bundlers to actually get into a working state.

Typescript is standalone that just-works and doesn't require the end programmer to have to build-their-own-compiler themselves. Add plugins is just gonna be that, making it more and more complicated while adding nothing really. Especially something like this issue that REALLY needs to work out of the box but isn't. And saying to people "oh you're using typescript but typescript is dumb like bundlers, you need to hook plugins to get it working" is something you don't want to say to developers or people.

I really strongly advice against any case of adding plugins to this package. If it doesn't work, we can fix it. If it works, why would you need a plugin just to make configging even more complicated?

Best regards:
TT

P.S.
what is it with developers and wanting plugins in literally everything?

@frank-dspeed
Copy link

frank-dspeed commented Oct 25, 2021

@TheThing in my case it is simply needed because there are many package authors with total diffrent opinions and i do not want to get blocked by them. I also do not want to hardfork everything and so on.

The only Alternativ to a plugin system in typescript is the usage of dev bundels that are typescript compatible.

also my conclusion is driven from the fact that there are tons of other environments not only nodejs

i only vote for resolve hooks and plugins because of all the diffrent environments as also package managers.

@Jack-Works
Copy link
Contributor

Node 14.8 added top-level await, but Node 12 doesn't have it. I think this omission is enough of a wart that starting at Node 12 is the wrong move.

What about removing Node12 and starts from Node 14?

@andrewbranch
Copy link
Member

andrewbranch commented Oct 29, 2021

One note here from https://github.com/microsoft/TypeScript/issues/46550#issuecomment-954348769—for users who have declaration emit enabled, error messages like this are probably going to be a common symptom of a dependency that needs to update their export map to include "types" conditions:

error TS2742: The inferred type of 'T' cannot be named without a reference to '../../../node_modules/async-call-rpc/out/full'. This is likely not portable. A type annotation is necessary.

It feels pretty non-obvious that that’s what’s going on so I wanted to make a note of it here. Basically, the declaration emitter wants to print a type like import("async-call-rpc/out/full").T, but it can’t because the package has an export map that doesn’t specify any "types". So the only way it can reach that module is with a relative import through node_modules, which is not allowed to be synthesized by declaration emit. If you look at the package’s package.json, you can see that the actual specifier that should be generated will probably be "async-call-rpc/full", but each of those entrypoints in the export map needs a "types" if it’s going to be resolvable by TypeScript.

@andrewbranch
Copy link
Member

☝️ Actually, I was wrong about this particular example. This may be a common symptom for some packages if their typings are stored in a separate folder from the JS that the export maps point to, like RxJs does. However, if the .d.ts files are colocated with the .js/.mjs/.cjs files pointed to by the export map, this should “just work.” This is the case for async-call-rpc, so the fact that the declaration emitter is complaining here is probably indicative of a module specifier resolution bug. (cc @Jack-Works)

@Jack-Works
Copy link
Contributor

Oh. Yes, I found that package actually exports the correct typing and JS file at /full, not /out/full.

@demurgos
Copy link

demurgos commented Oct 30, 2021

There are some rough edges with the new resolution mode. I've been trying out this mode for two weeks.

The main issues I've hit are:

Despite all of these issues, I still hope that the new resolution algorithm becomes available on stable TypeScript: the core seems good and bugs/UX can be iterated. The new resolution mode lets me get rid of tons hacks and makes it finally ergonomic to use ESM natively.

  • I no longer have to manually watch out for explicit extensions in relative imports
  • I can use exports with subpath mapping to expose my outputDir directly for deep-imports; while being fully compatible with npm|yarn link and project references, and without having to abuse typeVersions.
  • For complex cases where types are generated directly in the build directory (e.g. using wasm-bindgen), I am now able to use import maps and remove dummy files used just so TypeScript does not complain about missing files.
  • Other various quality of life improvements such as proper handling of import.meta or __dirname.

I was able to use node12 with Node, webpack, mocha, wasm-bindgen, native node modules and Angular 13.

In my personal experience, all of these improvements strongly outweight the current issues: it makes things so much simpler as it realigns TS behavior's with Node's. I hope that the current issues get fixed soon and I hope that the concerns will not delay the new resolution too much.

@andrewbranch
Copy link
Member

unstable import resolution issues in IntelliJ IDEA (looks to be related to #46389)

@demurgos did you get the right issue number here? This doesn't sound like it would be related to what you're talking about 🤔

@demurgos
Copy link

demurgos commented Nov 1, 2021

I wanted to link the issue 46396: "nodenext alternates between finding and not finding the imported package"

I've double checked my message: I linked it as #46396 but it looks like GitHub failed to resolve it properly. Editing my message to add a single space to force a refresh fixed the rendering.

@Exac
Copy link

Exac commented Nov 2, 2021

What if TypeScript 5 assumes "type": "module" by default?

It would be regrettable if we have to add "type": "module" to our package.json in the far future, when many have migrated.

@Jamesernator
Copy link

Jamesernator commented Nov 2, 2021

What if TypeScript 5 assumes "type": "module" by default?

It would be regrettable if we have to add "type": "module" to our package.json in the far future, when many have migrated.

Just to be clear it is Node (not TypeScript) that requires "type": "module" in package.json and that isn't likely to change probably ever. TypeScript just reads the value to understand how Node will run the module, TypeScript doesn't control how the module is run.

@frank-dspeed
Copy link

frank-dspeed commented Nov 3, 2021

at present i think the type fild in the package.json is less relevant as that is only a switch for the .js extension inside NodeJs Typescript at present 4.5+ detects the module type via import and export statments inside the .js files this should not change.

Typescript is not a NodeJS only Product at last i guess that.

ps i still have Javascript Projects without a package.json at all and i use the global installed typescript to typecheck them it works great and it should stay working.

@orta
Copy link
Contributor

orta commented Nov 3, 2021

For folks who are interested in testing esm-node out:

npm add typescript@4.5.0-dev.20211101--save-dev
yarn add typescript@4.5.0-dev.20211101 --dev
pnpm add typescript@4.5.0-dev.20211101 --dev

This is the closest nightly npm release to the RC, so can act as "4.5 but with ESM enabled" for your projects.

Vylpes pushed a commit to Vylpes/random-bunny that referenced this issue Sep 5, 2023
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [got](https://github.com/sindresorhus/got) | resolutions | major | [`^11.8.5` -> `^13.0.0`](https://renovatebot.com/diffs/npm/got/11.8.6/13.0.0) |

---

### Release Notes

<details>
<summary>sindresorhus/got</summary>

### [`v13.0.0`](https://github.com/sindresorhus/got/releases/tag/v13.0.0)

[Compare Source](sindresorhus/got@v12.6.1...v13.0.0)

As a reminder, Got continues to require ESM. For TypeScript users, this includes having [`"module": "node16", "moduleResolution": "node16"` in your tsconfig](https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c#how-can-i-make-my-typescript-project-output-esm).

##### Breaking

-   Require Node.js 16  [`52a1063`](sindresorhus/got@52a1063)
-   Change the [`enableUnixSockets`](https://github.com/sindresorhus/got/blob/main/documentation/2-options.md#enableunixsockets) option to be `false` by default  [`852c312`](sindresorhus/got@852c312)
    -   Most users don't need it.

##### Improvements

-   Allow specifying `undefined` for options ([#&#8203;2258](sindresorhus/got#2258))  [`1cefe8b`](sindresorhus/got@1cefe8b)

### [`v12.6.1`](https://github.com/sindresorhus/got/releases/tag/v12.6.1)

[Compare Source](sindresorhus/got@v12.6.0...v12.6.1)

-   Fix `get-stream` import statement ([#&#8203;2266](sindresorhus/got#2266))  [`67d5039`](sindresorhus/got@67d5039)

### [`v12.6.0`](https://github.com/sindresorhus/got/releases/tag/v12.6.0)

[Compare Source](sindresorhus/got@v12.5.3...v12.6.0)

-   Update dependencies  [`88c88fb`](sindresorhus/got@88c88fb) [`979272e`](sindresorhus/got@979272e)
-   Loosen URL validation strictness ([#&#8203;2200](sindresorhus/got#2200))  [`0ca0b7f`](sindresorhus/got@0ca0b7f)

### [`v12.5.3`](https://github.com/sindresorhus/got/releases/tag/v12.5.3)

[Compare Source](sindresorhus/got@v12.5.2...v12.5.3)

-   Fix abort event listeners not always being cleaned up ([#&#8203;2162](sindresorhus/got#2162))  [`3cc40b5`](sindresorhus/got@3cc40b5)

### [`v12.5.2`](https://github.com/sindresorhus/got/releases/tag/v12.5.2)

[Compare Source](sindresorhus/got@v12.5.1...v12.5.2)

-   Improve TypeScript 4.9 compatibility ([#&#8203;2163](sindresorhus/got#2163))  [`39f83b6`](sindresorhus/got@39f83b6)

### [`v12.5.1`](https://github.com/sindresorhus/got/releases/tag/v12.5.1)

[Compare Source](sindresorhus/got@v12.5.0...v12.5.1)

-   Fix compatibility with TypeScript and ESM  [`3b3ea67`](sindresorhus/got@3b3ea67)
-   Fix request body not being properly cached ([#&#8203;2150](sindresorhus/got#2150))  [`3e9d3af`](sindresorhus/got@3e9d3af)

### [`v12.5.0`](https://github.com/sindresorhus/got/releases/tag/v12.5.0)

[Compare Source](sindresorhus/got@v12.4.1...v12.5.0)

-   Disable method rewriting on 307 and 308 status codes ([#&#8203;2145](sindresorhus/got#2145))  [`e049e94`](sindresorhus/got@e049e94)
-   Upgrade dependencies  [`8630815`](sindresorhus/got@8630815) [`f0ac0b3`](sindresorhus/got@f0ac0b3) [`4c3762a`](sindresorhus/got@4c3762a)

### [`v12.4.1`](https://github.com/sindresorhus/got/releases/tag/v12.4.1)

[Compare Source](sindresorhus/got@v12.4.0...v12.4.1)

##### Fixes

-   Fix `options.context` being not extensible [`b671480`](sindresorhus/got@b671480)
-   Don't emit `uploadProgress` after promise cancelation [`693de21`](sindresorhus/got@693de21)

### [`v12.4.0`](https://github.com/sindresorhus/got/releases/tag/v12.4.0)

[Compare Source](sindresorhus/got@v12.3.1...v12.4.0)

##### Improvements

-   Support FormData without known length ([#&#8203;2120](sindresorhus/got#2120))  [`850773c`](sindresorhus/got@850773c)

##### Fixes

-   Don't call `beforeError` hooks with `HTTPError` if the `throwHttpErrors` option is `false` ([#&#8203;2104](sindresorhus/got#2104))  [`3927348`](sindresorhus/got@3927348)

### [`v12.3.1`](https://github.com/sindresorhus/got/releases/tag/v12.3.1)

[Compare Source](sindresorhus/got@v12.3.0...v12.3.1)

-   Don't freeze signal when freezing Options ([#&#8203;2100](sindresorhus/got#2100))  [`43b1467`](sindresorhus/got@43b1467)

### [`v12.3.0`](https://github.com/sindresorhus/got/releases/tag/v12.3.0)

[Compare Source](sindresorhus/got@v12.2.0...v12.3.0)

-   Add `.off()` method for events ([#&#8203;2092](sindresorhus/got#2092))  [`88056be`](sindresorhus/got@88056be)

### [`v12.2.0`](https://github.com/sindresorhus/got/releases/tag/v12.2.0)

[Compare Source](sindresorhus/got@v12.1.0...v12.2.0)

-   [Support `AbortController`](https://github.com/sindresorhus/got/blob/main/documentation/2-options.md#signal) ([#&#8203;2020](sindresorhus/got#2020))  [`6a6d2a9`](sindresorhus/got@6a6d2a9)
-   Add [`enableUnixSockets`](https://github.com/sindresorhus/got/blob/main/documentation/2-options.md#enableunixsockets) option ([#&#8203;2062](sindresorhus/got#2062))  [`461b3d4`](sindresorhus/got@461b3d4)

### [`v12.1.0`](https://github.com/sindresorhus/got/releases/tag/v12.1.0)

[Compare Source](sindresorhus/got@v12.0.4...v12.1.0)

##### Improvements

-   Add `response.ok` ([#&#8203;2043](sindresorhus/got#2043))  [`22d58fb`](sindresorhus/got@22d58fb)
    -   This is only useful if you have [`{throwHttpErrors: false}`](https://github.com/sindresorhus/got/blob/main/documentation/2-options.md#throwhttperrors)

##### Fixes

-   Do not redirect to UNIX sockets ([#&#8203;2047](sindresorhus/got#2047))  [`861ccd9`](sindresorhus/got@861ccd9)
    -   [CVE-2022-33987](https://nvd.nist.gov/vuln/detail/CVE-2022-33987)
    -   [Also back ported to v11](https://github.com/sindresorhus/got/releases/tag/v11.8.5)

### [`v12.0.4`](https://github.com/sindresorhus/got/releases/tag/v12.0.4)

[Compare Source](sindresorhus/got@v12.0.3...v12.0.4)

-   Remove stream lock - unreliable since Node 17.3.0 [`bb8eca9`](sindresorhus/got@bb8eca9)

### [`v12.0.3`](https://github.com/sindresorhus/got/releases/tag/v12.0.3)

[Compare Source](sindresorhus/got@v12.0.2...v12.0.3)

-   Allow more types in the `json` option ([#&#8203;2015](sindresorhus/got#2015))  [`eb045bf`](sindresorhus/got@eb045bf)

### [`v12.0.2`](https://github.com/sindresorhus/got/releases/tag/v12.0.2)

[Compare Source](sindresorhus/got@v12.0.1...v12.0.2)

-   Fix `encoding` option with `{responseType: 'json'}` ([#&#8203;1996](sindresorhus/got#1996))  [`0703318`](sindresorhus/got@0703318)

### [`v12.0.1`](https://github.com/sindresorhus/got/releases/tag/v12.0.1)

[Compare Source](sindresorhus/got@v12.0.0...v12.0.1)

-   Fix `nock` compatibility ([#&#8203;1959](sindresorhus/got#1959))  [`bf39d2c`](sindresorhus/got@bf39d2c)
-   Fix missing export of `Request` TypeScript type ([#&#8203;1940](sindresorhus/got#1940))  [`0f9f2b8`](sindresorhus/got@0f9f2b8)

### [`v12.0.0`](https://github.com/sindresorhus/got/releases/tag/v12.0.0)

[Compare Source](sindresorhus/got@v11.8.6...v12.0.0)

##### Introducing Got v12.0.0 🎉

Long time no see! The latest Got version (v11.8.2) was released just in February ❄️
We have been working hard on squashing bugs and improving overall experience.

If you find Got useful, you might want to [sponsor the Got maintainers](https://github.com/sindresorhus/got?sponsor=1).

##### This package is now pure ESM

**Please [read this](https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c).** Also see sindresorhus/got#1789.

-   **Please don't open issues about `[ERR_REQUIRE_ESM]` and `Must use import to load ES Module` errors.** This is a problem with your setup, not Got.
-   Please don't open issues about using Got with Jest. Jest does not fully support ESM.
-   Pretty much any problem with loading this package is a problem with your bundler, test framework, etc, not Got.
-   If you use TypeScript, you will want to stay on Got v11 until TypeScript 4.6 is out. [Why.](microsoft/TypeScript#46452)
-   If you use a bundler, make sure it supports ESM and that you have correctly configured it for ESM.
-   The Got issue tracker is not a support channel for your favorite build/bundler tool.

##### Required Node.js >=14

While working with streams, we encountered more Node.js bugs that needed workarounds.
In order to keep our code clean, we had to drop Node.js v12 as the code would get more messy.
We strongly recommend that you update Node.js to **v14 LTS**.

##### HTTP/2 support

Every Node.js release, the native `http2` module gets more stable.
Unfortunately there are still some issues on the Node.js side, so we decided to keep HTTP/2 disabled for now.
We may enable it by default in Got v13. It is still possible to turn it on via the `http2` option.

To run HTTP/2 requests, it is required to use Node.js **v15.10** or above.

##### Bug fixes

Woah, we possibly couldn't make a release if we didn't fix some bugs!

-   Do not throw on custom stack traces ([#&#8203;1491](sindresorhus/got#1491)) [`49c16ee`](sindresorhus/got@49c16ee)
-   Remove automatic `content-length` on ReadStream ([#&#8203;1510](sindresorhus/got#1510)) [`472b8ef`](sindresorhus/got@472b8ef)
-   Fix promise shortcuts in case of error status code ([#&#8203;1543](sindresorhus/got#1543)) [`ff918fb`](sindresorhus/got@ff918fb) [`1107cc6`](sindresorhus/got@1107cc6)
-   Invert the `methodRewriting` option [`51d88a0`](sindresorhus/got@51d88a0)
-   Fix `url` not being reused on retry in rare case ([#&#8203;1487](sindresorhus/got#1487)) [`462bc63`](sindresorhus/got@462bc63)
-   Fix hanging promise on HTTP/2 timeout ([#&#8203;1492](sindresorhus/got#1492)) [`a59fac4`](sindresorhus/got@a59fac4)
-   Prevent uncaught ParseErrors on initial successful response ([#&#8203;1527](sindresorhus/got#1527)) [`77df9c3`](sindresorhus/got@77df9c3)
-   Throw an error when retrying with consumed body ([#&#8203;1507](sindresorhus/got#1507)) [`62305d7`](sindresorhus/got@62305d7)
-   Fix a Node.js 16 bug that hangs Got streams [`06a2d3d`](sindresorhus/got@06a2d3d)
-   Fix default pagination handling for empty Link header ([#&#8203;1768](sindresorhus/got#1768)) [`1e1e506`](sindresorhus/got@1e1e506)
-   Fix incorrect `response.complete` when using cache [`9e15d88`](sindresorhus/got@9e15d88)
-   Fix `Cannot call end` error when `request` returns a `Writable` [`226cc39`](sindresorhus/got@226cc39)
-   Fix Request options not being reused on retry [`3c23eea`](sindresorhus/got@3c23eea)
-   Fix types being not compatible with CommonJS [`3c23eea`](sindresorhus/got@3c23eea)
-   Fix `got.paginate does not call init hooks` ([#&#8203;1574](sindresorhus/got#1574)) [`3c23eea`](sindresorhus/got@3c23eea)
-   Generate a new object when passing options to the native `https` module ([#&#8203;1567](sindresorhus/got#1567)) [`3c23eea`](sindresorhus/got@3c23eea)
-   Remove stream reuse check ([#&#8203;1803](sindresorhus/got#1803)) [`9ecc5ee`](sindresorhus/got@9ecc5ee)
-   Fix merging `searchParams` ([#&#8203;1814](sindresorhus/got#1814)) [`1018c20`](sindresorhus/got@1018c20) [`732e9bd`](sindresorhus/got@732e9bd)
-   Fix unhandled exception when lookup returns invalid IP early ([#&#8203;1737](sindresorhus/got#1737)) [`2453e5e`](sindresorhus/got@2453e5e)
-   Fix relative URLs when paginating [`439fb82`](sindresorhus/got@439fb82)
-   Require url to be an instance of URL when paginating ([#&#8203;1818](sindresorhus/got#1818)) [`eda69ff`](sindresorhus/got@eda69ff)
-   Fix `username` and `password` encoding in URL ([#&#8203;1169](sindresorhus/got#1169) [#&#8203;1317](sindresorhus/got#1317)) [`d65d0ca`](sindresorhus/got@d65d0ca)
-   Clone raw options [`1c4cefc`](sindresorhus/got@1c4cefc)
-   Fix invalid `afterResponse` return check  [`cbc8902`](sindresorhus/got@cbc8902)
-   Fix `https.alpnProtocols` not having an effect  [`e1099fb`](sindresorhus/got@e1099fb)

##### Improvements

-   Make the `context` option mergeable ([#&#8203;1459](sindresorhus/got#1459)) [`2b8ed1f`](sindresorhus/got@2b8ed1f)
-   Add generic argument to AfterResponseHook TypeScript type ([#&#8203;1589](sindresorhus/got#1589)) [`6fc04a9`](sindresorhus/got@6fc04a9)
-   Add read timeout ([#&#8203;1518](sindresorhus/got#1518)) [`e943672`](sindresorhus/got@e943672) *(blocked by nodejs/node#35923
-   Improve the pagination API ([#&#8203;1644](sindresorhus/got#1644)) [`2675046`](sindresorhus/got@2675046)
-   Change the stackAllItems option to be false by default ([#&#8203;1645](sindresorhus/got#1645)) [`1120370`](sindresorhus/got@1120370)
-   Throw when afterResponse hook returns an invalid value [`4f21eb3`](sindresorhus/got@4f21eb3)
-   Add `retry.backoffLimit` option [`41c4136`](sindresorhus/got@41c4136)
-   Add `noise` retry option [`e830077`](sindresorhus/got@e830077)
-   Enable more HTTPS options [`83575d5`](sindresorhus/got@83575d5) [`fe723a0`](sindresorhus/got@fe723a0) (thanks [@&#8203;Giotino](https://github.com/Giotino))
-   Define `error.code` [`f27e8d3`](sindresorhus/got@f27e8d3)
-   Set `options.url` even if some options are invalid [`8d6a680`](sindresorhus/got@8d6a680)
-   Improve memory usage when merging options [`2db5ec5`](sindresorhus/got@2db5ec5)
-   Support async generators as body [`854430f`](sindresorhus/got@854430f) [`3df52f3`](sindresorhus/got@3df52f3)
-   Add missing `once` types for Stream API [`3c23eea`](sindresorhus/got@3c23eea)
-   New error type: `RetryError` which always triggers a new retry when thrown [`3c23eea`](sindresorhus/got@3c23eea)
-   `error.options` is now enumerable [`3c23eea`](sindresorhus/got@3c23eea)
-   `defaults.handlers` don't need a default handler now [`3c23eea`](sindresorhus/got@3c23eea)
-   Add a parser for the `Link` header [`3c23eea`](sindresorhus/got@3c23eea)
-   General code improvements [`a5dd9aa`](sindresorhus/got@a5dd9aa)

##### Breaking changes

##### Improved option normalization

-   Got exports an `Option` class that is specifically designed to parse and validate Got options.
    It is made of setters and getters that provide fast normalization and more consistent behavior.

When passing an option does not exist, Got will throw an error. In order to retrieve the options before the error, use `error.options`.

```js
import got from 'got';

try {
    await got('https://httpbin.org/anything', {
        thisOptionDoesNotExist: true
    });
} catch (error) {
    console.error(error);
    console.error(error.options.url.href);
    // Unexpected option: thisOptionDoesNotExist
    // https://httpbin.org/anything
}
```

-   The `init` hook now accepts a second argument: `self`, which points to an `Options` instance.

In order to define your own options, you have to move them to `options.context` in an [`init` hook](https://github.com/sindresorhus/got/blob/main/documentation/lets-make-a-plugin.md#authorization) or store them in `options.context` directly.

-   The `init` hooks are ran only when passing an options object explicitly.

```diff
- await got('https://example.com'); // this will *not* trigger the init hooks
+ await got('https://example.com', {}); // this *will** trigger init hooks
```

-   [`options.merge()`](2-options.md) replaced `got.mergeOptions` and `Request.normalizeArguments`

```diff
- got.defaults.options = got.mergeOptions(got.defaults.options, {…});
+ got.defaults.options.merge(…);
```

This fixes issues like [#&#8203;1450](sindresorhus/got#1450)

-   Legacy `Url` instances are not supported anymore. You need to use WHATWG URL instead.

```diff
- await got(string, {port: 8443});
+ const url = new URL(string);
+ url.port = 8443;
+ await got(url);
```

-   No implicit timeout declaration.

```diff
- await got('https://example.com', {timeout: 5000})
+ await got('https://example.com', {timeout: {request: 5000})
```

-   No implicit retry declaration.

```diff
- await got('https://example.com', {retry: 5})
+ await got('https://example.com', {retry: {limit: 5})
```

-   `dnsLookupIpVersion` is now a number (4 or 6) or undefined

```diff
- await got('https://example.com', {dnsLookupIpVersion: 'ipv4'})
+ await got('https://example.com', {dnsLookupIpVersion: 4})
```

-   `redirectUrls` and `requestUrl` now give URL instances

```diff
- request.requestUrl
+ request.requestUrl.origin
+ request.requestUrl.href
+ request.requestUrl.toString()
```

```diff
- request.redirectUrls[0]
+ request.redirectUrls[0].origin
+ request.redirectUrls[0].href
+ request.redirectUrls[0].toString()
```

-   Renamed `request.aborted` to `request.isAborted`

```diff
- request.aborted
+ request.isAborted
```

Reason: consistency with `options.isStream`.

-   Renamed the `lookup` option to `dnsLookup`

```diff
- await got('https://example.com', {lookup: cacheable.lookup})
+ await got('https://example.com', {dnsLookup: cacheable.lookup})
```

-   The `beforeRetry` hook now accepts only two arguments: `error` and `retryCount`

```diff
await got('https://example.com', {
    hooks: {
        beforeRetry: [
-            (options, error, retryCount) => {
-                console.log(options, error, retryCount);
-            }
+            (error, retryCount) => {
+                console.log(error.options, error, retryCount);
+            }
        ]
    }
})
```

The `options` argument has been removed, however it's still accessible via `error.options`. All modifications on `error.options` will be reflected in the next requests (no behavior change, same as with Got 11).

-   The `beforeRedirect` hook's first argument (options) is now a cloned instance of the Request options.

This was done to make retrieving the original options possible: `plainResponse.request.options`.

```diff
await got('http://szmarczak.com', {
    hooks: {
        beforeRedirect: [
            (options, response) => {
-                console.log(options === response.request.options); //=> true [invalid! our original options were overriden]
+                console.log(options === response.request.options); //=> false [we can access the original options now]
            }
        ]
    }
})
```

-   The `redirect` event now takes two arguments in this order: `updatedOptions` and `plainResponse`.

```diff
- stream.on('redirect', (response, options) => …)
+ stream.on('redirect', (options, response) => …)
```

Reason: consistency with the `beforeRedirect` hook.

-   The `socketPath` option has been removed. Use the `unix:` protocol instead.

```diff
- got('/containers/json', {socketPath: '/var/run/docker.sock'})
+ got('unix:/var/run/docker.sock:/containers/json')
+ got('http://unix:/var/run/docker.sock:/containers/json')
```

-   The `retryWithMergedOptions` function in an `afterResponse` hook no longer returns a `Promise`.

It now throws `RetryError`, so this should this should be the last function being executed.
This was done to allow `beforeRetry` hooks getting called.

-   You can no longer set `options.agent` to `false`.
    To do so, you need to define all the `options.agent` properties: `http`, `https` and `http2`.

```diff
await got('https://example.com', {
-    agent: false
+    agent: {
+        http: false,
+        https: false,
+        http2: false
+    }
})
```

-   When passing a `url` option when paginating, it now needs to be an absolute URL - the `prefixUrl` option is always reset from now on. The same when retrying in an `afterResponse` hook.

```diff
- return {url: '/location'};
+ return {url: new URL('/location', response.request.options.url)};
```

There was confusion around the `prefixUrl` option. It was counterintuitive if used with the Pagination API. For example, it worked fine if the server replied with a relative URL, but if it was an absolute URL then the `prefixUrl` would end up duplicated. In order to fix this, Got now requires an absolute URL - no `prefixUrl` will be applied.

-   `got.extend(…)` will throw when passing some options that don't accept undefined - undefined no longer retains the old value, as setting undefined explicitly may reset the option

##### Documentation

We have redesigned the documentation so it's easier to navigate and find exactly what you are looking for. We hope you like it ❤️

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNC43NC4yIiwidXBkYXRlZEluVmVyIjoiMzQuNzQuMiJ9-->

Reviewed-on: https://gitea.vylpes.xyz/RabbitLabs/random-bunny/pulls/66
Co-authored-by: Renovate Bot <renovate@vylpes.com>
Co-committed-by: Renovate Bot <renovate@vylpes.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Discussion Issues which may not have code impact
Projects
None yet
Development

No branches or pull requests