Do not enforce newest URI rules on URLs #2200
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
szmarczak
changed the title
|
@szmarczak it would be nice if we could backport this fix into v11. |
Vylpes
pushed a commit
to Vylpes/random-bunny
that referenced
this pull request
Sep 5, 2023
This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [got](https://github.com/sindresorhus/got) | resolutions | major | [`^11.8.5` -> `^13.0.0`](https://renovatebot.com/diffs/npm/got/11.8.6/13.0.0) | --- ### Release Notes <details> <summary>sindresorhus/got</summary> ### [`v13.0.0`](https://github.com/sindresorhus/got/releases/tag/v13.0.0) [Compare Source](sindresorhus/got@v12.6.1...v13.0.0) As a reminder, Got continues to require ESM. For TypeScript users, this includes having [`"module": "node16", "moduleResolution": "node16"` in your tsconfig](https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c#how-can-i-make-my-typescript-project-output-esm). ##### Breaking - Require Node.js 16 [`52a1063`](sindresorhus/got@52a1063) - Change the [`enableUnixSockets`](https://github.com/sindresorhus/got/blob/main/documentation/2-options.md#enableunixsockets) option to be `false` by default [`852c312`](sindresorhus/got@852c312) - Most users don't need it. ##### Improvements - Allow specifying `undefined` for options ([#​2258](sindresorhus/got#2258)) [`1cefe8b`](sindresorhus/got@1cefe8b) ### [`v12.6.1`](https://github.com/sindresorhus/got/releases/tag/v12.6.1) [Compare Source](sindresorhus/got@v12.6.0...v12.6.1) - Fix `get-stream` import statement ([#​2266](sindresorhus/got#2266)) [`67d5039`](sindresorhus/got@67d5039) ### [`v12.6.0`](https://github.com/sindresorhus/got/releases/tag/v12.6.0) [Compare Source](sindresorhus/got@v12.5.3...v12.6.0) - Update dependencies [`88c88fb`](sindresorhus/got@88c88fb) [`979272e`](sindresorhus/got@979272e) - Loosen URL validation strictness ([#​2200](sindresorhus/got#2200)) [`0ca0b7f`](sindresorhus/got@0ca0b7f) ### [`v12.5.3`](https://github.com/sindresorhus/got/releases/tag/v12.5.3) [Compare Source](sindresorhus/got@v12.5.2...v12.5.3) - Fix abort event listeners not always being cleaned up ([#​2162](sindresorhus/got#2162)) [`3cc40b5`](sindresorhus/got@3cc40b5) ### [`v12.5.2`](https://github.com/sindresorhus/got/releases/tag/v12.5.2) [Compare Source](sindresorhus/got@v12.5.1...v12.5.2) - Improve TypeScript 4.9 compatibility ([#​2163](sindresorhus/got#2163)) [`39f83b6`](sindresorhus/got@39f83b6) ### [`v12.5.1`](https://github.com/sindresorhus/got/releases/tag/v12.5.1) [Compare Source](sindresorhus/got@v12.5.0...v12.5.1) - Fix compatibility with TypeScript and ESM [`3b3ea67`](sindresorhus/got@3b3ea67) - Fix request body not being properly cached ([#​2150](sindresorhus/got#2150)) [`3e9d3af`](sindresorhus/got@3e9d3af) ### [`v12.5.0`](https://github.com/sindresorhus/got/releases/tag/v12.5.0) [Compare Source](sindresorhus/got@v12.4.1...v12.5.0) - Disable method rewriting on 307 and 308 status codes ([#​2145](sindresorhus/got#2145)) [`e049e94`](sindresorhus/got@e049e94) - Upgrade dependencies [`8630815`](sindresorhus/got@8630815) [`f0ac0b3`](sindresorhus/got@f0ac0b3) [`4c3762a`](sindresorhus/got@4c3762a) ### [`v12.4.1`](https://github.com/sindresorhus/got/releases/tag/v12.4.1) [Compare Source](sindresorhus/got@v12.4.0...v12.4.1) ##### Fixes - Fix `options.context` being not extensible [`b671480`](sindresorhus/got@b671480) - Don't emit `uploadProgress` after promise cancelation [`693de21`](sindresorhus/got@693de21) ### [`v12.4.0`](https://github.com/sindresorhus/got/releases/tag/v12.4.0) [Compare Source](sindresorhus/got@v12.3.1...v12.4.0) ##### Improvements - Support FormData without known length ([#​2120](sindresorhus/got#2120)) [`850773c`](sindresorhus/got@850773c) ##### Fixes - Don't call `beforeError` hooks with `HTTPError` if the `throwHttpErrors` option is `false` ([#​2104](sindresorhus/got#2104)) [`3927348`](sindresorhus/got@3927348) ### [`v12.3.1`](https://github.com/sindresorhus/got/releases/tag/v12.3.1) [Compare Source](sindresorhus/got@v12.3.0...v12.3.1) - Don't freeze signal when freezing Options ([#​2100](sindresorhus/got#2100)) [`43b1467`](sindresorhus/got@43b1467) ### [`v12.3.0`](https://github.com/sindresorhus/got/releases/tag/v12.3.0) [Compare Source](sindresorhus/got@v12.2.0...v12.3.0) - Add `.off()` method for events ([#​2092](sindresorhus/got#2092)) [`88056be`](sindresorhus/got@88056be) ### [`v12.2.0`](https://github.com/sindresorhus/got/releases/tag/v12.2.0) [Compare Source](sindresorhus/got@v12.1.0...v12.2.0) - [Support `AbortController`](https://github.com/sindresorhus/got/blob/main/documentation/2-options.md#signal) ([#​2020](sindresorhus/got#2020)) [`6a6d2a9`](sindresorhus/got@6a6d2a9) - Add [`enableUnixSockets`](https://github.com/sindresorhus/got/blob/main/documentation/2-options.md#enableunixsockets) option ([#​2062](sindresorhus/got#2062)) [`461b3d4`](sindresorhus/got@461b3d4) ### [`v12.1.0`](https://github.com/sindresorhus/got/releases/tag/v12.1.0) [Compare Source](sindresorhus/got@v12.0.4...v12.1.0) ##### Improvements - Add `response.ok` ([#​2043](sindresorhus/got#2043)) [`22d58fb`](sindresorhus/got@22d58fb) - This is only useful if you have [`{throwHttpErrors: false}`](https://github.com/sindresorhus/got/blob/main/documentation/2-options.md#throwhttperrors) ##### Fixes - Do not redirect to UNIX sockets ([#​2047](sindresorhus/got#2047)) [`861ccd9`](sindresorhus/got@861ccd9) - [CVE-2022-33987](https://nvd.nist.gov/vuln/detail/CVE-2022-33987) - [Also back ported to v11](https://github.com/sindresorhus/got/releases/tag/v11.8.5) ### [`v12.0.4`](https://github.com/sindresorhus/got/releases/tag/v12.0.4) [Compare Source](sindresorhus/got@v12.0.3...v12.0.4) - Remove stream lock - unreliable since Node 17.3.0 [`bb8eca9`](sindresorhus/got@bb8eca9) ### [`v12.0.3`](https://github.com/sindresorhus/got/releases/tag/v12.0.3) [Compare Source](sindresorhus/got@v12.0.2...v12.0.3) - Allow more types in the `json` option ([#​2015](sindresorhus/got#2015)) [`eb045bf`](sindresorhus/got@eb045bf) ### [`v12.0.2`](https://github.com/sindresorhus/got/releases/tag/v12.0.2) [Compare Source](sindresorhus/got@v12.0.1...v12.0.2) - Fix `encoding` option with `{responseType: 'json'}` ([#​1996](sindresorhus/got#1996)) [`0703318`](sindresorhus/got@0703318) ### [`v12.0.1`](https://github.com/sindresorhus/got/releases/tag/v12.0.1) [Compare Source](sindresorhus/got@v12.0.0...v12.0.1) - Fix `nock` compatibility ([#​1959](sindresorhus/got#1959)) [`bf39d2c`](sindresorhus/got@bf39d2c) - Fix missing export of `Request` TypeScript type ([#​1940](sindresorhus/got#1940)) [`0f9f2b8`](sindresorhus/got@0f9f2b8) ### [`v12.0.0`](https://github.com/sindresorhus/got/releases/tag/v12.0.0) [Compare Source](sindresorhus/got@v11.8.6...v12.0.0) ##### Introducing Got v12.0.0 🎉 Long time no see! The latest Got version (v11.8.2) was released just in February ❄️ We have been working hard on squashing bugs and improving overall experience. If you find Got useful, you might want to [sponsor the Got maintainers](https://github.com/sindresorhus/got?sponsor=1). ##### This package is now pure ESM **Please [read this](https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c).** Also see sindresorhus/got#1789. - **Please don't open issues about `[ERR_REQUIRE_ESM]` and `Must use import to load ES Module` errors.** This is a problem with your setup, not Got. - Please don't open issues about using Got with Jest. Jest does not fully support ESM. - Pretty much any problem with loading this package is a problem with your bundler, test framework, etc, not Got. - If you use TypeScript, you will want to stay on Got v11 until TypeScript 4.6 is out. [Why.](microsoft/TypeScript#46452) - If you use a bundler, make sure it supports ESM and that you have correctly configured it for ESM. - The Got issue tracker is not a support channel for your favorite build/bundler tool. ##### Required Node.js >=14 While working with streams, we encountered more Node.js bugs that needed workarounds. In order to keep our code clean, we had to drop Node.js v12 as the code would get more messy. We strongly recommend that you update Node.js to **v14 LTS**. ##### HTTP/2 support Every Node.js release, the native `http2` module gets more stable. Unfortunately there are still some issues on the Node.js side, so we decided to keep HTTP/2 disabled for now. We may enable it by default in Got v13. It is still possible to turn it on via the `http2` option. To run HTTP/2 requests, it is required to use Node.js **v15.10** or above. ##### Bug fixes Woah, we possibly couldn't make a release if we didn't fix some bugs! - Do not throw on custom stack traces ([#​1491](sindresorhus/got#1491)) [`49c16ee`](sindresorhus/got@49c16ee) - Remove automatic `content-length` on ReadStream ([#​1510](sindresorhus/got#1510)) [`472b8ef`](sindresorhus/got@472b8ef) - Fix promise shortcuts in case of error status code ([#​1543](sindresorhus/got#1543)) [`ff918fb`](sindresorhus/got@ff918fb) [`1107cc6`](sindresorhus/got@1107cc6) - Invert the `methodRewriting` option [`51d88a0`](sindresorhus/got@51d88a0) - Fix `url` not being reused on retry in rare case ([#​1487](sindresorhus/got#1487)) [`462bc63`](sindresorhus/got@462bc63) - Fix hanging promise on HTTP/2 timeout ([#​1492](sindresorhus/got#1492)) [`a59fac4`](sindresorhus/got@a59fac4) - Prevent uncaught ParseErrors on initial successful response ([#​1527](sindresorhus/got#1527)) [`77df9c3`](sindresorhus/got@77df9c3) - Throw an error when retrying with consumed body ([#​1507](sindresorhus/got#1507)) [`62305d7`](sindresorhus/got@62305d7) - Fix a Node.js 16 bug that hangs Got streams [`06a2d3d`](sindresorhus/got@06a2d3d) - Fix default pagination handling for empty Link header ([#​1768](sindresorhus/got#1768)) [`1e1e506`](sindresorhus/got@1e1e506) - Fix incorrect `response.complete` when using cache [`9e15d88`](sindresorhus/got@9e15d88) - Fix `Cannot call end` error when `request` returns a `Writable` [`226cc39`](sindresorhus/got@226cc39) - Fix Request options not being reused on retry [`3c23eea`](sindresorhus/got@3c23eea) - Fix types being not compatible with CommonJS [`3c23eea`](sindresorhus/got@3c23eea) - Fix `got.paginate does not call init hooks` ([#​1574](sindresorhus/got#1574)) [`3c23eea`](sindresorhus/got@3c23eea) - Generate a new object when passing options to the native `https` module ([#​1567](sindresorhus/got#1567)) [`3c23eea`](sindresorhus/got@3c23eea) - Remove stream reuse check ([#​1803](sindresorhus/got#1803)) [`9ecc5ee`](sindresorhus/got@9ecc5ee) - Fix merging `searchParams` ([#​1814](sindresorhus/got#1814)) [`1018c20`](sindresorhus/got@1018c20) [`732e9bd`](sindresorhus/got@732e9bd) - Fix unhandled exception when lookup returns invalid IP early ([#​1737](sindresorhus/got#1737)) [`2453e5e`](sindresorhus/got@2453e5e) - Fix relative URLs when paginating [`439fb82`](sindresorhus/got@439fb82) - Require url to be an instance of URL when paginating ([#​1818](sindresorhus/got#1818)) [`eda69ff`](sindresorhus/got@eda69ff) - Fix `username` and `password` encoding in URL ([#​1169](sindresorhus/got#1169) [#​1317](sindresorhus/got#1317)) [`d65d0ca`](sindresorhus/got@d65d0ca) - Clone raw options [`1c4cefc`](sindresorhus/got@1c4cefc) - Fix invalid `afterResponse` return check [`cbc8902`](sindresorhus/got@cbc8902) - Fix `https.alpnProtocols` not having an effect [`e1099fb`](sindresorhus/got@e1099fb) ##### Improvements - Make the `context` option mergeable ([#​1459](sindresorhus/got#1459)) [`2b8ed1f`](sindresorhus/got@2b8ed1f) - Add generic argument to AfterResponseHook TypeScript type ([#​1589](sindresorhus/got#1589)) [`6fc04a9`](sindresorhus/got@6fc04a9) - Add read timeout ([#​1518](sindresorhus/got#1518)) [`e943672`](sindresorhus/got@e943672) *(blocked by nodejs/node#35923 - Improve the pagination API ([#​1644](sindresorhus/got#1644)) [`2675046`](sindresorhus/got@2675046) - Change the stackAllItems option to be false by default ([#​1645](sindresorhus/got#1645)) [`1120370`](sindresorhus/got@1120370) - Throw when afterResponse hook returns an invalid value [`4f21eb3`](sindresorhus/got@4f21eb3) - Add `retry.backoffLimit` option [`41c4136`](sindresorhus/got@41c4136) - Add `noise` retry option [`e830077`](sindresorhus/got@e830077) - Enable more HTTPS options [`83575d5`](sindresorhus/got@83575d5) [`fe723a0`](sindresorhus/got@fe723a0) (thanks [@​Giotino](https://github.com/Giotino)) - Define `error.code` [`f27e8d3`](sindresorhus/got@f27e8d3) - Set `options.url` even if some options are invalid [`8d6a680`](sindresorhus/got@8d6a680) - Improve memory usage when merging options [`2db5ec5`](sindresorhus/got@2db5ec5) - Support async generators as body [`854430f`](sindresorhus/got@854430f) [`3df52f3`](sindresorhus/got@3df52f3) - Add missing `once` types for Stream API [`3c23eea`](sindresorhus/got@3c23eea) - New error type: `RetryError` which always triggers a new retry when thrown [`3c23eea`](sindresorhus/got@3c23eea) - `error.options` is now enumerable [`3c23eea`](sindresorhus/got@3c23eea) - `defaults.handlers` don't need a default handler now [`3c23eea`](sindresorhus/got@3c23eea) - Add a parser for the `Link` header [`3c23eea`](sindresorhus/got@3c23eea) - General code improvements [`a5dd9aa`](sindresorhus/got@a5dd9aa) ##### Breaking changes ##### Improved option normalization - Got exports an `Option` class that is specifically designed to parse and validate Got options. It is made of setters and getters that provide fast normalization and more consistent behavior. When passing an option does not exist, Got will throw an error. In order to retrieve the options before the error, use `error.options`. ```js import got from 'got'; try { await got('https://httpbin.org/anything', { thisOptionDoesNotExist: true }); } catch (error) { console.error(error); console.error(error.options.url.href); // Unexpected option: thisOptionDoesNotExist // https://httpbin.org/anything } ``` - The `init` hook now accepts a second argument: `self`, which points to an `Options` instance. In order to define your own options, you have to move them to `options.context` in an [`init` hook](https://github.com/sindresorhus/got/blob/main/documentation/lets-make-a-plugin.md#authorization) or store them in `options.context` directly. - The `init` hooks are ran only when passing an options object explicitly. ```diff - await got('https://example.com'); // this will *not* trigger the init hooks + await got('https://example.com', {}); // this *will** trigger init hooks ``` - [`options.merge()`](2-options.md) replaced `got.mergeOptions` and `Request.normalizeArguments` ```diff - got.defaults.options = got.mergeOptions(got.defaults.options, {…}); + got.defaults.options.merge(…); ``` This fixes issues like [#​1450](sindresorhus/got#1450) - Legacy `Url` instances are not supported anymore. You need to use WHATWG URL instead. ```diff - await got(string, {port: 8443}); + const url = new URL(string); + url.port = 8443; + await got(url); ``` - No implicit timeout declaration. ```diff - await got('https://example.com', {timeout: 5000}) + await got('https://example.com', {timeout: {request: 5000}) ``` - No implicit retry declaration. ```diff - await got('https://example.com', {retry: 5}) + await got('https://example.com', {retry: {limit: 5}) ``` - `dnsLookupIpVersion` is now a number (4 or 6) or undefined ```diff - await got('https://example.com', {dnsLookupIpVersion: 'ipv4'}) + await got('https://example.com', {dnsLookupIpVersion: 4}) ``` - `redirectUrls` and `requestUrl` now give URL instances ```diff - request.requestUrl + request.requestUrl.origin + request.requestUrl.href + request.requestUrl.toString() ``` ```diff - request.redirectUrls[0] + request.redirectUrls[0].origin + request.redirectUrls[0].href + request.redirectUrls[0].toString() ``` - Renamed `request.aborted` to `request.isAborted` ```diff - request.aborted + request.isAborted ``` Reason: consistency with `options.isStream`. - Renamed the `lookup` option to `dnsLookup` ```diff - await got('https://example.com', {lookup: cacheable.lookup}) + await got('https://example.com', {dnsLookup: cacheable.lookup}) ``` - The `beforeRetry` hook now accepts only two arguments: `error` and `retryCount` ```diff await got('https://example.com', { hooks: { beforeRetry: [ - (options, error, retryCount) => { - console.log(options, error, retryCount); - } + (error, retryCount) => { + console.log(error.options, error, retryCount); + } ] } }) ``` The `options` argument has been removed, however it's still accessible via `error.options`. All modifications on `error.options` will be reflected in the next requests (no behavior change, same as with Got 11). - The `beforeRedirect` hook's first argument (options) is now a cloned instance of the Request options. This was done to make retrieving the original options possible: `plainResponse.request.options`. ```diff await got('http://szmarczak.com', { hooks: { beforeRedirect: [ (options, response) => { - console.log(options === response.request.options); //=> true [invalid! our original options were overriden] + console.log(options === response.request.options); //=> false [we can access the original options now] } ] } }) ``` - The `redirect` event now takes two arguments in this order: `updatedOptions` and `plainResponse`. ```diff - stream.on('redirect', (response, options) => …) + stream.on('redirect', (options, response) => …) ``` Reason: consistency with the `beforeRedirect` hook. - The `socketPath` option has been removed. Use the `unix:` protocol instead. ```diff - got('/containers/json', {socketPath: '/var/run/docker.sock'}) + got('unix:/var/run/docker.sock:/containers/json') + got('http://unix:/var/run/docker.sock:/containers/json') ``` - The `retryWithMergedOptions` function in an `afterResponse` hook no longer returns a `Promise`. It now throws `RetryError`, so this should this should be the last function being executed. This was done to allow `beforeRetry` hooks getting called. - You can no longer set `options.agent` to `false`. To do so, you need to define all the `options.agent` properties: `http`, `https` and `http2`. ```diff await got('https://example.com', { - agent: false + agent: { + http: false, + https: false, + http2: false + } }) ``` - When passing a `url` option when paginating, it now needs to be an absolute URL - the `prefixUrl` option is always reset from now on. The same when retrying in an `afterResponse` hook. ```diff - return {url: '/location'}; + return {url: new URL('/location', response.request.options.url)}; ``` There was confusion around the `prefixUrl` option. It was counterintuitive if used with the Pagination API. For example, it worked fine if the server replied with a relative URL, but if it was an absolute URL then the `prefixUrl` would end up duplicated. In order to fix this, Got now requires an absolute URL - no `prefixUrl` will be applied. - `got.extend(…)` will throw when passing some options that don't accept undefined - undefined no longer retains the old value, as setting undefined explicitly may reset the option ##### Documentation We have redesigned the documentation so it's easier to navigate and find exactly what you are looking for. We hope you like it ❤️ </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNC43NC4yIiwidXBkYXRlZEluVmVyIjoiMzQuNzQuMiJ9--> Reviewed-on: https://gitea.vylpes.xyz/RabbitLabs/random-bunny/pulls/66 Co-authored-by: Renovate Bot <renovate@vylpes.com> Co-committed-by: Renovate Bot <renovate@vylpes.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There are still sites in the wild that only support Latin1 encoded URLs, but GOT throws a URI malformed error even though the page can be retrieved. This PR removes the assertion made with
decodeURIto solve the problem.My question would be, why is
decodeURIused here for assertion?Samples:
Checklist