TypeError: fetch failed #1271

minas90 · 2022-03-10T15:23:26Z

Bug Description

Fetch fails with some URLs (after redirect?).

Reproducible By

let res = await fetch('https://www.luzernerzeitung.ch/kultur/literatur-im-pool-ld.2260742');

Expected Behavior

It should get the URL without any exception. The same code works just fine with 'node-fetch'.

Logs & Screenshots

    at Object.processResponse (node:internal/deps/undici/undici:7081:34)
    at Fetch.fetchFinale (node:internal/deps/undici/undici:7411:21)
    at Fetch.mainFetch (node:internal/deps/undici/undici:7305:21)
    at processTicksAndRejections (node:internal/process/task_queues:96:5) {
  cause: Error: incorrect header check
      at Zlib.zlibOnError [as onerror] (node:zlib:189:17) {
    errno: -3,
    code: 'Z_DATA_ERROR'
  }
}

Environment

macOS 11.6.4, Node v17.6.0

Additional context

The provided URL redirects and adds ?reduced=true at the end of the URL:
https://www.luzernerzeitung.ch/kultur/literatur-im-pool-ld.2260742?reduced=true
When I run the same code with this redirected URL, it works fine.

I guess a little tweak with the gzip code or redirect logic should fix this.

The text was updated successfully, but these errors were encountered:

KhafraDev · 2022-03-11T18:08:24Z

The error originates at

undici/lib/fetch/index.js

Line 1868 in d94dc15

decoders.push(zlib.createGunzip())

because we get a content-encoding header that is gzip.

mcollina · 2022-03-11T20:35:36Z

It seems the actual content is not gzipped.

minas90 · 2022-03-12T04:13:47Z

Yeah, before redirect the content size is 142 bytes and the gzipped size is 2x more. So maybe the server has a check to send the original content if the size is smaller. I think fetch should not get the content before redirect, it's waste of recourses.

➜  ~ curl "https://www.luzernerzeitung.ch/kultur/literatur-im-pool-ld.2260742" --silent --write-out "%{size_download}\n" --output /dev/null                                         
142
➜  ~ curl "https://www.luzernerzeitung.ch/kultur/literatur-im-pool-ld.2260742" --silent -H "Accept-Encoding: gzip,deflate" --write-out "%{size_download}\n" --output /dev/null 
142

<html>
<head><title>302 Found</title></head>
<body>
<center><h1>302 Found</h1></center>
<hr><center>openresty</center>
</body>
</html>

After redirect it is gzipped, you can check with curl.

➜  ~ curl "https://www.luzernerzeitung.ch/kultur/literatur-im-pool-ld.2260742?reduced=true" --silent --write-out "%{size_download}\n" --output /dev/null
323859
➜  ~ curl "https://www.luzernerzeitung.ch/kultur/literatur-im-pool-ld.2260742?reduced=true" --silent -H "Accept-Encoding: gzip,deflate" --write-out "%{size_download}\n" --output /dev/null
66129

mcollina · 2022-03-12T10:07:40Z

Ah, I understand now. Basically we need to avoid parsing the body for the redirect. Would you like to send a PR?

ronag · 2022-03-12T10:31:50Z

I’m not sure it’s spec compliant to not read the body? Or is a body on a redirect even allowed?

targos · 2022-03-12T10:43:23Z

Chrome, Firefox and Safari all have no issue with the reproduction code and follow the redirection.
Example in Chrome:

minas90 · 2022-03-12T10:46:01Z

@mcollina I'm still not familiar with the codebase, if you can fix it quickly please go ahead. Otherwise I will push a PR in a few days when I will have some free time.

@ronag It seems like browsers ignore the body when status is 302. It's not even mandatory, maybe only needed for browsers which can't handle redirect?

https://stackoverflow.com/questions/8059662/http-302-redirect-is-a-message-body-needed

ronag · 2022-03-12T10:49:54Z

I think we should read the body but just ignore the content so that we can keep the connection alive.

minas90 · 2022-03-12T10:52:06Z

Btw, it's an unlikely scenario, but it's possible to have the same issue with gzip handling, when body size is small and the server has some logic to avoid gzip for small body. For example responses from API calls, such as simply ok

ronag · 2022-03-12T11:33:05Z

For non-fetch undici ignores the body. Not sure yet where it goes in fetch.

ronag · 2022-03-12T11:51:00Z

This is actually quite tricky to fix.

Don't start prefetching the body before headers and status has been processed. Fixes: #1271

Don't start decoding the body before headers and status has been processed. Fixes: #1271

ronag · 2022-03-12T13:40:46Z

Going back to the spec:

For e.g. HEAD and CONNECT the spec says:

If response is not a network error and either request’s method is
HEAD or CONNECT, or internalResponse’s status is a null body status,
set internalResponse’s body to null and disregard any enqueuing toward
it (if any).

But there is nothing along those lines for redirects. Is this a spec bug? Since Chrome does seem to disregard the body in thise case. @annevk

annevk · 2022-03-14T09:18:51Z

Redirects can have a response body. I don't think it ever ends up being exposed though.

* fix: lazy decode body Don't start decoding the body before headers and status has been processed. Fixes: #1271 * fixup * fixup * fixup * fixup * fixup * fixup * fixup * fixup * fixup * fixup * fixup

* fix: lazy decode body Don't start decoding the body before headers and status has been processed. Fixes: nodejs#1271 * fixup * fixup * fixup * fixup * fixup * fixup * fixup * fixup * fixup * fixup * fixup

minas90 added the bug Something isn't working label Mar 10, 2022

mcollina added the good first issue Good for newcomers label Mar 12, 2022

ronag closed this as completed in da9950b Mar 12, 2022

ronag reopened this Mar 12, 2022

ronag mentioned this issue Mar 12, 2022

fix: lazy decode body #1276

Merged

ronag added a commit that referenced this issue Mar 12, 2022

fix: lazy read body

0a5a738

Don't start prefetching the body before headers and status has been processed. Fixes: #1271

ronag added a commit that referenced this issue Mar 12, 2022

fix: lazy decode body

5d1e7ce

Don't start decoding the body before headers and status has been processed. Fixes: #1271

ronag added a commit that referenced this issue Mar 12, 2022

fix: lazy decode body

c914c56

Don't start decoding the body before headers and status has been processed. Fixes: #1271

ronag removed the good first issue Good for newcomers label Mar 12, 2022

ronag closed this as completed in #1276 Mar 19, 2022

jaschaio mentioned this issue Jan 8, 2024

Error: incorrect header check Z_DATA_ERROR #2603

Closed

jandppw mentioned this issue Apr 17, 2024

Observations regarding instability of native fetch in AWS Lambda x86_64 nodejs20.x #3133

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TypeError: fetch failed #1271

TypeError: fetch failed #1271

minas90 commented Mar 10, 2022 •

edited

Loading

KhafraDev commented Mar 11, 2022 •

edited

Loading

mcollina commented Mar 11, 2022

minas90 commented Mar 12, 2022 •

edited

Loading

mcollina commented Mar 12, 2022

ronag commented Mar 12, 2022 •

edited

Loading

targos commented Mar 12, 2022

minas90 commented Mar 12, 2022

ronag commented Mar 12, 2022

minas90 commented Mar 12, 2022

ronag commented Mar 12, 2022

ronag commented Mar 12, 2022

ronag commented Mar 12, 2022

annevk commented Mar 14, 2022

TypeError: fetch failed #1271

TypeError: fetch failed #1271

Comments

minas90 commented Mar 10, 2022 • edited Loading

Bug Description

Reproducible By

Expected Behavior

Logs & Screenshots

Environment

Additional context

KhafraDev commented Mar 11, 2022 • edited Loading

mcollina commented Mar 11, 2022

minas90 commented Mar 12, 2022 • edited Loading

mcollina commented Mar 12, 2022

ronag commented Mar 12, 2022 • edited Loading

targos commented Mar 12, 2022

minas90 commented Mar 12, 2022

ronag commented Mar 12, 2022

minas90 commented Mar 12, 2022

ronag commented Mar 12, 2022

ronag commented Mar 12, 2022

ronag commented Mar 12, 2022

annevk commented Mar 14, 2022

minas90 commented Mar 10, 2022 •

edited

Loading

KhafraDev commented Mar 11, 2022 •

edited

Loading

minas90 commented Mar 12, 2022 •

edited

Loading

ronag commented Mar 12, 2022 •

edited

Loading