Blocked by cloudflare?

[dorei44]

I switched from node:http to undici but now I'm getting 403 on some sites that are protected by cloudflare when sending GET requests. Not sure what I'm missing. Any help would be greatly appreciated.

Example:

const undici = require("undici");
const https = require("node:https");

const url = new URL("https://google.com/");

const headers = {
  Host: url.hostname,
  "Upgrade-Insecure-Requests": "1",
  "User-Agent":
    "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36",
  Accept:
    "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
  "Sec-Fetch-Site": "same-site",
  "Sec-Fetch-Mode": "navigate",
  "Sec-Fetch-User": "?1",
  "Sec-Fetch-Dest": "document",
  "Accept-Encoding": "gzip, deflate, br",
  "Accept-Language": "en-US",
  "Sec-Ch-Ua":
    '"Not/A)Brand";v="99", "Google Chrome";v="115", "Chromium";v="115"',
  "Sec-Ch-Ua-Mobile": "?0",
  "Sec-Ch-Ua-Platform": '"Linux"',
};

https.get(
  url,
  {
    headers,
  },
  ({ statusCode }) => {
    console.log(`https: ${statusCode}`);
  }
);

undici
  .request(url, {
    headers,
  })
  .then(({ statusCode }) => {
    console.log(`undici: ${statusCode}`);
  });

Results:

https: 200
undici: 403

[metcoder95]

Hey! There are several moving parts that plays an important role here, and I sadly doubt that this has to do anything with undici.

Cloudflare returns 403 on several scenarios, depending on WAF configuration, IP rules and lastly the same origin server. Here you can find more info about that.

One part that seems particular here, is that you're trying to impersonate a browser, for http it works, but not the same for undici; and this might have to do with the way Undici establishes the connection compared to http.
If you remove the headers from the https version, you'll also receive a 403. That leads me to guess that is possible the request is failing Cloudflare's browsers integrity check, but that's just a guess

[toto6038]

I also encounter the issue when using Nuxt 3 in which undici is used in the underlying Nitro server.