UND_ERR_HEADERS_TIMEOUT received even when setting a larger headersTimeout in dispatcher for fetch()

[xconverge]

Bug Description

I am trying to make a fetch request that can take 5+ minutes.

I make a fetch() request with a custom Agent containing atleast these parameters (I have also overridden connect for overriding checkServerIdentity())

    headersTimeout: 3000000,
    bodyTimeout: 3000000,

fetch() call looks like this:

      fetch( url,  {
          dispatcher: this.apiParameters.httpAgent,
          method: "get",
        }

yet after about ~200 seconds I am getting a headers timeout

TypeError: fetch failed
    at fetch (/node_modules/undici/index.js:105:13)
    at location I called unidici fetch) {
  cause: HeadersTimeoutError: Headers Timeout Error
      at Timeout.onParserTimeout [as _onTimeout] (/node_modules/undici/lib/client.js:902:28)
      at listOnTimeout (node:internal/timers:566:11)
      at process.processTimers (node:internal/timers:507:7) {
    code: 'UND_ERR_HEADERS_TIMEOUT'
  }
}

Reproducible By

Performing a LONG fetch operation

Expected Behavior

I would expect some combination of me setting headerTimeout to either 0 or a very high number to not trigger this timeout.

Logs & Screenshots

Environment

Ubuntu 20 in WSL, Node 18

Additional context

I have seen some stuff related to "blocking" https://undici.nodejs.org/#/docs/api/Dispatcher but am unable to figure out where to set it and am unsure if its even related

[mcollina]

How is the agent instantiated?

[xconverge]

How is the agent instantiated?

I just got this sample working to reproduce the issue. Changing the delay at the top to less than 4 is fine, and greater than 5 minutes and it fails

I was able to do this same test with a Client and it seems to work fine, so something specific to fetch() I think

"use strict";

const { test } = require("tap");

const { Agent } = require("../../");
const { createServer } = require("http");
const FakeTimers = require("@sinonjs/fake-timers");

// When this is set to less than 4 it works, when set to 5 or greater it fails and returns a header error
const minutes = 6;

const msToDelay = 1000 * 60 * minutes;

test("Realllly long for a single fetch()", (t) => {
  t.plan(1);

  const clock = FakeTimers.install();
  t.teardown(clock.uninstall.bind(clock));

  const server = createServer((req, res) => {
    setTimeout(() => {
      res.end("hello");
    }, msToDelay);
    clock.tick(msToDelay + 1);
  });
  t.teardown(server.close.bind(server));

  server.listen(0, () => {
    fetch(`http://localhost:${server.address().port}`, {
      path: "/",
      method: "GET",
      dispatcher: new Agent({
        headersTimeout: 0,
        connectTimeout: 0,
        bodyTimeout: 0,
      }),
    })
      .then((response) => response.text())
      .then((response) => {
        t.equal("hello", response);
        t.end();
      })
      .catch((err) => {
        console.error(err);
        t.error(err);
      });

    clock.tick(msToDelay - 1);
  });
});

[xconverge]

versus this (just substituted fetch() and agent with a client) which runs "fine"

"use strict";

const { test } = require("tap");

const { Client } = require("../..");
const { createServer } = require("http");
const FakeTimers = require("@sinonjs/fake-timers");

// When this is set to less than 4 it works, when set to 5 or greater it fails and returns a header error
const minutes = 6;

const msToDelay = 1000 * 60 * minutes;

test("Realllly long for a single request", (t) => {
  t.plan(2);

  const clock = FakeTimers.install();
  t.teardown(clock.uninstall.bind(clock));

  const server = createServer((req, res) => {
    setTimeout(() => {
      res.end("hello");
    }, msToDelay);
    clock.tick(msToDelay + 1);
  });
  t.teardown(server.close.bind(server));

  server.listen(0, () => {
    const client = new Client(`http://localhost:${server.address().port}`, {
      headersTimeout: 0,
      connectTimeout: 0,
    });
    t.teardown(client.destroy.bind(client));

    client.request({ path: "/", method: "GET" }, (err, response) => {
      t.error(err);
      const bufs = [];
      response.body.on("data", (buf) => {
        bufs.push(buf);
      });
      response.body.on("end", () => {
        t.equal("hello", Buffer.concat(bufs).toString("utf8"));
      });
    });

    clock.tick(msToDelay - 1);
  });
});

[xconverge]

Also here is what I am testing with locally xconverge@de7f4c0

[xconverge]

PR opened for this.

[mcollina]

Thanks!