bilibili manga updates broken with undici 6.16.0

[yan12125]

Routes

/bilibili/manga/update/:comicid

Full routes

/bilibili/manga/update/32020

Related documentation

https://docs.rsshub.app/routes/social-media#%E6%BC%AB%E7%94%BB%E6%9B%B4%E6%96%B0

What is expected?

I can get the latest episode for a specific manga

What is actually happening?

The endpoint returns an error (see below)

Deployment information

RSSHub demo (https://rsshub.app)

Deployment information (for self-hosted)

No response

Additional info

FetchError: [POST] "https://manga.bilibili.com/twirp/comic.v2.Comic/ComicDetail?device=pc&platform=web": 400 Bad Request

This is not a duplicated issue

• I have searched existing issues to ensure this bug has not already been reported

[github-actions]

Searching for maintainers:

• /bilibili/manga/update/:comicid: @hoilc

To maintainers: if you are not willing to be disturbed, list your username in scripts/workflow/test-issue/call-maintainer.js. In this way, your username will be wrapped in an inline code block when tagged so you will not be notified.

If all routes can not be found, the issue will be closed automatically. Please use NOROUTE for a route-irrelevant issue or leave a comment if it is a mistake.
如果所有路由都无法匹配，issue 将会被自动关闭。如果 issue 和路由无关，请使用 NOROUTE 关键词，或者留下评论。我们会重新审核。

[yan12125]

I can reproduce HTTP 400 error with a self-hosted version. After some trials and errors, I noticed that a recent undici update breaks it. If I revert undici to 6.15.0, bilibili manga updates work fine.

I noticed there is a new HTTP request header with undici 6.16.0. Per mitmproxy, the ComicDetail request with undici 6.15.0 includes these headers:

host:             manga.bilibili.com
connection:       keep-alive
user-agent:       Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36   2.48
referer:          https://manga.bilibili.com/detail/mc32020
content-type:     application/json
accept:           application/json
accept-language:  *
sec-fetch-mode:   cors
accept-encoding:  br, gzip, deflate
content-length:   18

With undici 6.16.0, the request includes one more header:

host:             manga.bilibili.com
connection:       keep-alive
user-agent:       Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36
referer:          https://manga.bilibili.com/detail/mc32020
content-type:     application/json
accept:           application/json
accept-language:  *
origin:           client
sec-fetch-mode:   cors
accept-encoding:  br, gzip, deflate
content-length:   18

And the response complains about the origin header:

{
    "code": "invalid_argument",
    "meta": {
        "argument": "origin"
    },
    "msg": "origin client"
}

FWIW, I tried to override origin with:

diff --git a/lib/routes/bilibili/manga-update.ts b/lib/routes/bilibili/manga-update.ts
index 2b100dcea..6e0fc1457 100644
--- a/lib/routes/bilibili/manga-update.ts
+++ b/lib/routes/bilibili/manga-update.ts
@@ -36,6 +36,7 @@ async function handler(ctx) {
         },
         headers: {
             Referer: link,
+            Origin: 'https://manga.bilibili.com/',
         },
     });
     const data = response.data.data;

but the client string remains:

host:             manga.bilibili.com
connection:       keep-alive
user-agent:       Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.6261.112 Safari/537.36
origin:           https://manga.bilibili.com/, client
referer:          https://manga.bilibili.com/detail/mc32020
content-type:     application/json
accept:           application/json
accept-language:  *
sec-fetch-mode:   cors
accept-encoding:  br, gzip, deflate
content-length:   18

And bilibili still returns HTTP 400 about the invalid origin header.

[TonyRL]

Turns out it's this commit causing this issue which is introduced in nodejs/undici#3193.
Reverting only this commit make undici 6.16.0 works
Pinning undici to 6.15.0 for the time being.

---

Minimal repro:

import undici from "undici";

const res1 = await undici.request(
    "https://manga.bilibili.com/twirp/comic.v2.Comic/ComicDetail?device=pc&platform=web",
    {
        method: "POST",
        headers: {
            Accept: "*/*",
            'Content-Type': 'application/json',
        },
        body: JSON.stringify({
            comic_id: "32020",
        }),
    }
);

console.log(await res1.body.json());

const res2 = await undici.fetch(
    "https://manga.bilibili.com/twirp/comic.v2.Comic/ComicDetail?device=pc&platform=web",
    {
        method: "POST",
        headers: {
            Accept: "*/*",
            'Content-Type': 'application/json',
        },
        body: JSON.stringify({
            comic_id: "32020",
        }),
    }
);
console.log(await res2.json());
// Expected: both res1 and res2 work
// Actual: Only res1 works in 6.16.0

/cc: @gunjam, @KhafraDev and @tsctx

[yan12125]

Thank you for the fix!