Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: optimize getResolveErrorBodyCallback #2921

Merged
merged 3 commits into from
Mar 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion lib/api/api-request.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
'use strict'

const Readable = require('./readable')
const { Readable } = require('./readable')
const {
InvalidArgumentError,
RequestAbortedError
Expand Down
4 changes: 3 additions & 1 deletion lib/api/readable.js
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ const kContentType = Symbol('kContentType')

const noop = () => {}

module.exports = class BodyReadable extends Readable {
class BodyReadable extends Readable {
constructor ({
resume,
abort,
Expand Down Expand Up @@ -348,3 +348,5 @@ function consumeFinish (consume, err) {
consume.length = 0
consume.body = null
}

module.exports = { Readable: BodyReadable, chunksDecode }
14 changes: 8 additions & 6 deletions lib/api/util.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,20 @@ const assert = require('node:assert')
const {
ResponseStatusCodeError
} = require('../core/errors')
const { toUSVString } = require('../core/util')

const { chunksDecode } = require('./readable')
const CHUNK_LIMIT = 128 * 1024

async function getResolveErrorBodyCallback ({ callback, body, contentType, statusCode, statusMessage, headers }) {
assert(body)

let chunks = []
let limit = 0
let length = 0

for await (const chunk of body) {
chunks.push(chunk)
limit += chunk.length
if (limit > 128 * 1024) {
length += chunk.length
if (length > CHUNK_LIMIT) {
chunks = null
break
}
Expand All @@ -26,13 +28,13 @@ async function getResolveErrorBodyCallback ({ callback, body, contentType, statu

try {
if (contentType.startsWith('application/json')) {
const payload = JSON.parse(toUSVString(Buffer.concat(chunks)))
const payload = JSON.parse(chunksDecode(chunks, length))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is false. If I understand it correctly chunksDecode second parameter is about the amount of array items and not the total length of the whole Buffer.

Copy link
Member Author

@tsctx tsctx Mar 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. The length of the buffer.

undici/lib/api/readable.js

Lines 282 to 286 in aa86e89

function chunksDecode (chunks, length) {
if (chunks.length === 0 || length === 0) {
return ''
}
const buffer = chunks.length === 1 ? chunks[0] : Buffer.concat(chunks, length)

https://nodejs.org/docs/latest/api/buffer.html#static-method-bufferconcatlist-totallength

process.nextTick(callback, new ResponseStatusCodeError(`Response status code ${statusCode}${statusMessage ? `: ${statusMessage}` : ''}`, statusCode, headers, payload))
return
}

if (contentType.startsWith('text/')) {
const payload = toUSVString(Buffer.concat(chunks))
const payload = chunksDecode(chunks, length)
process.nextTick(callback, new ResponseStatusCodeError(`Response status code ${statusCode}${statusMessage ? `: ${statusMessage}` : ''}`, statusCode, headers, payload))
return
}
Expand Down
2 changes: 1 addition & 1 deletion test/readable.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

const { tspl } = require('@matteo.collina/tspl')
const { test, describe } = require('node:test')
const Readable = require('../lib/api/readable')
const { Readable } = require('../lib/api/readable')

describe('Readable', () => {
test('avoid body reordering', async function (t) {
Expand Down
Loading