Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

c: use proper type coercion in consume #44

Closed
wants to merge 1 commit into from

Conversation

indutny
Copy link
Member

@indutny indutny commented May 12, 2021

On 32bit platforms size_t is essentially uint32_t (or at times
even meager uint16_t). Loading uint64_t field value into size_t on
these platforms would truncate the high bits and leave only the low 32
(16) bits in place. This leads to various interesting errors in
downstream modules. See:

This patch makes all field loads go into their respective types.
Truncation doesn't happen in this case because C coercion rules will
cast both types to the largest necessary datatype to hold either of
them.

cc @Trott @mcollina @nodejs/http

I'd appreciate if we could fast-track this. We've been avoiding issues with this for awhile, but both wasm build in unidici and some arm users from the llhttp#110 hit this consistently on large uploads/downloads.

@indutny indutny requested review from mcollina and Trott May 12, 2021 06:29
@indutny
Copy link
Member Author

indutny commented May 12, 2021

cc @addaleax as well since you are so good at C/C++!

On 32bit platforms `size_t` is essentially `uint32_t` (or at times
even meager `uint16_t`). Loading `uint64_t` field value into `size_t` on
these platforms would truncate the high bits and leave only the low 32
(16) bits in place. This leads to various interesting errors in
downstream modules. See:

- nodejs/llhttp#110
- nodejs/undici#803

This patch makes all field loads go into their respective types.
Truncation doesn't happen in this case because C coercion rules will
cast both types to the largest necessary datatype to hold either of
them.
@indutny indutny force-pushed the fix/type-coercion-in-consume branch from d09bc9e to 0ea76ed Compare May 12, 2021 06:35
@targos
Copy link
Member

targos commented May 12, 2021

Does it affect 32-bit builds of Node.js too?

@indutny
Copy link
Member Author

indutny commented May 12, 2021

Does it affect 32-bit builds of Node.js too?

I believe it does.

Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@mcollina
Copy link
Member

+1 for fast-tracking this.

@indutny
Copy link
Member Author

indutny commented May 12, 2021

FWIW, here's how the generated code looks with this patch:

      size_t avail;
      uint64_t need;
      
      avail = endp - p;
      need = state->content_length;
      if (avail >= need) {
        p += need;
        state->content_length = 0;
        goto s_n_llhttp__internal__n_span_end_llhttp__on_body;
      }
      
      state->content_length -= avail;
      return s_n_llhttp__internal__n_consume_content_length;
      /* UNREACHABLE */;
      abort();

Note the uint64_t above. It was size_t for both variables before the patch.

indutny added a commit that referenced this pull request May 12, 2021
On 32bit platforms `size_t` is essentially `uint32_t` (or at times
even meager `uint16_t`). Loading `uint64_t` field value into `size_t` on
these platforms would truncate the high bits and leave only the low 32
(16) bits in place. This leads to various interesting errors in
downstream modules. See:

- nodejs/llhttp#110
- nodejs/undici#803

This patch makes all field loads go into their respective types.
Truncation doesn't happen in this case because C coercion rules will
cast both types to the largest necessary datatype to hold either of
them.

PR-URL: #44
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Daniele Belardi <dwon.dnl@gmail.com>
Reviewed-By: Robert Nagy <ronagy@icloud.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
@indutny
Copy link
Member Author

indutny commented May 12, 2021

Landed in 037dd2f. Thanks y'all!

@indutny indutny closed this May 12, 2021
@indutny indutny deleted the fix/type-coercion-in-consume branch May 12, 2021 18:32
@Chaphasilor
Copy link

Landed in 037dd2f. Thanks y'all!

Hey @indutny thanks for this fix! Could you share any info on how soon this will trickle down into actual Node, and on which versions it will work?
Or is it just a matter of rebuilding Node and it will use the latest version of llparse? :)

@indutny
Copy link
Member Author

indutny commented May 12, 2021

I'm still waiting for llhttp's CI run to complete, and will publish patch version releases after that. Will submit PRs to Node.js right after that. We should be able to have a discussion on the release timing there!

indutny added a commit to nodejs/node that referenced this pull request May 13, 2021
indutny added a commit to nodejs/node that referenced this pull request May 13, 2021
indutny added a commit to nodejs/node that referenced this pull request May 20, 2021
Fix: #37053
See: nodejs/llparse#44
PR-URL: #38665
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Daniele Belardi <dwon.dnl@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
danielleadams pushed a commit to nodejs/node that referenced this pull request May 31, 2021
Fix: #37053
See: nodejs/llparse#44
PR-URL: #38665
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Daniele Belardi <dwon.dnl@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants