-
Notifications
You must be signed in to change notification settings - Fork 29.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
querystring: reduce memory usage #34179
Conversation
Hey, thanks for your contribution 🙏 What's the impact on performance? (you can run the querystring benchmark :]) |
Hello, thanks for your suggestion 🙏
From the benchmark in linux, it looks almost same performance 😄 |
If I read your changes right, it's now doing out-of-bounds array element reads and V8 used to penalize those. Maybe it no longer does or maybe it's insignificant here. |
It looks pretty significant so I am -0 on landing this. Interesting change + results :] |
Yes, I did. And I used a lot in my personal coding :)
Thanks for letting me know! I did not know that!
I agree with you. I do not want to land unrecommended one, too :) Again, thank you both for advice! |
@sapics One idea that might be worth exploring here is using an |
Thanks @addaleax for your suggestion! The suggestion looks much better! I fixed my commit, force-push it and reopen this PR. Now, total memory usage would decrease Furthermore, we can get better benchmark ;) confidence improvement accuracy (*) (**) (***)
querystring\\querystring-parse.js n=1000000 type='altspaces' 0.22 % ±1.64% ±2.19% ±2.86%
querystring\\querystring-parse.js n=1000000 type='encodefake' *** 4.00 % ±1.11% ±1.48% ±1.92%
querystring\\querystring-parse.js n=1000000 type='encodelast' *** 2.22 % ±1.14% ±1.52% ±1.99%
querystring\\querystring-parse.js n=1000000 type='encodemany' 0.27 % ±0.94% ±1.25% ±1.63%
querystring\\querystring-parse.js n=1000000 type='manyblankpairs' -0.08 % ±0.30% ±0.40% ±0.52%
querystring\\querystring-parse.js n=1000000 type='manypairs' -0.56 % ±1.18% ±1.57% ±2.05%
querystring\\querystring-parse.js n=1000000 type='multicharsep' ** -2.18 % ±1.33% ±1.78% ±2.33%
querystring\\querystring-parse.js n=1000000 type='multivalue' 0.22 % ±1.17% ±1.55% ±2.03%
querystring\\querystring-parse.js n=1000000 type='multivaluemany' ** 1.46 % ±0.87% ±1.16% ±1.51%
querystring\\querystring-parse.js n=1000000 type='noencode' 0.39 % ±1.03% ±1.37% ±1.78%
querystring\\querystring-stringify.js n=1000000 type='array' ** 3.65 % ±2.69% ±3.59% ±4.70%
querystring\\querystring-stringify.js n=1000000 type='encodelast' * 1.51 % ±1.47% ±1.97% ±2.59%
querystring\\querystring-stringify.js n=1000000 type='encodemany' 0.68 % ±0.99% ±1.32% ±1.72%
querystring\\querystring-stringify.js n=1000000 type='multiprimitives' -2.49 % ±4.81% ±6.47% ±8.57%
querystring\\querystring-stringify.js n=1000000 type='noencode' 0.69 % ±1.79% ±2.38% ±3.11%
querystring\\querystring-unescapebuffer.js n=10000000 input='%20%21%22%23%24%25%26%27%28%29%2A%2B%2C%2D%2E%2F%30%31%32%33%34%35%36%37' *** 4.58 % ±0.79% ±1.06% ±1.40%
querystring\\querystring-unescapebuffer.js n=10000000 input='there is nothing to unescape here' *** -1.35 % ±0.75% ±1.00% ±1.31%
querystring\\querystring-unescapebuffer.js n=10000000 input='there%20are%20several%20spaces%20that%20need%20to%20be%20unescaped' *** 5.65 % ±0.65% ±0.86% ±1.12%
querystring\\querystring-unescapebuffer.js n=10000000 input='there%2Qare%0-fake%escaped values in%%%%this%9Hstring' * 0.33 % ±0.33% ±0.44% ±0.58% |
fa58d14
to
5bd319e
Compare
Yes, “small integers” (signed integers up to 31 bits in length, aka “smi”s) take up only a single 32-bit value, and V8 does have a special array representation for smi arrays, as far as I remember. |
5bd319e
to
689a4cd
Compare
Commit Queue failed- Loading data for nodejs/node/pull/34179 ✔ Done loading data for nodejs/node/pull/34179 ----------------------------------- PR info ------------------------------------ Title querystring: reduce memory usage (#34179) ⚠ Could not retrieve the email or name of the PR author's from user's GitHub profile! Branch sapics:querystring/reduce-memory-usage -> nodejs:master Labels author ready, querystring Commits 1 - querystring: reduce memory usage by Int8Array Committers 1 - sapics PR-URL: https://github.com/nodejs/node/pull/34179 Reviewed-By: Anna Henningsen Reviewed-By: Denys Otrishko ------------------------------ Generated metadata ------------------------------ PR-URL: https://github.com/nodejs/node/pull/34179 Reviewed-By: Anna Henningsen Reviewed-By: Denys Otrishko -------------------------------------------------------------------------------- ✖ Last GitHub CI failed ℹ Last Full PR CI on 2020-11-08T12:24:00Z: https://ci.nodejs.org/job/node-test-pull-request/34198/ ℹ Last Benchmark CI on 2020-07-05T13:02:13Z: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/638/ - Querying data for job/node-test-pull-request/34198/ ✔ Build data downloaded - Querying failures of job/node-test-commit/42045/ ✔ Data downloaded ✖ 4 failure(s) on the last Jenkins CI run ℹ This PR was created on Fri, 03 Jul 2020 06:36:29 GMT ✔ Approvals: 2 ✔ - Anna Henningsen (@addaleax): https://github.com/nodejs/node/pull/34179#pullrequestreview-442670690 ✔ - Denys Otrishko (@lundibundi): https://github.com/nodejs/node/pull/34179#pullrequestreview-442675347 -------------------------------------------------------------------------------- ✔ Aborted `git node land` session in /home/runner/work/node/node/.ncu Commit Queue action: https://github.com/nodejs/node/actions/runs/352514303 |
@sapics Can you fix the linter issue please? You need to declare |
This issue/PR was marked as stalled, it will be automatically closed in 30 days. If it should remain open, please leave a comment explaining why it should remain open. |
689a4cd
to
c1bc95b
Compare
@aduh95 Thank you for noticing linter issue! I fix it and push the commit. |
This comment has been minimized.
This comment has been minimized.
Landed in 0dee498...00b5ee6 |
PR-URL: #34179 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Denys Otrishko <shishugi@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
PR-URL: #34179 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Denys Otrishko <shishugi@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
PR-URL: #34179 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Denys Otrishko <shishugi@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
PR-URL: #34179 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Denys Otrishko <shishugi@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
PR-URL: #34179 Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Denys Otrishko <shishugi@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
This PR will reduce the memory usage when program loads
querystring
orinternal/querystring
. (reduce about 8 Byte * (256 - 103) count = 1,224 Byte, for each)It is also possible to remove
isHexTable
by replacingif(isHexTable[code])
toif(unhexTable[code]>=0)
, which reduce 8 * 103 Byte. Currently, I did not replace that for simplicity. If you feel better to replace it, please notice me.To avoid number->boolean casting, I did not do following replacement as #34179 (comment)
I also change the conditional expression asbecause I feel that the name of variableisHexTable
looks boolean name.Checklist
make -j4 test
(UNIX), orvcbuild test
(Windows) passes