Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use frozen string literals #144

Merged
merged 8 commits into from
Aug 16, 2023

Conversation

technicalpickles
Copy link
Contributor

I've been using memory_profiler on an extremely slow Rails system spec, and was surprised to see net/http came up in it. This particular test does an absolute ton of HTTP requests though.

I was looking through this code, and realized there's a lot of string literals used in some of the hot code paths, but frozen_string_literal had been explicitly set to false. There was only a few small changes after turning frozen_string_literal on to get the suite passing.

I tested my app against this, and it shaved off some 23s, from 2m15s to 1m52s 😱

I'll post before/after of the memory_profiler reports shortly.

@technicalpickles
Copy link
Contributor Author

Below is the relevant bits from a memory_profiler report. The tl;dr:

  • total allocated
    • memory: 10% reduction
    • objects: 7% reduction

Before

Total allocated: 5_363_399_946 bytes (45_328_676 objects)
Total retained: 180_774_831 bytes (1_244_196 objects)

allocated memory by file

96815592 /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/header.rb

allocated objects by file

1675464 /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/header.rb
330298 /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/response.rb
268506 /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/generic_request.rb

Allocated String Report

322618  "-"
 54114  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/header.rb:486

318394  " "
 16247  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/response.rb:175


187484  ""
 27071  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/response.rb:173

154335  ", "
 55180  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/header.rb:218
 54114  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/header.rb:479

148148  "\n"
 21659  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/response.rb:173

121761  "Content-Type"
 19476  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/header.rb:713
 19476  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/header.rb:728
  5411  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/response.rb:180
  5409  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/header.rb:486

 92393  "\r\n"
 43334  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/header.rb:270
 27058  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/header.rb:192
  5412  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/generic_request.rb:329
  5412  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/generic_request.rb:333
  5412  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/response.rb:173


 82734  "content-type"
 38952  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/header.rb:217
  5412  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/header.rb:273
  5411  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/header.rb:254
  5409  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/generic_request.rb:38
  5409  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/header.rb:195
  5409  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/header.rb:456

 80881  "Content-Length"
  5412  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/response.rb:180
  5410  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/header.rb:609
  5410  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/header.rb:610
  4329  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/header.rb:486

 71402  ";"
 19476  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/header.rb:713
 19476  /Users/josh.nichols/.rbenv/versions/3.2.2/lib/ruby/3.2.0/net/http/header.rb:728

After

Total allocated: 4_795_299_370 bytes (42_124_716 objects)
Total retained: 169_143_871 bytes (1_109_228 objects)

allocated memory by file

68355240 /Users/josh.nichols/workspace/net-http/lib/net/http/header.rb

allocated objects by file

1038079 /Users/josh.nichols/workspace/net-http/lib/net/http/header.rb
191421 /Users/josh.nichols/workspace/net-http/lib/net/http/response.rb

Allocated String Report

168326  "-"
        (nothing from net-http)

320917  " "
        (nothing from net-http)

138990  ""
  4251  /Users/josh.nichols/workspace/net-http/lib/net/http/response.rb:173


154335  ", "
        (nothing from net-http)

128933  "\n"
        (nothing from net-http)

 42489  "Content-Type"
  4250  /Users/josh.nichols/workspace/net-http/lib/net/http/response.rb:180
  4248  /Users/josh.nichols/workspace/net-http/lib/net/http/header.rb:494


     0  "\r\n"
        (nothing from net-http)

 65232  "content-type"
 30660  /Users/josh.nichols/workspace/net-http/lib/net/http/header.rb:225
  4251  /Users/josh.nichols/workspace/net-http/lib/net/http/header.rb:281
  4250  /Users/josh.nichols/workspace/net-http/lib/net/http/header.rb:262
  4248  /Users/josh.nichols/workspace/net-http/lib/net/http/generic_request.rb:42
  4248  /Users/josh.nichols/workspace/net-http/lib/net/http/header.rb:203
  4248  /Users/josh.nichols/workspace/net-http/lib/net/http/header.rb:464

 37507  "Content-Length"
  4251  /Users/josh.nichols/workspace/net-http/lib/net/http/response.rb:180
  3417  /Users/josh.nichols/workspace/net-http/lib/net/http/header.rb:494


     0  ";"
        (nothing from net-http)

lib/net/http.rb Outdated Show resolved Hide resolved
technicalpickles and others added 2 commits August 15, 2023 13:30
@natematykiewicz's suggestion to avoid a string allocation

Co-authored-by: Nate Matykiewicz <natematykiewicz@gmail.com>
@natematykiewicz's suggestion to avoid another string allocation

Co-authored-by: Nate Matykiewicz <natematykiewicz@gmail.com>
@yuki24 yuki24 merged commit 328d28d into ruby:master Aug 16, 2023
12 checks passed
@yuki24
Copy link
Member

yuki24 commented Aug 16, 2023

Thank you!

@natematykiewicz
Copy link
Contributor

Thanks for marking me as a co-author @technicalpickles!

@hsbt hsbt mentioned this pull request Aug 31, 2023
0x1eef added a commit to 0x1eef/net-http that referenced this pull request Sep 15, 2023
@technicalpickles technicalpickles deleted the frozen-string-literal branch October 5, 2023 20:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants