Large memory usage in Timeout::timeout

[asg0451]

I'm profiling my application with memory_profiler to try and reduce our memory usage. We don't have a leak, but the usage is still a bit higher than I'd like. One thing that stands out to me in the memory_profiler dump is this:

Total allocated: 2.12 GB (6186720 objects)
Total retained:  37.94 MB (19885 objects)

allocated memory by gem
-----------------------------------
   1.25 GB  timeout
...
allocated memory by file
-----------------------------------
   1.25 GB  /home/lfm/.rbenv/versions/2.3.3/lib/ruby/2.3.0/timeout.rb
...
allocated memory by location
-----------------------------------
   1.25 GB  /home/lfm/.rbenv/versions/2.3.3/lib/ruby/2.3.0/timeout.rb:81

That line corresponds to the Timeout::timeout method, excerpt:

 73   def timeout(sec, klass = nil)   #:yield: +sec+
     74     return yield(sec) if sec == nil or sec.zero?
     75     message = "execution expired".freeze
     76     from = "from #{caller_locations(1, 1)[0]}" if $DEBUG
     77     e = Error
     78     bl = proc do |exception|
     79       begin
     80         x = Thread.current
     81         y = Thread.start {
     82           Thread.current.name = from
     83           begin
     84             sleep sec
     85           rescue => e
     86             x.raise e
     87           else
     88             x.raise exception, message
     89           end
     90         }

This says one of two things to me -- either Timeout::timeout is really expensive and/or we're calling it way too much, or this is a limitation of the memory_profiler gem / red herring.

Can you confirm one way or the other?
I found this issue while Googling -- stompgem/stomp#121 -- which seems to point to this being a real thing. But I'm not entirely convinced by it

[SamSaffron]

I am guessing here, but my gut feel here without seeing more stuff is that you are using Timeout really heavily.

It is highly recommended you avoid using timeout where possible. It spawns a background thread that is responsible for killing stuff, @headius has written a lot about the problems of Timeout and wishes we could nuke it from Ruby. It is a very risky construct.

So ... my guess... each timeout you have is allocating a thread... each thread allocates a whole bunch of storage for thread local storage... this takes a while to GC.

[asg0451]

Thanks for the response @SamSaffron.
I suspect you are right. The application's job is to make approx. 1 zillion HTTP requests, and it uses the httpclient gem, which, turns out, uses Timeout::timeout for every request.

I'll explore some alternatives mentioned in the issue I linked. Thanks!
BTW do you happen to have a link to some of the stuff @headius has written that you mentioned?