Improve cpu/time performance by reusing request goroutines. #781
Labels
component/loki
keepalive
An issue or PR that will be kept alive and never marked as stale.
type/enhancement
Something existing could be improved
During benchmark I realized that 35% of cpu time is spent increasing goroutines stack size see flamegraph below. The remaining 65% is gzip decompression.
This is happening because Loki query code is very deep in term of function calls, and because each request creates a new goroutine with an initial stacksize of 2KB, which means each goroutines has to growth by factor 2 (this is how go runtime works) then copy the previous stack in the new one. Once a request is done, the goroutine is thrown away and next request will create a new one, that will also have to grow
My suggestion here is to create a worker/goroutine pool to use for each query request so that we can recycle goroutines instead of throwing them away.
WDYT ? This seems to increase the speed by 35%. I already get rid of all possible recursions we used to have.
/cc @gouthamve @tomwilkie
The text was updated successfully, but these errors were encountered: