pprof

cortex goroutine
File: cortex
Type: goroutine
Time: Dec 1, 2019 at 1:48pm (UTC)
Showing nodes accounting for 314, 94.29% of 333 total
Showing top 80 nodes out of 300
cortex N1 runtime gopark 267 (80.18%) N2 runtime selectgo 0 of 67 (20.12%) N2->N1 66 N3 runtime goparkunlock 0 of 168 (50.45%) N3->N1 168 N4 cache NewMemcached func1 0 of 100 (30.03%) N6 runtime chanrecv2 0 of 121 (36.34%) N4->N6 99 N59 cache (*Memcached) fetch 0 of 7 (2.10%) N4->N59 1 N5 ingester (*Ingester) flushLoop 0 of 50 (15.02%) N24 ingester (*Ingester) flushChunks 0 of 26 (7.81%) N5->N24 26 N51 util (*PriorityQueue) Dequeue 1 (0.3%) of 24 (7.21%) N5->N51 24 N7 runtime chanrecv 0 of 139 (41.74%) N6->N7 121 N7->N3 139 N8 instrument CollectedRequest 0 of 18 (5.41%) N27 bufio (*Reader) ReadSlice 0 of 11 (3.30%) N8->N27 7 N58 cache (*Memcached) Fetch func1 0 of 7 (2.10%) N8->N58 7 N73 memcache (*Client) GetMulti 0 of 7 (2.10%) N8->N73 7 N9 bytes (*Buffer) ReadFrom 0 of 22 (6.61%) N10 net (*conn) Read 0 of 31 (9.31%) N9->N10 7 N39 bytes makeSlice func1 4 (1.20%) N9->N39 4 N54 io (*LimitedReader) Read 1 (0.3%) of 11 (3.30%) N9->N54 11 N79 poll (*pollDesc) waitRead 0 of 33 (9.91%) N10->N79 31 N11 grpc (*Server) serveStreams func1 1 1 (0.3%) of 36 (10.81%) N26 grpc (*Server) handleStream 0 of 35 (10.51%) N11->N26 35 N12 cache (*backgroundCache) writeBackLoop 0 of 20 (6.01%) N12->N2 10 N12->N8 10 N13 grpc recvAndDecompress 0 of 24 (7.21%) N18 io ReadFull 0 of 25 (7.51%) N13->N18 8 N68 flate (*decompressor) Reset 1 (0.3%) N13->N68 1 N14 transport (*loopyWriter) run 0 of 18 (5.41%) N14->N2 7 N19 runtime Gosched 11 (3.30%) N14->N19 11 N15 grpc (*Server) processUnaryRPC 0 of 32 (9.61%) N15->N13 21 N29 client _Ingester_Push_Handler 0 of 11 (3.30%) N15->N29 11 N16 chunk (*seriesStore) PutOne 0 of 23 (6.91%) N36 bigtable (*Table) ApplyBulk 0 of 15 (4.50%) N16->N36 15 N57 chunk (*seriesStore) calculateIndexEntries 0 of 7 (2.10%) N16->N57 7 N60 cache (*instrumentedCache) Fetch 0 of 7 (2.10%) N16->N60 1 N17 io ReadAtLeast 0 of 25 (7.51%) N21 bufio (*Reader) Read 0 of 17 (5.11%) N17->N21 17 N78 transport (*transportReader) Read 0 of 8 (2.40%) N17->N78 8 N18->N17 25 N20 runtime chanrecv1 0 of 18 (5.41%) N20->N7 18 N21->N10 9 N56 tls (*Conn) readRecordOrCCS 0 of 8 (2.40%) N21->N56 8 N22 chunk (*Fetcher) worker 0 of 16 (4.80%) N22->N6 16 N23 http2 (*Framer) ReadFrame 0 of 17 (5.11%) N23->N18 17 N24->N16 23 N74 chunk (*Chunk) Encode 0 of 3 (0.9%) N24->N74 3 N25 flate (*decompressor) huffSym 6 (1.80%) N26->N15 32 N75 go-grpc-middleware ChainStreamServer func1 1 0 of 2 (0.6%) N26->N75 2 N28 bufio (*Reader) fill 0 of 14 (4.20%) N27->N28 11 N28->N10 14 N40 client (*LabelAdapter) Unmarshal 4 (1.20%) N29->N40 4 N76 go-grpc-middleware ChainUnaryServer func1 1 0 of 7 (2.10%) N29->N76 7 N30 go-grpc newOpenTracingClientStream func2 0 of 10 (3.00%) N30->N2 10 N31 grpc (*Server) handleRawConn func1 0 of 10 (3.00%) N77 transport (*http2Server) HandleStreams 0 of 10 (3.00%) N31->N77 10 N32 grpc newClientStream func5 0 of 10 (3.00%) N32->N2 10 N33 transport (*http2Server) keepalive 0 of 10 (3.00%) N33->N2 10 N34 transport newHTTP2Server func2 0 of 10 (3.00%) N34->N14 10 N35 runtime makeslice 3 (0.9%) N37 bigtable (*Table) doApplyBulk 0 of 11 (3.30%) N36->N37 11 N64 sync (*Mutex) Lock 0 of 4 (1.20%) N36->N64 2 N67 v2 (*bigtableClient) MutateRows 0 of 3 (0.9%) N37->N67 3 N38 grpc (*clientStream) withRetry 0 of 9 (2.70%) N47 grpc (*csAttempt) recvMsg 0 of 8 (2.40%) N38->N47 8 N52 runtime newobject 2 (0.6%) N38->N52 1 N41 grpc (*addrConn) resetTransport 0 of 8 (2.40%) N41->N20 8 N42 transport (*http2Client) reader 0 of 8 (2.40%) N42->N23 8 N43 transport (*recvBufferReader) Read 2 (0.6%) of 7 (2.10%) N43->N2 5 N44 transport newHTTP2Client func3 0 of 8 (2.40%) N44->N14 8 N45 memcache (*Client) GetMulti func2 0 of 7 (2.10%) N49 memcache (*Client) withAddrRw 0 of 7 (2.10%) N45->N49 7 N46 ingester (*userStates) getOrCreateSeries 2 (0.6%) of 5 (1.50%) N61 ingester labelPairs equal 2 (0.6%) N46->N61 2 N47->N13 3 N48 flate (*decompressor) nextBlock 0 of 9 (2.70%) N53 flate (*decompressor) huffmanBlock 1 (0.3%) of 7 (2.10%) N48->N53 7 N65 flate (*huffmanDecoder) init 1 (0.3%) of 2 (0.6%) N48->N65 2 N49->N27 4 N49->N64 2 N50 ingester (*Ingester) append 0 of 7 (2.10%) N50->N46 5 N63 chunkenc (*xorAppender) writeVDelta 2 (0.6%) N50->N63 2 N51->N3 23 N53->N25 6 N72 gzip (*Reader) Read 0 of 10 (3.00%) N54->N72 10 N55 http (*conn) serve 0 of 3 (0.9%) N70 http HandlerFunc ServeHTTP 0 of 1 (0.3%) N55->N70 1 N71 bufio (*Reader) Peek 0 of 3 (0.9%) N55->N71 2 N56->N9 7 N69 tls (*halfConn) incSeq 1 (0.3%) N56->N69 1 N57->N35 1 N57->N60 6 N58->N59 6 N59->N8 7 N60->N8 7 N62 index intersect 2 (0.6%) N80 runtime semacquire1 0 of 6 (1.80%) N64->N80 4 N65->N35 1 N66 logutil (*MergeLogger) outputLoop 0 of 3 (0.9%) N66->N6 3 N67->N38 1 N67->N52 1 N71->N28 3 N72->N48 9 N73->N20 6 N74->N35 1 N75->N62 2 N76->N50 7 N77->N23 9 N78->N43 7 N79->N1 33 N80->N3 6