rawkv: Context leak in BatchPut #696
Assignees
Comments
|
Another leak point was found in Codes to reproduce the issue: package main
import (
"context"
"fmt"
"log"
"net/http"
_ "net/http/pprof"
"github.com/tikv/client-go/v2/config"
"github.com/tikv/client-go/v2/rawkv"
"golang.org/x/sync/errgroup"
)
const THREAD_COUNT int = 100
const BATCH_SIZE int = 16
func main() {
cli, err := rawkv.NewClient(context.TODO(), []string{"127.0.0.1:4379"}, config.DefaultConfig().Security)
if err != nil {
panic(err)
}
defer cli.Close()
fmt.Printf("cluster ID: %d\n", cli.ClusterID())
ctx := context.TODO()
g, gctx := errgroup.WithContext(ctx)
g.Go(func() error {
log.Println(http.ListenAndServe("localhost:6060", nil))
return nil
})
for idx := 0; idx < THREAD_COUNT; idx += 1 {
keys := make([][]byte, 0, BATCH_SIZE)
values := make([][]byte, 0, BATCH_SIZE)
for i := 0; i < BATCH_SIZE; i += 1 {
keys = append(keys, []byte(fmt.Sprintf("%08d-%08d", idx, i)))
values = append(values, []byte("v"))
}
g.Go(func() error {
for {
if err := cli.BatchPut(gctx, keys, values); err != nil {
return err
}
if err := cli.BatchDelete(gctx, keys); err != nil {
return err
}
}
})
}
if err := g.Wait(); err != nil {
fmt.Printf("error: %v", err)
}
}Memory usage: Call graph: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Customers report that memory usage of TiKV-CDC keep increasing along with time, which using
BatchPutinterface to write data to downstream TiKV cluster.This issue seem to be caused by context leak of
BatchPutinterface.This following codes can reproduce this issue on master of
client-go:Memory usage of this program increased from hundreds of MBs to several GBs in minutes.
Top 3 of
pprofinuse heap memory:Callgraph of inuse heap memory size:
Callgraph of inuse heap objects count:
This issue seems to be caused by the leak of context here. When there is not error, the
cancel()will not be called, and thectxinbois leak.The text was updated successfully, but these errors were encountered: