README.md

go-singleflightx

x/sync/singleflight but better

Features

This library is inspired by x/sync/singleflight but adds many features:

• 🧬 generics
• 🍱 batching: fetch multiple keys in a single callback, with in-flight deduplication
• 📭 nullable result
• 🍕 sharded groups

🚀 Install

go get github.com/samber/go-singleflightx

This library is v0 and follows SemVer strictly. No breaking changes will be made to exported APIs before v1.0.0.

💡 Doc

GoDoc: https://pkg.go.dev/github.com/samber/go-singleflightx

Examples

Here is an example of a user retrieval in a caching layer:

import "github.com/samber/go-singleflightx"

func getUsersByID(userIDs []string) (map[string]User, error) {
    users := []User{}

    // 📍 SQL query here...
    err := sqlx.Select(&users, "SELECT * FROM users WHERE id IN (?);", userIDs...)
    if err != nil {
        return nil, err
    }

    var results = map[string]User{}
    for _, u := range users {
        results[u.ID] = u
    }

    return results, nil
}

func main() {
    var g singleflightx.Group[string, User]

    // 👇 concurrent queries will be dedup
    output := g.DoX([]string{"user-1", "user-2"}, getUsersByID)
}

output is of type map[K]singleflightx.Result[V], and will always have as many entries as requested, whatever the callback result.

type Result[V any] struct {
  	 Value  singleflightx.NullValue[V]  // 💡 A result is considered "null" if the callback did not return it.
  	 Err    error
  	 Shared bool
}

type NullValue[V any] struct {
	Value V
	Valid bool
}

Sharded groups, for high contention/concurrency environments

g := singleflightx.NewShardedGroup[K string, User](10, func (key string) uint {
    h := fnv.New64a()
    h.Write([]byte(key))
    return uint(h.Sum64())
})

// as usual, but if the keys match different shards, getUsersByID will be called twice
output := g.DoX([]string{"user-1", "user-2"}, getUsersByID) 

go-singleflightx + go-batchify

go-batchify groups concurrent tasks into a single batch. By adding go-singleflightx, you will be able to dedupe

import (
    "golang.org/x/sync/singleflight"
    "github.com/samber/go-batchify"
)

var group singleflight.Group

batch := batchify.NewBatchWithTimer(
    10,
    func (ids []int) (map[int]string, error) {
        return ..., nil
    },
    5*time.Millisecond,
)

http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
    idStr := r.URL.Query().Get("id")
    id, _ := strconv.Atoi(idStr)

    value, err, _ = group.Do(idStr, func() (interface{}, error) {
        return batch.Do(id)
    })

    // ...
})

🤝 Contributing

• Ping me on Twitter @samuelberthe (DMs, mentions, whatever :))
• Fork the project
• Fix open issues or request new features

Don't hesitate ;)

# Install some dev dependencies
make tools

# Run tests
make test
# or
make watch-test

👤 Contributors

💫 Show your support

Give a ⭐️ if this project helped you!

📝 License

Copyright © 2023 Samuel Berthe.

This project is MIT licensed.