Singleflight Concurrency Design Pattern In Golang

Published in

Level Up Coding

6 min read14 hours ago

In this article, I will share one of rarely used concurrency that we have in go. Start from the concept, advantage, disadvantage, and the usecases

Concept

singleflight (golang.org/x/sync/singleflight) is a concurrency method to prevent duplicate work from being executed due to multiple calls for the same resource. In another words, it provides a repeatable function call suppression mechanism by assigning a key to each function call. When functions with the same key are called concurrently, they will only be executed once and return the same result. Its essence is to reuse the results of function calls

Advantage

Reduced redundant work: It helps to preventing redundant call or resource-intensive operations for the same data。
Improving performance: By avoiding duplicate work, the system can handle concurrent requests more efficiently, leading to improved overall performance and reduced response times.
Prevention of thundering herd problem: It prevents the “thundering herd” problem, which occurs when multiple processes contend for the same resource simultaneously, leading to unnecessary resource contention and increased load on the system.
Enhanced scalability: By reducing unnecessary computations and database queries, the singleflight pattern contributes to better scalability, allowing the system to handle more concurrent requests without experiencing a significant performance degradation.

Disadvantage

Potential for increased memory usage: Storing and managing the state of ongoing requests may require additional memory, particularly if the system experiences a high volume of concurrent requests for the same resource. This can potentially impact the overall memory usage of the application.
Potential for delayed responses: In some cases, the singleflight pattern might introduce a slight delay in serving responses, especially when there are multiple concurrent requests for the same key. While this delay is usually minimal, it might impact real-time applications that require immediate responses.
Risk of stale data: Singleflight does not inherently handle data staleness. If the data is updated frequently or if it has a short expiration time, there is a risk that the cached data might become stale, leading to inconsistencies in the application. Care must be taken to ensure that the data retrieved through singleflight remains fresh and up to date.
Limited applicability: The singleflight pattern might not be suitable for all use cases. It is primarily beneficial for scenarios where the same data is requested frequently, and duplicate computations or queries need to be minimized. In situations where data freshness is critical or where data access patterns are highly variable, singleflight might not provide significant benefits.

Usecases

Basically, there are only 3 method for the single flight

// Do method, pass the key, and callback function. If the key is the same, the fn method will only be executed once and wait synchronously.
// Return value v: indicates the execution result of fn
// Return value err: represents the err returned by fn
// The third return value shared: indicates whether it is returned by the real fn or returned from the saved map[key], which mean that is shared
func (g *Group) Do(key string, fn func() (interface{}, error)) (v interface{}, err error, shared bool) {

// The DoChan method is similar to the Do method, but it returns a chan
func (g *Group) DoChan(key string, fn func() (interface{}, error)) <-chan Result {

// to control whether the value associated with the key is invalid. By default, as long as the fn method is executed, the internally maintained fn value will also be deleted (it will become invalid after the concurrency ends).
func (g *Group) Forget(key string) {

Also we can deep dive into the code singleflight/singleflight.go , it’s pretty straightforward. In the Do method, it is mainly controlled by waitgroup. The main process is as follows:

A map is being set in the Group. If the key does not exist, the call is instantiated (used to save the value information), and the corresponding relationship between key and call is stored in the map (mutex ensures concurrency safety)
If the key already exists in the map. it increments the dups counter in the corresponding call structure and unlocks the mutex. Then it waits for the call to finish executing using c.wg.Wait().
If the key doesn’t exist in the map, it creates a new call structure, adds 1 to the wait group with c.wg.Add(1), and sets the value of the key in the map m to this new call structure. Then it unlocks the mutex.
After unlocking the mutex, it calls the doCall method with the call structure, the key, and the function fn.
Finally, it returns the value and error from the call structure, along with a boolean indicating whether there were any duplicates during the execution.

To implement the singleflight concurrentcy, we can implement it using the Group as below example (Github Repo):


const (
 concurrentLimit = 10
)

var (
 errorNotExist = errors.New("not exist")
 key = "key"
 sfg = singleflight.Group{}
)

// Without Using Singleflight
func getDataWithNoSf(key string) (string, error) {
 data, err := getDataFromCache(key)
 if err == errorNotExist {
 data, err = getDataFromDB(key)
 if err != nil {
 log.Println(err)
 return "", err
 }

 } else if err != nil {
 return "", err
 }
 return data, nil
}

// Using Singleflight
func getDataWithSf(key string) (string, error) {
 data, err := getDataFromCache(key)
 if err == errorNotExist {
 v, err, _ := sfg.Do(key, func() (interface{}, error) { // The Difference Is In Here
 return getDataFromDB(key)
 })
 if err != nil {
 log.Println(err)
 return "", err
 }

 data = v.(string)
 } else if err != nil {
 return "", err
 }
 return data, nil
}

// Simulate retrieving value from the cache, where the cache doesn't have the value
func getDataFromCache(key string) (string, error) {
 return "", errorNotExist
}

// Simulate fetching data from the database
func getDataFromDB(key string) (string, error) {
 log.Printf("get %s from the database", key)
 return "sample result", nil
}


func simulateConcurrentProcessesWithNoSf() {
 var wg sync.WaitGroup
 wg.Add(concurrentLimit)

 for i := 0; i < concurrentLimit; i++ {
 go func() {
 defer wg.Done()
 data, err := getDataWithNoSf(key)
 if err != nil {
 log.Print(err)
 return
 }
 log.Println(data)
 }()
 }
 wg.Wait()
}

func simulateConcurrentProcessesWithSf() {
 var wg sync.WaitGroup
 wg.Add(concurrentLimit)

 for i := 0; i < concurrentLimit; i++ {
 go func() {
 defer wg.Done()
 data, err := getDataWithSf(key)
 if err != nil {
 log.Print(err)
 return
 }
 log.Println(data)
 }()
 }
 wg.Wait()
}

Benchmark:

BenchmarkSimulateConcurrentProcessesWithNoSf-12: This benchmark represents the performance without using the singleflight package. It took an average of 6350 nanoseconds per operation, with an average allocation of 176 bytes and 11 allocations per operation.

BenchmarkSimulateConcurrentProcessesWithSf-12: This benchmark represents the performance when using the singleflight package. It took an average of 12330 nanoseconds per operation, with an average allocation of 1028 bytes and 28 allocations per operation.

Logging:

From these results, we can conclude that using the singleflight package introduces a performance overhead, leading to increased time per operation, more memory allocation, and more allocations per operation compared to not using it. However, it's important to note that the singleflight package is beneficial in scenarios where multiple goroutines request the same resource simultaneously, preventing redundant work and improving overall performance in such scenarios.