Golang concurrent fork/join mode

In the field of software development, there is an increasing demand for faster and more efficient data processing. Parallel computing techniques, such as the fork/join pattern, provide a powerful solution for utilizing multiple CPU cores to execute tasks concurrently, thus significantly reducing the execution time of large-scale computations. This article explores the implementation of the fork/join pattern in Go by breaking down an example that sums an array using concurrent goroutines.

Introduction to the Fork/Join Concurrency Model

The Fork/Join pattern is a parallel technique that involves breaking a task into smaller chunks, processing those chunks in parallel (forking), and then combining the results of those tasks into a final result (joining). This pattern is particularly useful when tasks are independent of each other and can be executed concurrently without affecting each other.

Simple example: summing an array concurrently

1. Initialization: The program initializes an integer array. The program also sets the number of parts the array should be divided into, with each part being processed by a separate program.

2. Concurrency settings:

2.1 Fork: Divide the array into specified parts and start a goroutine for each part to calculate the sum of the part.

2.2 Channel Communication: Each “program” will send the calculated sum back to the main process through the channel to ensure synchronous communication.

2.3 Join: When all Goroutines complete the calculation, the main process collects and summarizes these partial results to get the total.
2.4 Logging: Logging Throughout the process, the program prints information showing the partitioning of the array, the sum calculated by each worker, and the partial sums received.

Code Sample

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62

import (
 "fmt"
 "sync"
)

var (
 parts = 4
)

func main() {
 numbers := ()int{3, 1, 4, 1, 5, 9, 2, 6, 5, 3, 5} // Example array
 totalSum := concurrentSum(numbers, parts)         // Divide the array into 4 parts for summing
 fmt.Println("Total Sum:", totalSum)
}

// sumPart sums a part of the array and sends the result to a channel.
func sumPart(workerId int, nums ()int, result chan<- int, wg *sync.WaitGroup) {
 defer wg.Done() // Ensure the WaitGroup counter decrements on function completion.
 sum := 0
 for _, num := range nums {
  sum += num
 }
 fmt.Printf("Worker %d calculated sum: %d\n", workerId, sum)
 result <- sum // Send the partial sum to the result channel.
}

// concurrentSum takes an array and the number of parts to divide it into,
// then sums the array elements using concurrent goroutines.
func concurrentSum(numbers ()int, parts int) int {
 n := len(numbers)
 partSize := n / parts // Determine the size of each subarray
 fmt.Printf("Dividing the array of size %d into %d parts of size %d\n", n, 
  parts, partSize)
 results := make(chan int, parts) // Channel to collect results with a buffer size
 
var wg sync.WaitGroup
 // Fork step: Launch a goroutine for each part of the array
 for i := 0; i < parts; i++ {
  start := i * partSize
  end := start + partSize
  if i == parts-1 { // Ensure the last goroutine covers the remainder of the array
   end = n
  }
  wg.Add(1)
  go sumPart(i, numbers(start:end), results, &wg)
 }

 // Close the results channel once all goroutines are done
 go func() {
  wg.Wait()
  close(results)
 }()

 // Join step: Combine results
 totalSum := 0
 for sum := range results {
  fmt.Printf("Received partial sum: %d\n", sum)
  totalSum += sum
 }

 return totalSum
}

in conclusion

The above example demonstrates the efficiency of the fork/join pattern when using Go for concurrent programming. By assigning the task of summing the array to multiple workers, the program runs significantly faster on multi-core processors, demonstrating the power and simplicity of using Go for concurrent programming tasks. This pattern can also be applied to a variety of other computing problems.

————-The End————-

subscribe to my blog by scanning my public wechat account