Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

11. Concurrency

A goroutine is a function running independently of the one that started it. It costs a keyword and a few kilobytes of stack that grows as needed, so a Go program can have thousands where it would have a handful of threads. That cheapness is the point: structuring a program as concurrent pieces becomes an ordinary decision rather than an architectural commitment.

Concurrency is not parallelism. Concurrency is a way of writing a program as independent activities; parallelism is those activities running at the same instant on different cores. Go gives you the first and lets the runtime decide about the second.

11.1 Goroutines

go f(x) calls f(x) in a new goroutine and returns immediately. The result is discarded, so a goroutine communicates by writing somewhere the rest of the program can see.

// Starting goroutines and waiting for them to finish.
package main

import (
	"fmt"
	"strings"
	"sync"
)

func main() {
	words := []string{"alpha", "beta", "gamma", "delta"}
	upper := make([]string, len(words))

	var wg sync.WaitGroup
	for i, w := range words {
		wg.Add(1)
		go func() {
			defer wg.Done()
			upper[i] = strings.ToUpper(w) // each goroutine owns one slot
		}()
	}
	wg.Wait()

	fmt.Println(upper)
}

[ALPHA BETA GAMMA DELTA]

A sync.WaitGroup counts outstanding work. Add before starting, Done when finished — deferred, so a panic still counts — and Wait blocks until the count reaches zero. Without the Wait, main would return and the program would exit with the goroutines half-run; nothing waits for a goroutine on your behalf.

Each goroutine here writes upper[i] for its own i and reads nothing another one writes, so there is no sharing and no lock. Writing to distinct elements of a slice from different goroutines is safe. Appending to one slice from several is not.

This program also depends on the per-iteration loop variable from section 3.2. Before Go 1.22 every goroutine would have seen the last i and w.

11.2 Channels

A channel is a typed pipe. ch <- v sends, <-ch receives, and both block until the other side is ready, making a channel a synchronization device and not just a queue.

// Sending, receiving, closing, and ranging.
package main

import "fmt"

// squares sends the squares of 1..n and closes the channel when done.
func squares(n int) <-chan int {
	out := make(chan int)
	go func() {
		defer close(out)
		for i := 1; i <= n; i++ {
			out <- i * i
		}
	}()
	return out
}

func main() {
	// Ranging over a channel ends when the channel is closed.
	for v := range squares(5) {
		fmt.Print(v, " ")
	}
	fmt.Println()

	// A buffered channel accepts sends until the buffer is full.
	buf := make(chan string, 2)
	buf <- "one"
	buf <- "two"
	close(buf)
	fmt.Println(len(buf), cap(buf))

	// Receiving from a closed channel yields the zero value at once.
	// The comma-ok form tells you which happened.
	for {
		v, ok := <-buf
		if !ok {
			fmt.Println("closed")
			break
		}
		fmt.Println("got", v)
	}
}

1 4 9 16 25
2 2
got one
got two
closed

make(chan int) is unbuffered: a send waits for a receiver and a receive waits for a sender, so the two goroutines meet at the moment of handoff. make(chan T, n) is buffered: sends succeed without a receiver until the buffer is full. Use unbuffered by default, because the blocking is usually the behavior you want, and reach for a buffer when you know the producer should run ahead.

close(ch) says no more values are coming. Receiving from a closed channel returns the zero value immediately, and v, ok := <-ch reports which happened. Ranging over a channel receives until it is closed, so squares closes with a defer.

Two rules about closing. The sender closes, never the receiver, because only the sender knows there is no more. And closing is optional — it exists to signal receivers, so a channel nobody ranges over need never be closed. Sending on a closed channel panics; closing twice panics.

Note the return type <-chan int. A channel type can be direction-limited, and squares returns a receive-only view, so a caller cannot send to it or close it. Use these in signatures; they document the direction and the compiler enforces it.

11.3 Select

select waits on several channel operations and proceeds with whichever is ready first, choosing at random among those ready together.

// select waits on several channel operations at once.
package main

import (
	"fmt"
	"time"
)

func main() {
	ticks := time.Tick(10 * time.Millisecond)
	done := time.After(35 * time.Millisecond)

	n := 0
	for {
		select {
		case <-ticks:
			n++
		case <-done:
			fmt.Println("ticks before the deadline:", n)
			return
		}
	}
}

ticks before the deadline: 3

A default case makes a select non-blocking: it runs when nothing else is ready. That turns a receive into a poll and a send into a drop-if-full, both of which are occasionally what you want and usually a sign of a design that should be blocking instead.

time.After returns a channel that delivers once, and that is how you put a timeout on any blocking operation. A nil channel blocks forever, so setting a channel variable to nil disables its case in a loop’s select — the standard way to stop listening to one source without restructuring the loop.

11.4 A Worker Pool

The shape you will reach for most: fan work out to a fixed number of goroutines and collect the results.

// A worker pool: fan work out to N goroutines, collect the results.
package main

import (
	"fmt"
	"slices"
	"sync"
)

func main() {
	const workers = 4

	jobs := make(chan int)
	results := make(chan int)

	var wg sync.WaitGroup
	for range workers {
		wg.Add(1)
		go func() {
			defer wg.Done()
			for n := range jobs {
				results <- n * n
			}
		}()
	}

	// Feed the jobs, then close so the workers' range loops end.
	go func() {
		defer close(jobs)
		for i := 1; i <= 10; i++ {
			jobs <- i
		}
	}()

	// Close results once every worker has stopped.
	go func() {
		wg.Wait()
		close(results)
	}()

	var got []int
	for r := range results {
		got = append(got, r)
	}

	slices.Sort(got) // arrival order is not defined
	fmt.Println(got)
}

[1 4 9 16 25 36 49 64 81 100]

Three goroutine roles and a closing rule for each. The workers range over jobs and stop when it is closed. The feeder closes jobs when it has sent everything. A separate goroutine waits for all the workers and then closes results, which it must, because no single worker knows when the last one has finished.

Getting that wrong is the standard concurrency bug in Go. Close results in a worker and the other workers panic sending to it; never close it and the collecting range blocks forever, a deadlock the runtime will report if nothing else is running:

fatal error: all goroutines are asleep - deadlock!

The results are sorted because arrival order depends on scheduling. If your output depends on which goroutine finished first, it is not deterministic, and a test asserting on it will pass on your machine and fail in CI.

Do not leak goroutines. A goroutine blocked on a channel nobody will ever use again never returns, and its stack is never collected. Every goroutine you start should have a clear reason it will end: a channel that gets closed, a context that gets canceled, or a loop with a bound.

11.5 Sharing Memory

Two goroutines touching the same variable, at least one of them writing, is a data race. It is not a slow path or a wrong ordering; it is undefined behavior, and the program may do anything.

// Two goroutines writing one variable. Run it with -race.
package main

import (
	"fmt"
	"sync"
)

func main() {
	var wg sync.WaitGroup
	counter := 0

	for range 1000 {
		wg.Add(1)
		go func() {
			defer wg.Done()
			counter++ // read, add, write: not atomic
		}()
	}
	wg.Wait()

	fmt.Println("counter:", counter)
}

Three runs of that program printed 860, 892, and 860. It should print 1000. counter++ is a read, an add, and a write, and goroutines interleave between them, so increments are lost.

Go ships a detector for this. go run -race and go test -race instrument memory access and report the conflict with both stacks:

WARNING: DATA RACE
Read at 0x00c00011c038 by goroutine 16:
  main.main.func1()
      ch11/race/main.go:17 +0x7b

Previous write at 0x00c00011c038 by goroutine 12:
  main.main.func1()
      ch11/race/main.go:17 +0x8d

It only reports races that actually occur during the run, so it finds nothing in code your tests do not exercise concurrently, but it has close to no false positives: if it reports a race, there is a race. Run your tests with -race in CI.

The fixes:

// The same counter, made safe two ways.
package main

import (
	"fmt"
	"sync"
	"sync/atomic"
)

func main() {
	var wg sync.WaitGroup

	var mu sync.Mutex
	guarded := 0

	var counted atomic.Int64

	for range 1000 {
		wg.Add(1)
		go func() {
			defer wg.Done()

			mu.Lock()
			guarded++
			mu.Unlock()

			counted.Add(1)
		}()
	}
	wg.Wait()

	fmt.Println(guarded, counted.Load())
}

1000 1000

sync.Mutex makes a section exclusive. Lock, touch the data, unlock — and defer mu.Unlock() when the section is longer than two lines or can return early. The mutex and the data it protects belong in the same struct, with the mutex declared immediately above them, so a reader can see the association. sync.RWMutex allows many concurrent readers and is worth it only when reads dominate and are slow.

The sync/atomic types handle a single value without a lock, and atomic.Int64 with its Add and Load methods is all most programs need.

Go’s advice is to prefer the channel: do not communicate by sharing memory; share memory by communicating. Give each piece of data one owner and pass it along a channel instead of locking it. That is genuinely better for pipelines and worker pools. It is worse for a shared counter or a cache, where a mutex is simpler and faster, and reaching for a channel to protect one integer is a mistake people make trying to be idiomatic.

11.6 Context

A context.Context carries a cancellation signal down a call tree, and it is how a Go program stops work it no longer needs.

ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()

select {
case res := <-work(ctx):
	use(res)
case <-ctx.Done():
	return ctx.Err() // context deadline exceeded
}

The convention is that a Context is the first parameter of any function that blocks, named ctx, and that it is passed down rather than stored in a struct. Every standard-library call that can wait accepts one, so a timeout at the top of a request propagates to the database query at the bottom without either layer knowing about the other.

Always call cancel, deferred, even when the operation finished. Not calling it leaks the context’s own goroutine and timer.

11.7 What To Reach For

Most concurrency in real programs is one of three shapes. A worker pool, as above, when you have many independent items and want a bounded number in flight. A pipeline of stages connected by channels, each stage a goroutine reading from one and writing to the next. And one goroutine per request in a server, which the net/http package already does for you, where your job is only to make what they share safe.

Reach for concurrency when the program waits on something — network, disk, other processes — or when the work is genuinely parallel and large enough to pay for the coordination. A concurrent version of a fast sequential loop is usually slower and always harder to read.

Exercise 11-1. Rewrite the goroutines program to send results on a channel instead of writing into a slice. Note what you have to do to keep the order.

Exercise 11-2. Add a -workers flag to a version of wf from Chapter 10 that counts several files concurrently, one goroutine per file, merging the maps at the end. Measure whether it is faster.

Exercise 11-3. Write a program that starts a goroutine blocked on a channel receive and then returns from main. Then make the same program deadlock, and read what the runtime prints.

Exercise 11-4. Take the race program and fix it with a channel instead of a mutex: one goroutine owns the counter and receives increment messages. Compare the two for length and for speed with -bench.