What are atomic operations for newbies?

atomic operations

concurrency

multithreading

computer science

beginners guide

What are atomic operations for newbies?

Master System Design with Codemia

Enhance your system design skills with over 120 practice problems, detailed solutions, and hands-on exercises.

Start Practicing Learn More

Introduction

An atomic operation is one that completes entirely or not at all — no other thread can observe it in a partially completed state. The word "atomic" comes from Greek "atomos" (indivisible). In concurrent programming, atomicity prevents race conditions where two threads reading and writing the same data produce corrupted results. Common atomic operations include compare-and-swap (CAS), atomic increment/decrement, and atomic load/store. Most languages provide atomic primitives through standard libraries or language features.

The Problem Without Atomicity

python

1import threading
2
3counter = 0
4
5def increment():
6    global counter
7    for _ in range(100_000):
8        counter += 1  # NOT atomic!
9
10threads = [threading.Thread(target=increment) for _ in range(4)]
11for t in threads:
12    t.start()
13for t in threads:
14    t.join()
15
16print(counter)  # Expected: 400000, Actual: ~250000-350000 (race condition)

counter += 1 looks like one operation but is actually three: read the value, add 1, write the result back. Two threads can read the same value simultaneously, both add 1, and both write back the same result — losing one increment.

Fix with Locks

python

1import threading
2
3counter = 0
4lock = threading.Lock()
5
6def increment():
7    global counter
8    for _ in range(100_000):
9        with lock:  # Only one thread at a time
10            counter += 1
11
12threads = [threading.Thread(target=increment) for _ in range(4)]
13for t in threads:
14    t.start()
15for t in threads:
16    t.join()
17
18print(counter)  # Always 400000

A lock (mutex) ensures only one thread executes the critical section at a time. This is correct but slower than true atomic operations because of lock acquisition overhead.

Atomic Operations in Java

java

1import java.util.concurrent.atomic.AtomicInteger;
2
3public class AtomicExample {
4    // AtomicInteger provides atomic increment, decrement, compare-and-set
5    private static final AtomicInteger counter = new AtomicInteger(0);
6
7    public static void main(String[] args) throws InterruptedException {
8        Runnable task = () -> {
9            for (int i = 0; i < 100_000; i++) {
10                counter.incrementAndGet();  // Atomic — no lock needed
11            }
12        };
13
14        Thread[] threads = new Thread[4];
15        for (int i = 0; i < 4; i++) {
16            threads[i] = new Thread(task);
17            threads[i].start();
18        }
19        for (Thread t : threads) {
20            t.join();
21        }
22
23        System.out.println(counter.get());  // Always 400000
24    }
25}

AtomicInteger.incrementAndGet() uses hardware compare-and-swap (CAS) instructions, which are faster than locks because they don't require context switching or blocking.

Atomic Operations in C++

cpp

1#include <atomic>
2#include <thread>
3#include <vector>
4#include <iostream>
5
6std::atomic<int> counter{0};
7
8void increment() {
9    for (int i = 0; i < 100000; ++i) {
10        counter.fetch_add(1, std::memory_order_relaxed);
11    }
12}
13
14int main() {
15    std::vector<std::thread> threads;
16    for (int i = 0; i < 4; ++i) {
17        threads.emplace_back(increment);
18    }
19    for (auto& t : threads) {
20        t.join();
21    }
22    std::cout << counter.load() << std::endl;  // Always 400000
23    return 0;
24}

C++ std::atomic<T> provides fine-grained control over memory ordering. memory_order_relaxed is the fastest option when you only need atomicity without ordering guarantees relative to other variables.

Compare-and-Swap (CAS)

java

1// CAS is the fundamental building block of lock-free algorithms
2AtomicInteger value = new AtomicInteger(5);
3
4// compareAndSet(expected, new): only updates if current == expected
5boolean success = value.compareAndSet(5, 10);  // true, value is now 10
6boolean fail = value.compareAndSet(5, 20);     // false, value is still 10
7
8// CAS loop pattern — retry until successful
9public static void atomicMultiply(AtomicInteger ai, int factor) {
10    int oldValue;
11    int newValue;
12    do {
13        oldValue = ai.get();
14        newValue = oldValue * factor;
15    } while (!ai.compareAndSet(oldValue, newValue));
16}

CAS is the hardware primitive behind most atomic operations. It reads the current value, computes the new value, and writes it only if the current value hasn't changed since the read. If another thread modified it, CAS fails and the operation retries.

Go Atomic Operations

1package main
2
3import (
4    "fmt"
5    "sync"
6    "sync/atomic"
7)
8
9func main() {
10    var counter int64 = 0
11    var wg sync.WaitGroup
12
13    for i := 0; i < 4; i++ {
14        wg.Add(1)
15        go func() {
16            defer wg.Done()
17            for j := 0; j < 100000; j++ {
18                atomic.AddInt64(&counter, 1)
19            }
20        }()
21    }
22
23    wg.Wait()
24    fmt.Println(atomic.LoadInt64(&counter)) // Always 400000
25}

What Operations Are Naturally Atomic?

python

1# In CPython (with the GIL), these are effectively atomic:
2x = 42          # Storing a single value
3y = x           # Reading a single value
4my_list.append(item)  # List append
5
6# These are NOT atomic (even with the GIL):
7x += 1          # Read, compute, write
8my_dict[key] = value  # May trigger resize
9my_list[i] = x  # If i triggers IndexError path
10
11# IMPORTANT: Do not rely on GIL for correctness.
12# Other Python implementations (PyPy, Jython) may not have a GIL.

In most languages, reading or writing a single aligned word-sized value is atomic at the hardware level, but compound operations (read-modify-write) are never atomic without explicit synchronization.

Common Pitfalls

Assuming compound operations are atomic: counter++, counter += 1, and x = x + 1 are all read-modify-write sequences, not atomic. Even single-line expressions can involve multiple machine instructions.
Relying on language-specific guarantees: CPython's GIL makes some operations effectively atomic, but this is an implementation detail, not a language guarantee. Code that depends on the GIL breaks on other Python implementations.
ABA problem with CAS: CAS checks if the value is the same, but another thread might have changed it from A to B and back to A. CAS sees A and succeeds, missing the intermediate change. Use versioned references (AtomicStampedReference in Java) to detect this.
Memory ordering confusion: Atomic operations guarantee atomicity but not necessarily visibility across cores. In C++, memory_order_relaxed provides atomicity without ordering. Use memory_order_seq_cst (default) when you need operations to be visible in order.
Overusing atomics instead of higher-level abstractions: Atomic operations are building blocks for lock-free algorithms. For most application code, locks, concurrent collections, or channels are simpler and less error-prone.

Summary

Atomic operations complete entirely or not at all — no thread sees a partial state
The fundamental primitive is compare-and-swap (CAS), implemented in hardware
Java: AtomicInteger, AtomicLong, AtomicReference
C++: std::atomic<T> with memory ordering options
Go: sync/atomic package with AddInt64, LoadInt64, CompareAndSwapInt64
Python: use threading.Lock (no built-in atomic primitives outside the GIL)
Atomic operations are faster than locks but harder to reason about — prefer higher-level concurrency abstractions for application code