What You Will Learn

• Why compilers and CPUs reorder memory operations in the first place
• The difference between a read barrier, a write barrier, and a full barrier
• Why the kernel splits barriers into SMP-only and device-facing variants
• How a real DMA descriptor write pattern uses a barrier to stay correct
• Modern, safer alternatives to raw barriers for many common cases

Prerequisites

• Basic C pointer and struct knowledge
• Some exposure to multi-core CPU concepts (caches, cores)
• Familiarity with basic device driver structure is helpful but not required

Why Reordering Happens

Both the compiler and the CPU are constantly looking for ways to execute your code faster without changing its observable single-threaded behavior. That freedom includes rearranging independent loads and stores, buffering writes in a store queue, and letting reads race ahead of writes to unrelated addresses. On a single core running one thread, none of this is visible. The moment a second core — or a hardware device sharing memory with the CPU — starts observing those same memory locations, reordering can expose a sequence of events that never matches what your source code said.

The Three Classic Barrier Types

The kernel’s <asm/barrier.h> header exposes three fundamental barrier primitives:

These “full” barriers order memory as seen by anything sharing the bus — including memory-mapped I/O devices — which makes them essential for driver code, but also relatively expensive.

SMP-Only Barriers: The Cheaper Cousins

Not every ordering problem involves a hardware device. When you only need to order memory as seen by other CPU cores — not by a peripheral — the kernel provides SMP-scoped equivalents that compile down to nothing at all on uniprocessor builds:

smp_mb();
smp_rmb();
smp_wmb();

On a multi-core build these still emit the necessary hardware ordering instruction, but they skip the extra guarantees needed for device I/O. In current kernel code, smp_* variants are the default choice for pure CPU-to-CPU synchronization; the plain mb()/rmb()/wmb() family is reserved specifically for CPU-to-device ordering, such as setting up a DMA descriptor before a device reads it.

A Realistic DMA Ordering Example

Imagine a driver preparing a descriptor for a network card to read via DMA. The descriptor has an address field and a “valid” flag. If the device sees the valid flag set before the address field is actually written to memory, it will read garbage. A write barrier prevents exactly that:

struct my_dma_desc {
    u64 buffer_addr;
    u32 flags;
};

static void prepare_descriptor(struct my_dma_desc *desc, u64 addr)
{
    desc->buffer_addr = addr;

    /* Ensure the address write reaches memory before
     * the device-visible "valid" flag is set. */
    wmb();

    desc->flags = DESC_FLAG_VALID;
}

This pattern — write the payload, insert a barrier, then write the flag that signals the payload is ready — is the single most common use of memory barriers you’ll encounter across kernel driver code.

Modern Alternatives You Should Know

Recent kernels increasingly favor higher-level primitives that bundle a barrier with the memory access itself, reducing the chance of misplacing one:

These don’t replace wmb()/rmb() for device-facing ordering, but for pure CPU-to-CPU synchronization they’re now generally preferred because the barrier can’t accidentally end up on the wrong side of the access it’s meant to protect.

Common Mistakes

• Using smp_wmb() where a device is involved — SMP-only barriers don’t guarantee ordering as seen by a peripheral; use the full wmb() instead.
• Assuming a lock automatically implies a full barrier everywhere it’s needed — spinlock/unlock do provide ordering guarantees, but for device-facing MMIO you still often need an explicit barrier.
• Placing the barrier on the wrong side — a write barrier must sit between the writes it’s ordering, not after both of them.
• Reaching for mb() everywhere “to be safe” — full barriers are expensive; use the narrowest one (rmb/wmb/smp_*) that actually matches your requirement.

Best Practices

• Always consult the datasheet or SoC peripheral manual for your specific hardware before relying on driver-level barrier placement alone.
• Prefer smp_store_release()/smp_load_acquire() pairs for CPU-only synchronization over raw barrier calls.
• Comment every barrier in driver code explaining exactly what ordering it enforces — future maintainers (including you) will thank you.

Performance and Security Considerations

Barriers are not free — a full mb() can stall the pipeline while outstanding memory operations drain, so using the narrowest barrier that satisfies your correctness requirement matters for real-time and high-throughput code paths. From a security perspective, missing barriers around published shared state are a classic source of subtle race-condition bugs, so their placement deserves the same scrutiny as locking logic during code review.

Summary / Key Takeaways

• Memory barriers stop the CPU and compiler from reordering operations across a point in your code where order actually matters.
• rmb()/wmb()/mb() are for CPU-to-device ordering; smp_* variants are for CPU-to-CPU ordering only.
• Modern code increasingly prefers acquire/release primitives that bundle the barrier with the access.

Conclusion

Memory barriers are one of those topics that feel abstract until you hit a bug that only reproduces on a multi-core system or with real hardware — at which point they become essential. Understanding the difference between device-facing and CPU-only barriers, and knowing when a higher-level primitive can do the job more safely, will save you from an entire category of intermittent, hard-to-reproduce driver bugs. This lesson is part of our ongoing free linux kernel development course covering synchronization from first principles through to production driver code.

Frequently Asked Questions