← Previous Lecture    Next Lecture →

Why the Kernel Needs 64-bit Atomic Counters

Most beginner kernel tutorials introduce atomic operations using the 32-bit atomic_t type, and for many use cases — reference counts, small flags, short-lived counters — a 32-bit value is more than enough. But modern systems generate numbers that overflow 32 bits surprisingly fast. A busy network interface can pass a few billion bytes an hour; a storage driver can track sector counts that climb into the billions over the uptime of a server; a tracing subsystem may need to count events across a session that runs for weeks.

A 32-bit unsigned counter wraps around at roughly 4.29 billion. Once you’re counting bytes, packets, or high-frequency events on modern hardware, that ceiling is reached in hours, not years. This is precisely the gap that 64-bit atomic integers close, and it’s why every actively maintained subsystem that tracks large or fast-growing values on 64-bit platforms leans on the 64-bit atomic API instead of quietly letting a 32-bit counter roll over.

atomic64_t vs atomic_t vs atomic_long_t

Before writing any code, it helps to know exactly which type solves which problem. Here is a practical comparison used throughout our free Linux device drivers course:

Modern kernel note: On present-day kernels, nearly every mainstream architecture Linux supports (x86-64, ARM64, RISC-V 64-bit) ships with native 64-bit atomic instructions, so atomic64_t is no longer the “special case” it once was on 32-bit-only hardware. That said, kernel code that still needs to build for 32-bit targets should prefer atomic_long_t or explicit conditional handling, since not every legacy 32-bit architecture has hardware support for atomic 64-bit read-modify-write instructions.

Declaring and Using atomic64_t Safely

The API mirrors the 32-bit version closely, which makes it easy to pick up if you already know atomic_t. Here is a simple, original example showing a driver-style byte counter:

#include <linux/atomic.h>

/* Declare and initialize a 64-bit atomic counter */
static atomic64_t bytes_transferred = ATOMIC64_INIT(0);

void record_bytes(u64 nbytes)
{
    /* Atomically add nbytes to the running total */
    atomic64_add(nbytes, &bytes_transferred);
}

u64 get_total_bytes(void)
{
    /* Atomically read the current total */
    return atomic64_read(&bytes_transferred);
}

void reset_counter(void)
{
    atomic64_set(&bytes_transferred, 0);
}

Notice the pattern: every operation that touches bytes_transferred goes through an atomic64_* function. This is the single most important rule with atomic types — never read or write the underlying variable directly with plain C operators such as += or a bare assignment, because doing so throws away the atomicity guarantee and reopens the door to race conditions.

Frequently Used atomic64_t Operations

Real-World Use Case: Tracking Statistics in a Network Driver

A very common place you’ll encounter atomic64_t in real kernel source is network and block drivers that maintain running statistics — total bytes sent, total bytes received, total errors — values exposed later through ethtool or /proc style interfaces. Multiple CPU cores can be transmitting or receiving packets on the same network interface at the same time, so the counter update has to be safe under concurrent access from several CPUs without a heavyweight lock. A per-interface atomic64_t counter, incremented on every packet, is a lightweight way to do this correctly.

How Atomic64 Operations Work Internally

Every atomic64 function is ultimately a thin wrapper that expands into an architecture-specific instruction sequence. On the inside, the kernel follows the classic three-step Read-Modify-Write pattern, except the CPU guarantees the three steps happen as one indivisible unit rather than three separate instructions that another core could interleave with.

Performance Considerations

Atomic operations are fast, but they are not free. Every atomic64 update touches a shared cache line, and if multiple CPU cores are hammering the same atomic64_t counter simultaneously, that cache line bounces back and forth between per-core caches — a phenomenon called cache-line bouncing or “cache ping-pong.” On high core-count systems this can become a real bottleneck for very hot counters.

A common modern mitigation is to switch from a single shared atomic64_t to per-CPU counters (using the kernel’s percpu infrastructure), where each CPU updates its own local copy without any cross-core synchronization at all, and the values are only summed together when someone actually needs to read the aggregate total. This trades a little complexity for a significant reduction in cache contention on hot paths.

Security Considerations

It’s tempting to think of a simple counter as harmless, but unsynchronized counters have historically been at the root of real security bugs — most notably reference-count races that lead to use-after-free conditions once a count is decremented incorrectly by two racing code paths. This is exactly why the kernel introduced the dedicated refcount_t type on top of the atomic infrastructure, adding overflow and underflow detection specifically for object lifetime counting. If your atomic64_t value controls object lifetime rather than pure statistics, prefer refcount_t over a raw atomic type.

Common Mistakes and Troubleshooting

• Mixing atomic and non-atomic access: Reading the variable with a plain C dereference anywhere in the code defeats the entire purpose of using atomic64_t.
• Assuming atomicity implies ordering: Atomicity guarantees the operation itself is indivisible; it does not by itself guarantee memory ordering with respect to other variables. Use the explicit memory-barrier variants (documented in Documentation/atomic_t.txt) when ordering with other data matters.
• Forgetting overflow at the application layer: atomic64_t practically never wraps around in the lifetime of a system, but if you cast the result down to a 32-bit value anywhere in your driver or a userspace-facing interface, you can silently reintroduce the original overflow problem.
• Using atomic64_t for reference counting: Prefer refcount_t, which is purpose-built with overflow protection for this exact scenario.

Best Practices

• Always initialize with ATOMIC64_INIT() or atomic64_set() before first use.
• Never mix atomic and non-atomic access to the same variable.
• Use per-CPU counters instead of a single shared atomic64_t for very hot, high-frequency counters.
• Use refcount_t instead of atomic64_t for object lifetime management.
• Consult the official kernel documentation on atomic operators whenever you’re unsure which variant of an operation to use.

Summary and Key Takeaways

• atomic64_t provides the same safety guarantees as atomic_t but for 64-bit values.
• Use it when a counter can realistically exceed 32-bit range — bytes, packets, long-running event counts.
• All access must go strictly through the atomic64_* API — no exceptions.
• On hot paths across many CPUs, consider per-CPU counters to avoid cache-line bouncing.
• For reference counting specifically, prefer refcount_t over raw atomic64_t.

Frequently Asked Questions

← Previous Lecture    Next Lecture →