Picture a single global counter that every CPU core increments on every network packet or every scheduler tick. On an 8-core or 64-core server, that single memory location becomes a hot cache line: every core keeps invalidating every other core’s cached copy, and if you add a lock to make it safe, cores start queuing up behind each other. The counter that was supposed to be cheap becomes the most expensive line in your code path.

The kernel’s answer is to give each CPU its own private instance of the variable. Core 0 updates its own copy, core 1 updates its own copy, and so on — there is no shared cache line to fight over and, crucially, no lock is needed at all, because no other core can ever touch another core’s copy. This is the core idea behind Linux kernel per-CPU variables, and it is used everywhere performance matters: networking statistics, memory allocator caches, RCU bookkeeping, and the pointer to the currently running task on each core.

The simplest way to declare a per-CPU variable is at compile time, using DEFINE_PER_CPU(). The kernel build system reserves one instance of the variable for every possible CPU on the system, laid out so each core’s copy sits in that core’s own per-CPU memory area.

#include <linux/percpu.h>

/* One private 'long' counter per CPU, zero-initialized */
DEFINE_PER_CPU(long, pkt_drop_count);

/* Declared in a header when another file needs to reach it */
DECLARE_PER_CPU(long, pkt_drop_count);

Reading or writing this variable directly by name would be meaningless — pkt_drop_count is really an offset into a per-CPU memory block, not a normal address. The kernel gives you dedicated accessor macros instead, and this is where a lot of confusion shows up for newcomers, because there are two generations of accessors in the tree today.

The Older Style: get_cpu_var() / put_cpu_var()

This pattern disables kernel preemption, hands you a reference to the current CPU’s copy, and re-enables preemption once you call the matching “put”:

unsigned long flags_unused;
long val;

val = ++get_cpu_var(pkt_drop_count);
pr_info("drops on this cpu: %ld\n", val);
put_cpu_var(pkt_drop_count);

It works, but disabling preemption for the whole critical section is heavier than it needs to be for a simple counter bump, and it is easy to forget the matching put_cpu_var() if a function has multiple return paths.

The Modern, Preferred Style: this_cpu_* Operations

Current kernels favour the this_cpu_* family — this_cpu_read(), this_cpu_write(), this_cpu_add(), this_cpu_inc(), this_cpu_dec(), and similar. These map to single, architecture-specific instructions on x86 and arm64 that update the per-CPU copy safely without needing to disable preemption at all in most cases, because the underlying instruction is inherently a single, uninterruptible operation on that CPU’s data:

this_cpu_inc(pkt_drop_count);

long snapshot = this_cpu_read(pkt_drop_count);
pr_info("drops on this cpu: %ld\n", snapshot);

The practical guidance for anyone writing new kernel or driver code today is simple: reach for this_cpu_* first, and only fall back to get_cpu_var()/put_cpu_var() when you specifically need the preemption-disabled window that comes with it (for example, while also touching a related per-CPU structure that must stay on the same core for several steps).

Static DEFINE_PER_CPU() variables are fixed at compile time, which is fine for a handful of globals but does not work when a driver needs a per-CPU structure only for as long as it is loaded, or needs one instance per device rather than per boot. For that, the kernel provides alloc_percpu():

struct link_stats {
    unsigned int tx, rx;
};

static struct link_stats __percpu *stats;

static int __init stats_init(void)
{
    stats = alloc_percpu(struct link_stats);
    if (!stats)
        return -ENOMEM;
    return 0;
}

static void __exit stats_exit(void)
{
    free_percpu(stats);
}

To touch the current CPU’s instance of a dynamically allocated per-CPU structure, use get_cpu_ptr() and release it with put_cpu_ptr() — the pair disables preemption for the duration, just like get_cpu_var() does for a simple integer:

struct link_stats *s = get_cpu_ptr(stats);
s->tx += 1;
put_cpu_ptr(stats);

If you are writing a real device driver rather than a standalone experiment, prefer the resource-managed devm_alloc_percpu() over the plain alloc_percpu(). It takes your driver’s struct device * and ties the per-CPU memory’s lifetime to the device, so the kernel frees it automatically on driver unbind — one less manual cleanup path for you to get wrong:

stats = devm_alloc_percpu(dev, struct link_stats);
if (!stats)
    return -ENOMEM;
/* no explicit free_percpu() needed — devres handles it */

A common companion task when experimenting with per-CPU data is pinning a kernel thread to a specific core, so you can prove that two threads updating “the same” per-CPU variable on two different cores never collide. Older kernel programming material sometimes reaches for a well-known workaround here: looking up the address of the unexported sched_setaffinity() function with kallsyms_lookup_name() and calling it through a raw function pointer.

That trick no longer works on current kernels. kallsyms_lookup_name() itself was removed from the exported symbol table starting with kernel 5.7, precisely to close off this kind of unauthorized access to non-exported internals. Any tutorial or old blog post that still relies on it will fail to build a module against a modern kernel.

The good news is that the proper, exported APIs are more than enough for this use case and involve no hacks at all:

#include <linux/kthread.h>
#include <linux/sched.h>

struct task_struct *t;

/* Bind before the thread ever runs */
t = kthread_create(my_thread_fn, NULL, "pcpu_demo");
if (!IS_ERR(t)) {
    kthread_bind(t, 0);   /* pin to CPU 0 */
    wake_up_process(t);
}

/* Or change affinity of an already-running kernel thread */
set_cpus_allowed_ptr(t, cpumask_of(1));  /* move to CPU 1 */

kthread_bind() is meant for a thread that has not started running yet, while set_cpus_allowed_ptr() is the exported, GPL-safe way to move a thread that is already running — both are standard, documented kernel APIs, not workarounds.

Once you know what to look for, per-CPU data is everywhere in the kernel source tree:

The common thread in every one of these is the same design goal covered above: keep the hot path lock-free by giving every core its own copy, and only pay a coordination cost on the rare, cold path where a combined value is actually needed.

• Treating the per-CPU symbol as a normal pointer. Dereferencing &pkt_drop_count directly instead of going through an accessor macro will not do what you expect — it is an offset, not a live address.
• Forgetting that preemption re-enables between get and put. If you migrate to another CPU between two separate get_cpu_var() calls without holding the section open, you may end up touching two different cores’ copies without realizing it.
• Assuming a per-CPU counter gives you an instantaneous global total. A snapshot summed across cores while other cores are still updating their own copies is only ever an approximation, which is fine for statistics but not for correctness-critical counts.
• Using alloc_percpu() in a driver and hand-rolling cleanup. Prefer devm_alloc_percpu() where a struct device is available, so unbind and error paths cannot leak the allocation.
• Relying on kallsyms_lookup_name() to reach unexported symbols. This is unsupported and, as covered above, has been unexported since kernel 5.7 — it will not build on current kernels.

• Default to this_cpu_* operations for simple reads, writes, and increments; reserve get_cpu_var()/put_cpu_var() for cases that genuinely need preemption held off across several steps.
• Use devm_alloc_percpu() instead of raw alloc_percpu() whenever you are inside a proper driver probe function with a struct device * available.
• Keep the code between a “get” and its matching “put” as short as possible — you are holding off preemption on that core for the duration.
• Bind kernel threads to CPUs using kthread_bind() at creation time, or set_cpus_allowed_ptr() afterward — never through symbol-lookup workarounds.
• When you need a global total from per-CPU counters, sum them with for_each_possible_cpu() only on the cold reporting path, not on every hot-path update.

Performance: the entire point of per-CPU variables is avoiding cache-line bouncing between cores. Grouping unrelated fields into the same per-CPU structure can accidentally reintroduce false sharing between fields that different code paths update at different rates, so keep hot fields that are updated together close, and separate fields with very different access patterns into different per-CPU allocations where it matters.

Security: the retirement of kallsyms_lookup_name() as an exported symbol is itself a security-driven change — it closes a well-known path that let modules reach into arbitrary non-exported kernel internals, including internals with per-CPU data, bypassing the normal, audited export mechanism. Writing kernel code that only uses documented, exported per-CPU APIs keeps your driver both portable across kernel versions and free of this class of ABI risk.

• Per-CPU variables give each core its own private copy of a variable, removing the need for locking on the hot path.
• Static per-CPU data is declared with DEFINE_PER_CPU(); dynamic per-CPU data is allocated with alloc_percpu() or, inside a driver, devm_alloc_percpu().
• Modern kernel code should prefer this_cpu_* operations over the older get_cpu_var()/put_cpu_var() pattern for simple accesses.
• Kernel thread CPU affinity should be set with kthread_bind() or set_cpus_allowed_ptr() — not with unexported-symbol hacks, which no longer build on current kernels.

Linux kernel per-CPU variables are one of the simplest and most effective concurrency tools available to a kernel or driver developer: instead of fighting over a shared value with locks, you give every core its own value and skip the fight entirely. Once you are comfortable with the static and dynamic APIs, and you know to reach for the modern this_cpu_* operations and exported affinity APIs instead of older workarounds, you have a technique you will recognize throughout the kernel source tree — from the scheduler to the network stack to the memory allocator.