Linux Kernel Atomic Operations: atomic_t and refcount_t Explained-Linux Device Driver Training

← Previous Lecture    Next Lecture →

Linux Kernel Atomic Operations: atomic_t and refcount_t Explained
Free Linux Kernel Development Course — Kernel Synchronization, Part 2
Kernel 6.x Ready
Beginner Friendly
Original Code Examples

If you have been following this free Linux kernel development course, you already know that two kernel threads updating the same plain int at the same time is a recipe for a race condition. That is exactly where linux kernel atomic operations come in. Instead of wrapping a simple counter with a mutex or a spinlock every single time, the kernel gives us two purpose-built data types — atomic_t and refcount_t — that update whole numbers safely, in a single indivisible step, without any lock at all.

In this lecture of our free linux device drivers course we will build a mental model for both interfaces, see why refcount_t exists even though it is built on top of atomic_t internally, and write two small, original kernel-module examples that you can compile and load on any modern kernel 6.x system.

Topics covered in this lecture:
atomic_t refcount_t atomic64_t Reference Counting Read-Modify-Write Kernel 6.x
What You Will Learn
Why plain integers are unsafe in shared kernel state The atomic_t API and how it differs from a normal int 64-bit atomics with atomic64_t The refcount_t API for safe reference counting How refcount_t prevents use-after-free bugs A hands-on kernel module using both APIs
Prerequisites

This lecture assumes you have already gone through the earlier lectures in our free embedded systems course on critical sections, mutexes, and spinlocks. You should be comfortable building and loading a simple kernel module on a kernel 6.x based Linux system (Ubuntu or any distribution with kernel headers installed).

A Quick Recap: Why Not Just Use a Plain int?

Earlier in this course we saw that something as simple as counter++ is actually three separate CPU steps: read the value, add one to it, and write it back. If two CPU cores run that sequence on the same memory location at the same time, one update can silently get lost. The usual fix is a spinlock or a mutex around the increment. That works, but it is overkill for the single most common pattern in the kernel: a plain integer counter that only ever gets incremented, decremented, or compared.

The kernel solves this narrower problem with a dedicated data type instead of a general-purpose lock. That is the whole idea behind atomic_t.

The atomic_t Interface

atomic_t is a special wrapper type built specifically to hold a 32-bit signed integer that the CPU can update in one indivisible instruction, with no separate lock needed. Every operation on an atomic_t variable — reading it, setting it, adding to it, or comparing it — is guaranteed to complete as a single step from the point of view of every other CPU in the system.

A few of the most commonly used atomic_t functions look like this:

atomic_t counter = ATOMIC_INIT(0);

atomic_set(&counter, 5);          /* counter = 5           */
atomic_read(&counter);            /* read current value    */
atomic_inc(&counter);             /* counter++              */
atomic_dec(&counter);             /* counter--              */
atomic_add(3, &counter);          /* counter += 3           */
atomic_sub(2, &counter);          /* counter -= 2           */
atomic_dec_and_test(&counter);    /* counter--, true if 0   */

Notice that you never touch the plain integer directly. Every read and every write goes through one of these helper functions, and the kernel makes sure each one maps to a lock-free, CPU-level atomic instruction.

Two Threads Incrementing an atomic_t Safely
Thread A
atomic_inc(&counter)
executes as one CPU step
Thread B
atomic_inc(&counter)
waits its turn at CPU level, no lock() call needed
Both updates land correctly — final value is always +2, never +1

64-bit Counters: atomic64_t

atomic_t only holds a 32-bit value. When a driver needs a wider running total — total bytes transferred over the lifetime of a device, for example — the kernel provides atomic64_t (also known as atomic_long_t). The function names simply gain a 64: atomic64_set(), atomic64_read(), atomic64_inc(), and so on. The behaviour and guarantees are identical to their 32-bit counterparts.

The refcount_t Interface: Purpose-Built for Reference Counting

Reference counting is one specific job that atomic_t gets used for constantly inside the kernel: an object is created with a count of 1, every new user of that object bumps the count up, every user that is done with it brings the count back down, and when the count hits zero the object is safely freed.

The trouble is, a plain atomic_t reference counter has no idea what a “legal” value looks like. Nothing stops buggy code from decrementing it past zero, or from wrapping a 32-bit counter all the way around after billions of increments. Either mistake can lead to a use-after-free bug, which is one of the most dangerous classes of kernel security issues.

refcount_t exists to close exactly that gap. It is a thin wrapper around atomic_t, but it adds strict rules:

  • A refcount_t only ever holds an unsigned value between 1 and just under UINT_MAX.
  • Trying to increment a counter that has already reached zero, or decrement it below zero, triggers a loud kernel warning instead of silently corrupting memory.
  • Once a counter reaches the special saturated value, it locks at that value and refuses to move further, which blocks the wrap-around trick that use-after-free exploits often rely on.

Since kernel version 5.5, this full checking behaviour is simply how refcount_t works everywhere in the tree — the generic implementation was made fast enough that every architecture uses the same safe, fully checked logic by default, so you do not need to think about picking a “faster but weaker” variant on a modern kernel.

Common refcount_t functions mirror the atomic_t ones you already know:

refcount_t refs;

refcount_set(&refs, 1);              /* start of life, count = 1 */
refcount_inc(&refs);                 /* a new user takes a reference */
refcount_dec(&refs);                 /* a user releases its reference */
refcount_dec_and_test(&refs);        /* release + check if it hit zero */
refcount_read(&refs);                /* read the current count */

atomic_t vs refcount_t at a Glance

Aspect atomic_t refcount_t
Value type Signed integer Unsigned integer only
Best use case General-purpose counters, flags, statistics Object lifetime / reference counting only
Underflow / overflow behaviour Silently wraps around Warns and saturates, will not wrap
Typical valid range Full 32-bit signed range 1 to just under UINT_MAX

Hands-On: A Simple Kernel Module Using Both APIs

Below is an original, minimal kernel module (not taken from any book) that demonstrates both types together. It simulates a small in-kernel object pool: atomic_t tracks how many times the module’s device file has been opened in total, and refcount_t tracks how many active users currently hold the object open, freeing a simulated resource once the last user closes it. It compiles cleanly on kernel 6.x.

#include <linux/module.h>
#include <linux/init.h>
#include <linux/fs.h>
#include <linux/refcount.h>
#include <linux/atomic.h>
#include <linux/miscdevice.h>
#include <linux/printk.h>

#define EP_DEV_NAME "ep_atomic_refcount_demo"

/* Lifetime open counter - just for statistics, never freed on it */
static atomic_t ep_open_count = ATOMIC_INIT(0);

/* Active-user reference count - drives real resource cleanup */
static refcount_t ep_active_refs;
static bool ep_resource_live;

static int ep_demo_open(struct inode *inode, struct file *filp)
{
    atomic_inc(&ep_open_count);

    if (!ep_resource_live) {
        refcount_set(&ep_active_refs, 1);
        ep_resource_live = true;
        pr_info(EP_DEV_NAME ": resource allocated, refs = %u\n",
                refcount_read(&ep_active_refs));
    } else {
        refcount_inc(&ep_active_refs);
        pr_info(EP_DEV_NAME ": new user attached, refs = %u\n",
                refcount_read(&ep_active_refs));
    }

    pr_info(EP_DEV_NAME ": total opens so far = %d\n",
            atomic_read(&ep_open_count));
    return 0;
}

static int ep_demo_release(struct inode *inode, struct file *filp)
{
    if (refcount_dec_and_test(&ep_active_refs)) {
        ep_resource_live = false;
        pr_info(EP_DEV_NAME ": last user left, resource freed\n");
    } else {
        pr_info(EP_DEV_NAME ": one user left, refs = %u\n",
                refcount_read(&ep_active_refs));
    }
    return 0;
}

static const struct file_operations ep_demo_fops = {
    .owner   = THIS_MODULE,
    .open    = ep_demo_open,
    .release = ep_demo_release,
};

static struct miscdevice ep_demo_misc = {
    .minor = MISC_DYNAMIC_MINOR,
    .name  = EP_DEV_NAME,
    .fops  = &ep_demo_fops,
};

static int __init ep_demo_init(void)
{
    int ret = misc_register(&ep_demo_misc);

    if (ret) {
        pr_err(EP_DEV_NAME ": misc_register failed\n");
        return ret;
    }
    pr_info(EP_DEV_NAME ": loaded, /dev/%s ready\n", EP_DEV_NAME);
    return 0;
}

static void __exit ep_demo_exit(void)
{
    misc_deregister(&ep_demo_misc);
    pr_info(EP_DEV_NAME ": unloaded\n");
}

module_init(ep_demo_init);
module_exit(ep_demo_exit);

MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("EmbeddedPathashala demo: atomic_t and refcount_t together");

Try opening /dev/ep_atomic_refcount_demo from two different terminals and watch dmesg. The first open allocates the simulated resource with a reference count of 1; the second concurrent open just bumps the reference count; and the resource is only freed once the last open file descriptor is closed. Meanwhile, ep_open_count keeps climbing forever as a plain lifetime statistic — a perfect example of when atomic_t is the right tool, and when refcount_t is the right tool.

Object Lifetime Driven by refcount_t
open() #1
refcount_set(1)
→
open() #2
refcount_inc()
→
release() #1
refcount_dec_and_test() = false
→
release() #2
refcount_dec_and_test() = true, freed

Frequently Asked Questions

Is atomic_t the same as declaring a volatile int?

No. volatile only stops the compiler from caching a value in a register; it says nothing about CPU-level atomicity or memory ordering between cores. atomic_t gives you a real, indivisible read-modify-write guarantee across all CPUs.

Can I use atomic_t for reference counting instead of refcount_t?

Technically yes, and older kernel code still does. But refcount_t gives you free protection against underflow, overflow, and use-after-free bugs, so for any new reference-counting code you should prefer refcount_t.

What happens if I decrement a refcount_t below zero?

The kernel fires a loud WARN() and keeps the counter pinned at its saturated value instead of letting it wrap around, so the bug is caught early instead of turning into silent memory corruption.

Do I still need a spinlock if I am using atomic_t?

Not for the single counter itself. But if your critical section touches more than one variable together, or a variable plus some other state, you still need a spinlock or mutex, because atomic_t only guarantees atomicity for one integer at a time.

Is refcount_t available on 32-bit and 64-bit kernels?

Yes, refcount_t is architecture independent and works the same way whether your driver runs on a 32-bit or 64-bit kernel build.

What is the difference between atomic_t and atomic64_t?

They are functionally identical in what operations they support; the only difference is the width of the integer they hold, 32-bit for atomic_t and 64-bit for atomic64_t.

Continue the Free Linux Kernel Development Course

Next, we move on to Read-Modify-Write atomic operators in more depth, followed by reader-writer spinlocks and lock-free per-CPU variables.

Previous Lecture Next Lecture

2 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *