← Previous Lecture Next Lecture →
If you have been following this free Linux kernel development course, you already know that two kernel threads updating the same plain int at the same time is a recipe for a race condition. That is exactly where linux kernel atomic operations come in. Instead of wrapping a simple counter with a mutex or a spinlock every single time, the kernel gives us two purpose-built data types — atomic_t and refcount_t — that update whole numbers safely, in a single indivisible step, without any lock at all.
In this lecture of our free linux device drivers course we will build a mental model for both interfaces, see why refcount_t exists even though it is built on top of atomic_t internally, and write two small, original kernel-module examples that you can compile and load on any modern kernel 6.x system.
This lecture assumes you have already gone through the earlier lectures in our free embedded systems course on critical sections, mutexes, and spinlocks. You should be comfortable building and loading a simple kernel module on a kernel 6.x based Linux system (Ubuntu or any distribution with kernel headers installed).
A Quick Recap: Why Not Just Use a Plain int?
Earlier in this course we saw that something as simple as counter++ is actually three separate CPU steps: read the value, add one to it, and write it back. If two CPU cores run that sequence on the same memory location at the same time, one update can silently get lost. The usual fix is a spinlock or a mutex around the increment. That works, but it is overkill for the single most common pattern in the kernel: a plain integer counter that only ever gets incremented, decremented, or compared.
The kernel solves this narrower problem with a dedicated data type instead of a general-purpose lock. That is the whole idea behind atomic_t.
The atomic_t Interface
atomic_t is a special wrapper type built specifically to hold a 32-bit signed integer that the CPU can update in one indivisible instruction, with no separate lock needed. Every operation on an atomic_t variable — reading it, setting it, adding to it, or comparing it — is guaranteed to complete as a single step from the point of view of every other CPU in the system.
A few of the most commonly used atomic_t functions look like this:
atomic_t counter = ATOMIC_INIT(0);
atomic_set(&counter, 5); /* counter = 5 */
atomic_read(&counter); /* read current value */
atomic_inc(&counter); /* counter++ */
atomic_dec(&counter); /* counter-- */
atomic_add(3, &counter); /* counter += 3 */
atomic_sub(2, &counter); /* counter -= 2 */
atomic_dec_and_test(&counter); /* counter--, true if 0 */
Notice that you never touch the plain integer directly. Every read and every write goes through one of these helper functions, and the kernel makes sure each one maps to a lock-free, CPU-level atomic instruction.
64-bit Counters: atomic64_t
atomic_t only holds a 32-bit value. When a driver needs a wider running total — total bytes transferred over the lifetime of a device, for example — the kernel provides atomic64_t (also known as atomic_long_t). The function names simply gain a 64: atomic64_set(), atomic64_read(), atomic64_inc(), and so on. The behaviour and guarantees are identical to their 32-bit counterparts.
The refcount_t Interface: Purpose-Built for Reference Counting
Reference counting is one specific job that atomic_t gets used for constantly inside the kernel: an object is created with a count of 1, every new user of that object bumps the count up, every user that is done with it brings the count back down, and when the count hits zero the object is safely freed.
The trouble is, a plain atomic_t reference counter has no idea what a “legal” value looks like. Nothing stops buggy code from decrementing it past zero, or from wrapping a 32-bit counter all the way around after billions of increments. Either mistake can lead to a use-after-free bug, which is one of the most dangerous classes of kernel security issues.
refcount_t exists to close exactly that gap. It is a thin wrapper around atomic_t, but it adds strict rules:
- A
refcount_tonly ever holds an unsigned value between 1 and just underUINT_MAX. - Trying to increment a counter that has already reached zero, or decrement it below zero, triggers a loud kernel warning instead of silently corrupting memory.
- Once a counter reaches the special saturated value, it locks at that value and refuses to move further, which blocks the wrap-around trick that use-after-free exploits often rely on.
Since kernel version 5.5, this full checking behaviour is simply how refcount_t works everywhere in the tree — the generic implementation was made fast enough that every architecture uses the same safe, fully checked logic by default, so you do not need to think about picking a “faster but weaker” variant on a modern kernel.
Common refcount_t functions mirror the atomic_t ones you already know:
refcount_t refs;
refcount_set(&refs, 1); /* start of life, count = 1 */
refcount_inc(&refs); /* a new user takes a reference */
refcount_dec(&refs); /* a user releases its reference */
refcount_dec_and_test(&refs); /* release + check if it hit zero */
refcount_read(&refs); /* read the current count */
atomic_t vs refcount_t at a Glance
| Aspect | atomic_t | refcount_t |
|---|---|---|
| Value type | Signed integer | Unsigned integer only |
| Best use case | General-purpose counters, flags, statistics | Object lifetime / reference counting only |
| Underflow / overflow behaviour | Silently wraps around | Warns and saturates, will not wrap |
| Typical valid range | Full 32-bit signed range | 1 to just under UINT_MAX |
Hands-On: A Simple Kernel Module Using Both APIs
Below is an original, minimal kernel module (not taken from any book) that demonstrates both types together. It simulates a small in-kernel object pool: atomic_t tracks how many times the module’s device file has been opened in total, and refcount_t tracks how many active users currently hold the object open, freeing a simulated resource once the last user closes it. It compiles cleanly on kernel 6.x.
#include <linux/module.h>
#include <linux/init.h>
#include <linux/fs.h>
#include <linux/refcount.h>
#include <linux/atomic.h>
#include <linux/miscdevice.h>
#include <linux/printk.h>
#define EP_DEV_NAME "ep_atomic_refcount_demo"
/* Lifetime open counter - just for statistics, never freed on it */
static atomic_t ep_open_count = ATOMIC_INIT(0);
/* Active-user reference count - drives real resource cleanup */
static refcount_t ep_active_refs;
static bool ep_resource_live;
static int ep_demo_open(struct inode *inode, struct file *filp)
{
atomic_inc(&ep_open_count);
if (!ep_resource_live) {
refcount_set(&ep_active_refs, 1);
ep_resource_live = true;
pr_info(EP_DEV_NAME ": resource allocated, refs = %u\n",
refcount_read(&ep_active_refs));
} else {
refcount_inc(&ep_active_refs);
pr_info(EP_DEV_NAME ": new user attached, refs = %u\n",
refcount_read(&ep_active_refs));
}
pr_info(EP_DEV_NAME ": total opens so far = %d\n",
atomic_read(&ep_open_count));
return 0;
}
static int ep_demo_release(struct inode *inode, struct file *filp)
{
if (refcount_dec_and_test(&ep_active_refs)) {
ep_resource_live = false;
pr_info(EP_DEV_NAME ": last user left, resource freed\n");
} else {
pr_info(EP_DEV_NAME ": one user left, refs = %u\n",
refcount_read(&ep_active_refs));
}
return 0;
}
static const struct file_operations ep_demo_fops = {
.owner = THIS_MODULE,
.open = ep_demo_open,
.release = ep_demo_release,
};
static struct miscdevice ep_demo_misc = {
.minor = MISC_DYNAMIC_MINOR,
.name = EP_DEV_NAME,
.fops = &ep_demo_fops,
};
static int __init ep_demo_init(void)
{
int ret = misc_register(&ep_demo_misc);
if (ret) {
pr_err(EP_DEV_NAME ": misc_register failed\n");
return ret;
}
pr_info(EP_DEV_NAME ": loaded, /dev/%s ready\n", EP_DEV_NAME);
return 0;
}
static void __exit ep_demo_exit(void)
{
misc_deregister(&ep_demo_misc);
pr_info(EP_DEV_NAME ": unloaded\n");
}
module_init(ep_demo_init);
module_exit(ep_demo_exit);
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("EmbeddedPathashala demo: atomic_t and refcount_t together");
Try opening /dev/ep_atomic_refcount_demo from two different terminals and watch dmesg. The first open allocates the simulated resource with a reference count of 1; the second concurrent open just bumps the reference count; and the resource is only freed once the last open file descriptor is closed. Meanwhile, ep_open_count keeps climbing forever as a plain lifetime statistic — a perfect example of when atomic_t is the right tool, and when refcount_t is the right tool.
refcount_set(1)
refcount_inc()
refcount_dec_and_test() = false
refcount_dec_and_test() = true, freed
Frequently Asked Questions
No. volatile only stops the compiler from caching a value in a register; it says nothing about CPU-level atomicity or memory ordering between cores. atomic_t gives you a real, indivisible read-modify-write guarantee across all CPUs.
Technically yes, and older kernel code still does. But refcount_t gives you free protection against underflow, overflow, and use-after-free bugs, so for any new reference-counting code you should prefer refcount_t.
The kernel fires a loud WARN() and keeps the counter pinned at its saturated value instead of letting it wrap around, so the bug is caught early instead of turning into silent memory corruption.
Not for the single counter itself. But if your critical section touches more than one variable together, or a variable plus some other state, you still need a spinlock or mutex, because atomic_t only guarantees atomicity for one integer at a time.
Yes, refcount_t is architecture independent and works the same way whether your driver runs on a 32-bit or 64-bit kernel build.
They are functionally identical in what operations they support; the only difference is the width of the integer they hold, 32-bit for atomic_t and 64-bit for atomic64_t.
Continue the Free Linux Kernel Development Course
Next, we move on to Read-Modify-Write atomic operators in more depth, followed by reader-writer spinlocks and lock-free per-CPU variables.
Previous Lecture Next Lecture
2 Comments