Why Locking Matters in Device Drivers

Device drivers are one of the most concurrency-heavy corners of the kernel. A single driver can be entered from process context (a user calling read()/write()), from a hardware interrupt, and sometimes from a softirq or timer — all touching the same buffers and state. Locking is how you make sure those entry points don’t corrupt each other’s work. This is exactly why locking is treated as a core module in any serious free Linux device drivers course.

Spinlocks: The Workhorse of Kernel Locking

A spinlock is the simplest kernel lock: a thread that can’t acquire it simply loops (“spins”) checking the lock repeatedly instead of going to sleep. That makes spinlocks cheap to acquire and release, but expensive to hold for long, because every other CPU waiting on it is burning cycles doing nothing useful.

spin_lock() vs spin_lock_irqsave()

Use plain spin_lock()/spin_unlock() when the shared data is never touched from interrupt context. If an interrupt handler can also touch the same data, use spin_lock_irqsave()/spin_unlock_irqrestore(), which disables local interrupts before taking the lock and restores their previous state afterward — otherwise you risk the CPU interrupting itself into a deadlock on the same lock.

static DEFINE_SPINLOCK(dev_lock);

/* safe even if an IRQ handler also touches shared_buf */
void update_buffer(struct my_dev *dev, int value)
{
    unsigned long flags;

    spin_lock_irqsave(&dev_lock, flags);
    dev->shared_buf[dev->head++] = value;
    spin_unlock_irqrestore(&dev_lock, flags);
}

Mutexes: Sleeping Locks for Process Context

A mutex behaves differently: a thread that can’t acquire it goes to sleep instead of spinning, and gets woken up when the lock becomes free. This makes mutexes far cheaper to hold for longer operations, but they come with two hard rules: you may never take a mutex from interrupt context, and you may never take a mutex and then go to sleep while holding a spinlock.

static DEFINE_MUTEX(config_mutex);

int update_config(struct my_dev *dev, struct config *new_cfg)
{
    int ret;

    if (mutex_lock_interruptible(&config_mutex))
        return -ERESTARTSYS;

    ret = apply_config(dev, new_cfg);   /* may sleep internally */

    mutex_unlock(&config_mutex);
    return ret;
}

When to Use a Mutex Instead of a Spinlock

Choose a mutex when the protected section might sleep — for example, if it calls copy_from_user(), allocates memory with GFP_KERNEL, or waits on I/O. Choose a spinlock when the section is short, non-blocking, and might run in interrupt or atomic context.

Spinlock vs Mutex vs Semaphore: Quick Comparison

Property	Spinlock	Mutex	Semaphore
Waiting behaviour	Busy-waits (spins)	Sleeps	Sleeps
Usable in interrupt context	Yes (with irqsave)	No	No
Owner tracking	No	Yes	No
Typical use	Short, non-blocking sections	Longer, possibly-blocking sections	Counting resources, legacy code
Modern kernel guidance	Preferred for hot paths	Preferred over semaphores for mutual exclusion	Mostly superseded by mutex/completion

qspinlock: The Modern Spinlock Implementation

Older kernels used a simple “ticket lock” for spinlocks, where waiters spin on a shared cache line, causing heavy cache-line bouncing as core counts grew. Current kernels default to the queued spinlock (qspinlock) on SMP systems: each waiting CPU spins on its own local queue node instead of a shared counter, dramatically cutting cross-core cache traffic on systems with many cores. As a driver developer you don’t have to implement this yourself — spin_lock() automatically uses qspinlock under the hood — but understanding why it exists helps you reason about lock contention on modern multi-core hardware.

RCU: Read-Copy-Update for Read-Heavy Data

RCU is the mechanism of choice when data is read far more often than it’s written — routing tables, module lists, and similar structures. Readers under RCU take no lock at all and pay almost zero synchronization cost; writers create a new copy of the data, publish the new pointer, and wait for a “grace period” to ensure no reader is still using the old version before freeing it.

rcu_read_lock();
p = rcu_dereference(shared_ptr);
if (p)
    process(p);
rcu_read_unlock();

/* writer side */
struct my_data *new_data = kmalloc(sizeof(*new_data), GFP_KERNEL);
/* ... populate new_data ... */
rcu_assign_pointer(shared_ptr, new_data);
synchronize_rcu();
kfree(old_data);

RCU trades write-side complexity for near-zero read-side cost — a good fit whenever reads dominate.

Lock Debugging With Lockdep

The kernel’s lock validator, lockdep, tracks every lock acquisition and release across the system and flags impossible orderings before they can cause a real deadlock in the field. Enable it in a debug kernel via:

make menuconfig
# Kernel hacking ---> Lock Debugging (spinlocks, mutexes, etc...) ---> Lock Debugging: prove locking correctness

Running your driver under a lockdep-enabled kernel during development catches ordering bugs long before they turn into an intermittent production deadlock.

Real-World Use Case: A Shared Ring Buffer in a Character Driver

Here is an original example combining a spinlock-protected ring buffer with a mutex-protected configuration path, the kind of pattern you’ll see repeatedly in real drivers:

#define RING_SIZE 64

struct my_dev {
    int buf[RING_SIZE];
    int head, tail;
    spinlock_t buf_lock;      /* protects buf/head/tail, touched by IRQ */
    struct mutex cfg_lock;    /* protects slower config changes */
    int sample_rate;
};

/* producer: called from the device's IRQ handler */
static void push_sample(struct my_dev *dev, int sample)
{
    unsigned long flags;

    spin_lock_irqsave(&dev->buf_lock, flags);
    dev->buf[dev->head] = sample;
    dev->head = (dev->head + 1) % RING_SIZE;
    spin_unlock_irqrestore(&dev->buf_lock, flags);
}

/* consumer: called from a user read() syscall, process context */
static int pop_sample(struct my_dev *dev, int *out)
{
    unsigned long flags;
    int ret = 0;

    spin_lock_irqsave(&dev->buf_lock, flags);
    if (dev->head == dev->tail) {
        ret = -EAGAIN;
    } else {
        *out = dev->buf[dev->tail];
        dev->tail = (dev->tail + 1) % RING_SIZE;
    }
    spin_unlock_irqrestore(&dev->buf_lock, flags);
    return ret;
}

/* config change: process context only, may sleep, uses mutex instead */
static int set_sample_rate(struct my_dev *dev, int rate)
{
    mutex_lock(&dev->cfg_lock);
    dev->sample_rate = rate;
    mutex_unlock(&dev->cfg_lock);
    return 0;
}

Notice how the fast, IRQ-reachable path uses a spinlock, while the slower, sleep-friendly configuration path uses a mutex — exactly the split described earlier in this lesson.

Common Mistakes and Troubleshooting

• Taking a mutex in an interrupt handler. This will trigger a kernel warning (or worse) — mutexes can sleep and interrupt context cannot.
• Nesting locks in inconsistent order. If path A takes lock1 then lock2, and path B takes lock2 then lock1, you have a classic deadlock waiting to happen — lockdep will catch this for you if enabled.
• Forgetting irqsave when IRQ context is involved. A plain spin_lock() won’t protect you if the same lock is also taken from an interrupt handler on the same CPU.
• Holding a spinlock across a call that can sleep. Allocating memory with GFP_KERNEL or calling copy_from_user() while holding a spinlock is a common bug that silently corrupts system stability.
• Using RCU for write-heavy data. RCU shines for read-heavy structures; on write-heavy data the grace-period and copy overhead can make it slower than a plain lock.

Best Practices

• Match the lock to the context: spinlock for anything reachable from interrupts, mutex for anything that might sleep.
• Always take multiple locks in the same global order across your entire driver.
• Keep critical sections as short as possible, especially under a spinlock.
• Use RCU for data that is read far more often than it’s written.
• Enable lockdep and KCSAN in every development build, not just before a release.

Performance Considerations

Spinlocks cost CPU cycles while waiters spin; mutexes cost a context switch when contended but free the CPU for other work while waiting. On modern multi-core systems, the switch to qspinlock has significantly reduced cache-line bouncing under heavy spinlock contention compared to older ticket-lock implementations. For read-dominated data structures, RCU typically outperforms both by removing reader-side synchronization almost entirely, at the cost of more complex writer-side logic and delayed reclamation of old data.

Security Considerations

Locking bugs in drivers are a recurring source of real kernel vulnerabilities. Missing locks around reference counts can lead to use-after-free conditions; inconsistent lock ordering across driver paths can be triggered by unprivileged users to cause denial-of-service through deadlock; and RCU misuse (freeing data before a grace period completes) can produce exploitable use-after-free bugs. Treat driver locking correctness as seriously as you’d treat input validation.

Summary / Key Takeaways

• Spinlocks busy-wait and are safe in interrupt context; mutexes sleep and are only for process context.
• Use spin_lock_irqsave() whenever the same data is touched from an interrupt handler.
• Modern kernels use qspinlock by default, reducing contention overhead on many-core systems.
• RCU gives near-zero-cost reads for read-heavy shared data, at the cost of write-side complexity.
• Lockdep and KCSAN should be part of every driver developer’s regular development workflow.

Conclusion

Locking is where the theory from the previous lesson becomes real, working driver code. Get the spinlock-vs-mutex decision right, respect interrupt context rules, and lean on RCU where reads dominate, and you’ll avoid the vast majority of concurrency bugs that trip up new kernel developers. That’s the practical core of what this free Linux device drivers course and the wider free Linux kernel development course are built to teach.

Frequently Asked Questions