Mutex vs Spinlock in Linux Kernel: A Practical Locking Guide-Free Linux Device Drivers Training

Mutex vs Spinlock in Linux Kernel: A Practical Locking Guide
Free Linux Kernel Development Course — Updated for Kernel 6.12+ and PREEMPT_RT
Difficulty: Intermediate
Reading Time: 14 min
Kernel Version: 6.12+
What You Will Learn
The real difference between a mutex and a spinlock
When a driver should sleep vs when it must spin
How PREEMPT_RT changed spinlock behaviour
Writing safe lock/unlock code in a kernel module
Common locking bugs and how to avoid them
Prerequisites

Before working through this lecture of our free Linux kernel development course, you should already be comfortable with:

  • Writing and loading a basic Linux kernel module (insmod/rmmod)
  • The idea of process context vs interrupt context
  • Basic C pointers and structures
  • A Linux machine (kernel 6.1 or newer) with kernel headers installed for building modules
Why Locking Decisions Matter in Kernel Code

Every non-trivial Linux kernel module eventually touches data that more than one thread, CPU, or interrupt handler can access at the same time. The moment that happens, you need a lock — and the mutex vs spinlock in Linux kernel decision is one of the first real design choices a new driver author has to make. Pick the wrong one and you either kill performance or, worse, crash the machine by sleeping where sleeping is not allowed. This lecture walks through both locks from first principles, using current kernel behaviour rather than decade-old benchmarks, so that what you learn here still applies on a fresh 6.12+ kernel.

What Actually Happens Inside a Critical Section

A critical section is simply the block of code that touches shared data — for example, updating a linked list that an interrupt handler might also walk. If two execution contexts enter that block at the same time, the shared data can end up in a state neither context expected. A lock does not “protect data” by magic; it simply guarantees that only one context is allowed to be inside the critical section at any instant, and it decides what every other context does while it waits.

Three Threads Competing for One Lock
Thread A
Requests lock
Thread B
Gets lock → runs critical section
Thread C
Requests lock
A and C now wait for B to release the lock. How they wait — by sleeping or by spinning — is exactly what separates a mutex from a spinlock.
Mutex vs Spinlock in Linux Kernel: The Core Difference

Both locks solve the same problem — mutual exclusion — but they disagree on one question: what should a waiting thread do? A mutex answers “go to sleep and let the scheduler run something useful.” A spinlock answers “keep checking the lock in a tight loop until it’s free.” That single design decision drives every practical rule you will read below.

How the Kernel Mutex Lock Works

When a thread calls mutex_lock() and finds the lock already held, the kernel puts that thread to sleep and schedules something else on the CPU. This means a mutex is a blocking call — it can only be used from a context where sleeping is legal, which in practice means normal process context, never inside interrupt handlers, softirqs, tasklets, or any code path holding a spinlock.

#include <linux/mutex.h>

static DEFINE_MUTEX(ep_data_lock);
static int shared_counter;

static void update_shared_counter(int delta)
{
    mutex_lock(&ep_data_lock);
    shared_counter += delta;
    mutex_unlock(&ep_data_lock);
}

Tip: On modern kernels you can also use the scope-based guard(mutex)(&ep_data_lock); helper, which automatically unlocks when the enclosing block ends — it removes an entire class of “forgot to unlock” bugs.

How the Kernel Spinlock Works

A spinlock never sleeps. A waiting CPU simply keeps re-checking the lock value until it becomes free. That sounds wasteful, and on a single-core system it would be, but on today’s multi-core hardware the waiting CPU is a different CPU from the one running the critical section — so the wait costs a few cycles of busy polling instead of two expensive context switches. Because a spinlock never sleeps, it is the only lock you may use inside interrupt handlers.

#include <linux/spinlock.h>

static DEFINE_SPINLOCK(ep_irq_lock);
static int packet_count;

static irqreturn_t ep_irq_handler(int irq, void *dev_id)
{
    unsigned long flags;

    spin_lock_irqsave(&ep_irq_lock, flags);
    packet_count++;
    spin_unlock_irqrestore(&ep_irq_lock, flags);

    return IRQ_HANDLED;
}

Warning: Never call any sleeping function — mutex_lock(), kmalloc(GFP_KERNEL), msleep() — while holding a spinlock. The kernel cannot schedule() in that context, and doing so will trigger a “scheduling while atomic” bug or a full lockup.

Mutex vs Spinlock: Side-by-Side Comparison
Property Mutex Spinlock
Waiting behaviourSleepsBusy-waits (spins)
Usable in interrupt contextNoYes
Safe to sleep inside itYesNo
Best forLong or blocking critical sectionsVery short, non-blocking critical sections
Relative overheadHigher (context switches)Lower on multi-core hardware
Behaviour on PREEMPT_RTSame — sleeping lockBecomes a sleeping, priority-inheriting lock too
Choosing Between Mutex and Spinlock on Modern Kernels

The classic rule is simple: if the code can run in an atomic or interrupt context, or must not sleep, use a spinlock; if the critical section is allowed to sleep and runs only in normal process context, use a mutex. That rule still holds — but one major change in recent kernels affects how spinlocks actually behave underneath.

PREEMPT_RT and the New Meaning of “Spinlock”

The real-time preemption patch set, PREEMPT_RT, was merged into the mainline kernel with the 6.12 release. On a kernel built with PREEMPT_RT enabled, most regular spin_lock() calls are internally implemented on top of a priority-inheriting rt-mutex instead of a pure busy-wait loop. In practice this means a “spinlock” on an RT kernel can sleep, which changes some of the old assumptions driver authors relied on — code that assumed a spinlock critical section could never be preempted needs re-checking on RT kernels. Raw, always-spinning locks are still available through raw_spinlock_t for the few paths, such as scheduler internals, that truly cannot tolerate any sleeping.

Decision Flow: Mutex or Spinlock?
Can this code sleep or block?
↓
Yes, runs in process context → use mutex_lock() No, atomic/interrupt context → use spin_lock_irqsave()
Practical Code Walkthrough: A Small Character Driver

Here is a minimal, original example showing both locks used correctly in the same driver — a mutex guards a large buffer that the read() method may need to sleep while copying, and a spinlock guards a small counter touched from an interrupt handler.

#include <linux/module.h>
#include <linux/mutex.h>
#include <linux/spinlock.h>
#include <linux/fs.h>
#include <linux/uaccess.h>

#define EP_BUF_SIZE 4096

static char ep_buffer[EP_BUF_SIZE];
static DEFINE_MUTEX(ep_buffer_lock);   /* protects ep_buffer, may sleep */

static int irq_event_count;
static DEFINE_SPINLOCK(ep_count_lock); /* protects irq_event_count, no sleep */

static ssize_t ep_read(struct file *filp, char __user *ubuf,
                        size_t len, loff_t *off)
{
    ssize_t ret;

    mutex_lock(&ep_buffer_lock);
    ret = simple_read_from_buffer(ubuf, len, off, ep_buffer, EP_BUF_SIZE);
    mutex_unlock(&ep_buffer_lock);

    return ret;
}

static irqreturn_t ep_isr(int irq, void *dev_id)
{
    unsigned long flags;

    spin_lock_irqsave(&ep_count_lock, flags);
    irq_event_count++;
    spin_unlock_irqrestore(&ep_count_lock, flags);

    return IRQ_HANDLED;
}

Notice the pattern: the mutex sits in the read() path where copying to user space may need to fault in a page and therefore sleep, while the spinlock sits in the interrupt handler where sleeping is never an option.

Real-World Use Cases
Scenario Recommended Lock
Protecting a config structure updated rarely from sysfsMutex
Updating a packet counter inside a network driver ISRSpinlock
Guarding a linked list walked by both a workqueue and a timerSpinlock (timers run in softirq context)
Serialising access to an I2C bus during a multi-step transactionMutex
Common Mistakes and Troubleshooting
  • Sleeping while holding a spinlock — allocating memory with GFP_KERNEL or calling mutex_lock() inside a spinlock critical section. Fix: use GFP_ATOMIC or move the allocation outside the locked region.
  • Forgetting _irqsave variants — using plain spin_lock() on data also touched by an interrupt handler causes a deadlock if the interrupt fires on the same CPU while the lock is held. Always use spin_lock_irqsave() when an ISR shares the lock.
  • Holding a lock across a blocking user-space copy — wrap only the minimum code that touches shared data; don’t hold a mutex across an entire syscall if avoidable.
  • Double-locking the same mutex from one thread — a regular mutex is not recursive; enable CONFIG_PROVE_LOCKING (lockdep) during development to catch this immediately.
Best Practices for Locking in Kernel Code
  • Keep every critical section as short as possible, regardless of which lock you use.
  • Always run development kernels with lockdep enabled to catch ordering and sleeping bugs early.
  • Document, next to every lock declaration, exactly which data it protects.
  • Prefer the scope-based guard()/scoped_guard() helpers on kernels that support them to avoid unlock-forgetting bugs.
  • Never assume lock-free “it’s just one write” is safe — use atomic operations or a lock instead of guessing.
Performance Considerations

Spinlocks avoid the scheduling overhead of a context switch, which makes them cheaper for very short critical sections on multi-core systems. But a spinlock held too long wastes CPU cycles on every waiting core, so it should never guard anything beyond a handful of instructions. Mutexes cost more per lock/unlock cycle because of the potential sleep-and-wake path, but that cost is irrelevant once the critical section itself takes longer than a couple of context switches would.

Security Considerations

Incorrect locking is a common root cause of kernel security bugs, particularly race conditions that lead to use-after-free issues when reference-counted objects are freed without proper protection. Always pair a lock with the exact data it guards, review lock ordering carefully to avoid deadlocks that can be triggered by a malicious or unprivileged process, and treat any lockdep warning in a driver as a bug to fix, not a message to ignore.

Key Takeaways
  • A mutex is a sleeping lock; a spinlock is a busy-wait lock.
  • Interrupt context can only use spinlocks, never mutexes.
  • On a PREEMPT_RT kernel, most spinlocks behave like sleeping, priority-inheriting locks internally.
  • Keep critical sections short no matter which lock you choose.
  • Use lockdep during development to catch locking bugs before they reach production.
Conclusion

The mutex vs spinlock in Linux kernel decision isn’t about which lock is “better” — it’s about matching the lock’s waiting behaviour to the context your code runs in. Once you internalise that a mutex sleeps and a spinlock spins, most of the rules in this lecture follow naturally. As you move on to later lectures in this free Linux kernel development course, you’ll see both locks reused constantly in real driver code, alongside more specialised tools like RCU and seqlocks for read-heavy data.

Frequently Asked Questions (FAQ)
Can I use a spinlock instead of a mutex everywhere to keep things simple?

Technically yes in many cases, but any critical section that takes noticeably longer than a couple of microseconds will waste CPU cycles on every waiting core. Use a spinlock only for short, non-blocking sections.

What happens if I call mutex_lock() inside an interrupt handler?

The kernel will attempt to sleep in a context where scheduling isn’t allowed, which triggers a “scheduling while atomic” bug and can crash or hang the system.

Is a spinlock still a busy-wait loop on a single-core system?

On single-core kernels compiled without SMP support, most spinlock operations effectively reduce to disabling preemption, since there’s no second CPU to spin on.

Why did PREEMPT_RT change how spinlocks behave?

PREEMPT_RT prioritises low, predictable latency. Converting most spinlocks into priority-inheriting sleeping locks avoids priority inversion and keeps high-priority real-time tasks from being blocked by long busy-wait sections.

Are mutexes recursive in the Linux kernel?

No. A standard kernel mutex is not recursive — locking it twice from the same thread will deadlock. Use lockdep to catch this during development.

What tool helps catch locking bugs before they happen in production?

The kernel’s lock validator, lockdep (enabled through CONFIG_PROVE_LOCKING), tracks lock acquisition order across the system and flags potential deadlocks and unsafe sleeping calls at development time.

Is this lecture part of a full free embedded systems course?

Yes — this lecture belongs to EmbeddedPathashala’s free Linux kernel development course, part of our broader free embedded systems course covering kernel internals, device drivers, and Bluetooth/BLE development.

Continue Your Free Linux Kernel Development Course

This lecture is part of EmbeddedPathashala’s free linux device drivers course and free embedded systems course. Explore more lectures on kernel internals, synchronisation, and driver development.

Browse the Course Index Join EmbeddedPathashala

2 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *