• Basic C programming and pointers
• A working Linux kernel module build setup (kernel headers + gcc + make)
• Familiarity with writing a simple character or misc driver
• Comfort reading dmesg output

A modern Linux system runs on multiple CPU cores, and the kernel is fully preemptible. That means two things can go wrong at once inside your driver:

• Two CPUs can run your driver’s code simultaneously and both try to update the same variable.
• A single CPU can be interrupted mid-update by an interrupt handler that also touches the same variable.

Any piece of code that touches shared state in this way is called a critical section. The kernel gives you several locking primitives to protect a critical section; the spinlock is the one you reach for when the critical section is short and you cannot afford to sleep.

The rule of thumb: if the code you need to protect might block, allocate memory with sleeping allowed, or call anything that could sleep, a spinlock is the wrong tool.

The moment you call spin_lock(), the kernel raises your code’s “preemption disabled” counter and, for the _irqsave variants, also disables local interrupts. From that point until you call the matching unlock, your code is running in atomic context. In atomic context the scheduler is not allowed to switch away from you. If something inside your critical section tries to sleep, there is nobody left to schedule the next task, and the kernel treats this as a bug.

Below is a trimmed-down misc driver write handler that stores an incoming value under a spinlock. It is written fresh for this lecture and targets a 6.12+ kernel.

static DEFINE_SPINLOCK(my_lock);
static int stored_value;

static ssize_t my_write(struct file *filp, const char __user *buf,
                         size_t count, loff_t *off)
{
    int local_val;

    if (copy_from_user(&local_val, buf, sizeof(local_val)))
        return -EFAULT;

    spin_lock(&my_lock);
    stored_value = local_val;   /* short, non-blocking update */
    spin_unlock(&my_lock);

    return sizeof(local_val);
}

Notice that copy_from_user(), which can fault and sleep, happens before the lock is taken. That ordering is the whole trick.

Here is the broken version many beginners write by accident, where a blocking call is placed inside the locked region:

spin_lock(&my_lock);
stored_value = local_val;
msleep(10);              /* BUG: this can sleep while atomic! */
spin_unlock(&my_lock);

On a debug-enabled kernel this will immediately print a “scheduling while atomic” splat the first time the code path runs. On a kernel without the right debug config, the very same bug may run for months without a single visible symptom, only to corrupt data or hang a CPU under load later.

Modern kernels ship a config option that makes this class of bug loud and immediate instead of silent. Enable it in your test kernel:

make menuconfig
# Kernel Hacking -> Lock Debugging -> Sleep inside atomic section checking
# CONFIG_DEBUG_ATOMIC_SLEEP=y

With this enabled, any attempt to sleep while a spinlock is held, while preemption is disabled, or inside an RCU read-side critical section produces an immediate kernel warning with a full call stack, naming the offending function directly. Always build and test new driver code against a debug kernel before it goes anywhere near production hardware.

• Interrupt handlers that update a small ring buffer index shared with the bottom half.
• Character and misc drivers that maintain simple counters or flags read by multiple user processes.
• Network drivers protecting a short queue-length field on the hot transmit path.
• Block drivers guarding request-list bookkeeping that must never block the I/O submission path.

• Keep every spinlock critical section as short as physically possible.
• Never call anything that can block: no kmalloc(GFP_KERNEL), no msleep(), no user-copy functions.
• Match spin_lock_irqsave() with spin_unlock_irqrestore() whenever an interrupt handler shares the same lock.
• Prefer a mutex if the protected section might ever need to sleep.
• Always test with CONFIG_DEBUG_ATOMIC_SLEEP and lockdep enabled before release.

A spinlock is cheap only when contention is low and the critical section is tiny. Under heavy contention, spinning CPUs waste cycles that could be doing useful work, and on large multi-socket machines cache-line bouncing of the lock word becomes a real bottleneck. If profiling shows a spinlock as a hotspot, consider per-CPU data structures, RCU, or finer-grained locking instead of a single global lock.

Unprotected shared state in a driver is not just a stability bug, it can be a security bug. A race condition on a size or index field can be exploited from user space to trigger an out-of-bounds read or write in kernel memory. Treat every field touched by more than one execution context as a potential attack surface until it is properly locked.

Since the real-time preemption patches merged into the mainline kernel, regular spinlock_t on a PREEMPT_RT build is internally implemented as a priority-inheriting sleeping lock (an rt_mutex) rather than a true busy-wait lock. This changes the rules slightly: on a RT kernel a “spinlock” critical section can be preempted by a higher priority task, though it still must not call functions that are unsafe in a raw atomic context. If you need the old, strictly non-sleeping guarantee even on an RT kernel, use raw_spinlock_t instead.

• A spinlock protects short critical sections that must never sleep.
• The moment you hold a spinlock, your code runs in atomic context.
• Sleeping inside atomic context triggers the “scheduling while atomic” bug.
• CONFIG_DEBUG_ATOMIC_SLEEP turns this from a silent time bomb into an immediate, loud warning.
• On PREEMPT_RT kernels, regular spinlocks behave differently — use raw_spinlock_t when you truly need non-preemptible behaviour.

Spinlocks are one of the first synchronization primitives every kernel and driver developer learns, and they are also one of the easiest to misuse. The good news is that the mistake has a single, well-defined shape: something that can sleep ends up running while a lock that forbids sleeping is held. Once you internalise that one rule, and get into the habit of testing against a debug kernel, this entire class of bug becomes easy to catch before it ever reaches a customer. In the next lecture in this free linux kernel development course, we go one level deeper and trace exactly how the kernel’s tracing infrastructure lets you see this bug happen in real time.

Q1. What is a spinlock in the Linux kernel?
A spinlock is a locking primitive that makes a CPU busy-wait in a loop until a lock becomes free, used to protect very short critical sections that must never sleep.

Q2. What does “scheduling while atomic” mean?
It means the kernel tried to switch to another task while the current code was not allowed to be preempted, typically because a spinlock is held and something inside it tried to sleep.

Q3. Can I sleep while holding a spinlock?
No, never, on a standard mainline kernel. Doing so is a serious bug that can hang or crash the system.

Q4. What is the difference between spin_lock and spin_lock_irqsave?
spin_lock_irqsave() additionally saves the current interrupt state and disables local interrupts, which is required when the same lock is also taken from an interrupt handler.

Q5. When should I use a mutex instead of a spinlock?
Use a mutex whenever the protected code might need to sleep, for example while allocating memory with GFP_KERNEL or copying data to and from user space.

Q6. How do I detect this bug during development?
Enable CONFIG_DEBUG_ATOMIC_SLEEP and lockdep in your test kernel build; both will report the exact function where the violation happened.

Q7. Do spinlocks work the same way on PREEMPT_RT kernels?
No, on PREEMPT_RT builds a regular spinlock becomes a priority-inheriting sleeping lock internally; use raw_spinlock_t if you need the classic non-sleeping behaviour.

Q8. Is this lecture based on a real production bug?
The concept is a well-known and extremely common class of kernel bug; the code samples here are written fresh for this course to demonstrate the pattern clearly.