6.12+
Beginner to Intermediate
~14 min
If you are writing kernel modules or device drivers that run on today’s multi-core systems, you cannot avoid concurrency. Two CPUs, two interrupt handlers, or two kernel threads can touch the same piece of shared data at almost the same instant, and if you don’t protect that data correctly, you get corrupted state, crashes, or a system that simply freezes. This is exactly where solid Linux kernel locking guidelines come in. In this lecture, part of our free Linux kernel development course, we break down how experienced kernel developers reason about locks, why deadlocks happen, and how modern tooling in kernel 6.12 and later helps you catch these bugs before they ship.
This lecture is written for anyone following a free linux device drivers course or a broader free embedded systems course path, and it assumes you already know what a critical section is.
Prerequisites
- Basic understanding of what a critical section is (a code region that accesses shared data)
- Familiarity with writing a simple Linux kernel module (insmod / rmmod, printk)
- Some idea of what a mutex or a spinlock is used for (we cover the choice between them in the next lecture)
- A kernel 6.12 or later build environment if you want to try lockdep yourself (any recent Ubuntu, Fedora, or Yocto-based target works)
Why Locking Guidelines Matter in Kernel Development
Locking sounds simple on paper: take a lock, do your work, release the lock. The actual API calls in the Linux kernel are genuinely easy to use. The hard part is not the syntax — it is designing your locking scheme so that two different code paths never end up waiting on each other forever. Good kernel locking guidelines exist precisely because the failure mode of bad locking is not a clean crash with a stack trace — it is a silent hang that only shows up under load, days after your driver shipped.
1. Get the Locking Granularity Right
Locking granularity refers to how much code sits between a lock and its matching unlock — in other words, how “big” your critical section is.
| Approach | Benefit | Risk |
|---|---|---|
| Too few, coarse locks | Simple to reason about | Heavy contention, poor scalability on multi-core CPUs |
| Many, fine-grained locks | Better performance under concurrency | Harder to track which lock protects which data, higher deadlock risk |
The practical middle ground most production kernel code follows: one lock protects one clearly-owned shared data structure, and the lock is usually embedded as a member inside that structure itself. This makes ownership obvious to anyone reading the code later, including future you.
2. Document and Follow a Strict Lock Ordering
When a code path needs more than one lock, the order in which those locks are acquired must be identical everywhere in the codebase. This single rule prevents the majority of multi-lock deadlocks. Many kernel subsystems even document their lock hierarchy directly in the source comments so every contributor follows the same order.
3. Avoid Recursive Locking and Watch for Starvation
Do not try to acquire a lock your own code path already holds. Also make sure that once a lock is taken, the holder releases it quickly — a thread that sits on a lock too long can starve other waiters, especially on real-time configurations.
4. Keep the Design Simple
Complex, clever locking schemes are the ones that eventually deadlock. Simpler locking, even if it costs a small amount of performance, is usually the safer engineering trade-off in a kernel module or driver.
What Exactly Is a Deadlock?
A deadlock is a state where none of the involved threads can make forward progress — each one is waiting on a resource that another blocked thread is holding, forever. In kernel space this typically shows up as a hard system hang. Let’s walk through the common deadlock patterns you need to recognize.
Pattern 1: The Self-Deadlock
The simplest possible deadlock: a single thread tries to acquire a lock it is already holding. Since most kernel locks are not recursive by default, the thread blocks waiting for itself to release a lock it never will.
Pattern 2: The AB-BA Deadlock (Two Locks, Two CPUs)
Result: Both threads wait forever — classic AB-BA deadlock
This pattern is called AB-BA because Thread A takes lock order A-then-B while Thread B takes the reverse order B-then-A. It can also be extended across three or more locks — for example an A→B, B→C, C→A circular chain across three threads produces the exact same kind of standoff. This is why a single, globally documented lock order matters so much.
Pattern 3: Interrupt Context Deadlocks
This one catches many newcomers off guard. Suppose your process-context code takes a lock, and an interrupt fires on another core whose handler tries to take the same lock. If your original code path did not disable interrupts while holding that lock, the handler can preempt it and spin forever waiting for a lock the interrupted code can never release until the handler finishes. The fix is a locking primitive that also disables local interrupts for the duration of the critical section — something we cover hands-on in the next lecture on mutexes and spinlocks.
Livelock: The Sneaky Cousin of Deadlock
A livelock looks different from a deadlock but is just as harmful: the threads involved are technically still running, not blocked, but they make zero real progress — for example two threads repeatedly backing off and retrying in a way that keeps colliding. A well-known real-world example is an interrupt storm on a busy network interface; modern network drivers avoid this by switching to polling (NAPI) under heavy interrupt load instead of handling every single interrupt individually.
lockdep: The Kernel’s Built-In Deadlock Detector
lockdep is the Linux kernel’s runtime lock dependency validator. When enabled (CONFIG_PROVE_LOCKING), it tracks the order in which every lock in the system is acquired and released, builds a dependency graph in the background, and flags any ordering that could theoretically produce a deadlock — even if that exact deadlock has never actually happened yet on your test machine. This proactive detection is one of the most valuable debugging tools available to kernel and driver developers, and it remains a core part of kernel 6.12 and newer releases.
Enable it during development with:
CONFIG_PROVE_LOCKING=y
CONFIG_DEBUG_LOCKDEP=yAny driver you are actively developing should be tested on a lockdep-enabled kernel at least once before it goes anywhere near production hardware.
Locking on a PREEMPT_RT Kernel (6.12 and Later)
Kernel 6.12 mainlined the PREEMPT_RT real-time patches after roughly two decades of out-of-tree development. This is directly relevant to locking guidelines because on an RT-enabled kernel, ordinary spinlocks become sleeping locks internally, backed by priority-inheritance-aware primitives, while raw_spinlock_t keeps the traditional non-preemptible behavior. That means the same lock-ordering discipline you learn here now also affects scheduling latency, not just correctness — a badly ordered lock chain on an RT kernel can turn into a priority inversion problem, not just a deadlock. We go deeper into this distinction in the companion lecture on mutex versus spinlock.
Real-World Use Cases
A character device driver reading from a shared ring buffer while an interrupt handler writes into it needs disciplined locking with interrupts disabled during the critical section, or you get exactly the interrupt-context deadlock pattern described above.
The kernel networking stack has to protect route tables, socket buffers, and per-CPU queues simultaneously, which is why lock ordering documentation is taken extremely seriously in that subsystem.
Filesystems juggle inode locks, page locks, and journal locks together — a textbook case where a single undocumented lock order change has historically introduced real deadlocks in mainline kernel history.
Common Mistakes and Troubleshooting
| Mistake | What Happens | Fix |
|---|---|---|
| Taking locks in different order in two code paths | AB-BA deadlock under load | Document and enforce one global lock order |
| Holding a lock while calling code that may sleep | Can deadlock on preemptible/RT kernels | Keep critical sections short, avoid blocking calls inside them |
| Not disabling interrupts around a lock shared with an ISR | Interrupt-context deadlock | Use the irq-safe lock variant for that critical section |
| Ignoring lockdep warnings as “noise” | Real deadlock ships to production | Treat every lockdep splat as a real bug until proven otherwise |
Best Practices Checklist
- Keep critical sections as short as practically possible
- One lock, one clearly documented purpose — embed it in the struct it protects
- Write down and enforce a single lock acquisition order project-wide
- Never nest locks unless the order is documented and reviewed
- Always test new locking code with lockdep enabled
- On PREEMPT_RT targets, distinguish raw_spinlock_t use cases from ordinary spinlock_t use cases deliberately
Performance and Security Considerations
Performance: Over-locking (too coarse) kills scalability on multi-core SoCs common in embedded Linux boards; over-fragmenting locks (too fine) adds overhead and deadlock surface area. Measure contention with kernel tracing tools before optimizing blindly.
Security: A deadlocked kernel thread holding a lock can become a denial-of-service vector if an attacker can trigger the exact interleaving reliably — this is one more reason lockdep validation belongs in your CI pipeline for any driver exposed to untrusted input paths (like USB or network-facing drivers).
Summary / Key Takeaways
- Good locking granularity balances performance against complexity
- A single, documented lock ordering rule prevents most multi-lock deadlocks
- Self-deadlocks, AB-BA deadlocks, and interrupt-context deadlocks are the three patterns to watch for
- Livelock is a distinct failure mode from deadlock, often solved with polling techniques like NAPI
- lockdep is the single most valuable tool for catching these bugs early, and remains essential on kernel 6.12+
- PREEMPT_RT mainlining in 6.12 changes how spinlocks behave, adding a scheduling-latency dimension to your locking guidelines
Conclusion
Locking guidelines are not academic advice — they are the difference between a driver that survives stress testing and one that hangs a production device at 3 a.m. The good news is that the rules themselves are simple: keep critical sections tight, document your lock order, avoid recursion, and lean on lockdep instead of trusting your own judgment alone. Once you’re comfortable with these guidelines, the natural next step is understanding exactly when to reach for a mutex versus a spinlock, which is what we cover in the next lecture of this free Linux kernel development course.
Frequently Asked Questions
Q1. What is the difference between a deadlock and a livelock in the Linux kernel?
A deadlock leaves threads blocked and idle forever, while a livelock leaves threads actively running but making no real progress, often due to repeated collision and retry.
Q2. What causes an AB-BA deadlock?
It happens when two threads acquire the same two locks in opposite order — one takes Lock A then Lock B, the other takes Lock B then Lock A, and both end up waiting on each other.
Q3. Why does lock ordering matter so much in kernel development?
Because a single undocumented or inconsistent lock order is the root cause of the majority of real-world multi-lock kernel deadlocks.
Q4. What is lockdep and do I need it in production?
lockdep is the kernel’s runtime lock dependency validator that flags risky lock orderings. It is meant for development and testing kernels, not typically enabled on production systems due to overhead.
Q5. Does PREEMPT_RT change how I should think about locking?
Yes — on a PREEMPT_RT kernel most spinlocks become sleeping, priority-inheritance-aware locks, so lock ordering mistakes can also cause priority inversion and latency spikes, not just deadlocks.
Q6. Can a single thread deadlock itself?
Yes, this is called a self-deadlock, and it happens when a thread tries to reacquire a non-recursive lock it already holds.
Q7. What is NAPI and how does it relate to livelock?
NAPI (New API) is a technique used by network drivers to switch from pure interrupt-driven handling to polling under heavy load, preventing the livelock caused by interrupt storms.
Q8. Is this locking guidance still valid for the latest kernel releases?
Yes, the core principles are stable kernel design fundamentals; this lecture is updated to reflect kernel 6.12+ realities including the mainlined PREEMPT_RT support.
2 Comments