What You Will Learn

Prerequisites

How the Linux Kernel Organises Its Virtual Address Space

Every process running on a Linux system has its own Virtual Address Space (VAS). This VAS is divided into two regions: user space (where your application code runs) and kernel space (where the kernel itself resides). Kernel space is shared across all processes — the same kernel code and kernel data structures are mapped into the top portion of every process’s address space.

Within kernel space, the kernel must track and manage every thread that is currently alive on the system — both user-mode threads (threads belonging to user-space applications) and kernel threads (threads created entirely within the kernel itself, such as kworker, ksoftirqd, or kthreadd). Each of these threads requires dedicated resources inside kernel space, and the two most important ones are:

• A task structure — struct task_struct — which stores all metadata about the thread
• A kernel-mode stack — a small, fixed-size region of kernel memory used whenever that thread executes kernel code

How Many Stacks Does a Running Linux System Have?

This is a question that surprises most people when they first encounter it. Let’s reason through it carefully because this is fundamental to understanding how the Linux kernel manages every thread on a live system.

Consider a snapshot of a running system where:

• There are 1,053 user-mode threads (threads belonging to user-space processes)
• There are 181 kernel threads (threads that live entirely in kernel space)

This gives a total of 1,234 threads on the system. How many stacks does this require?

The key insight here is that every user-mode thread carries two stacks: one in user space (for executing user-space code) and one in kernel space (for executing kernel code on behalf of that thread — for example, during a system call). Kernel threads, on the other hand, live entirely in kernel space and have no user-space mapping at all, so they only get a single kernel-mode stack.

The task_struct — The Kernel’s Control Block for Every Thread

For every thread alive on the system — whether it is a user thread or a kernel thread — the Linux kernel maintains a data structure called struct task_struct. This is essentially the kernel’s “control block” for a thread. It stores everything the kernel needs to know about that thread: its state, its priority, its open file descriptors, its memory mappings, its signal handlers, and much more.

From our example system snapshot:

• There are 1,234 task structures in kernel memory — one per thread
• 1,053 of these represent user threads
• 181 of these represent kernel threads

What is a Kernel-Mode Stack and How Does it Work?

Every time a user-mode thread enters kernel space — through a system call, a hardware interrupt, a page fault, or any other exception — the CPU switches from using the thread’s user-mode stack to its dedicated kernel-mode stack. This is a fundamental mechanism of the x86 and ARM64 privilege level switch.

Key Properties of the Kernel-Mode Stack in Linux 6.x

Property	32-bit Linux	64-bit Linux (x86_64, ARM64)
Size	2 pages (typically 8 KB)	4 pages (typically 16 KB)
Allocation time	At thread creation — during kernel_clone() (previously _do_fork() in older kernels)
Growth direction	Downward — toward lower virtual addresses (same as user-mode stacks)
Size at runtime	Fixed — cannot grow dynamically
Stack Pointer register	ESP (32-bit)	RSP on x86_64 / SP on ARM64

How a Stack Frame is Built on the Kernel Stack

Each time a function is called inside the kernel, the CPU sets up a stack frame on the kernel-mode stack. A stack frame contains:

• The return address (so the CPU knows where to jump back after the function returns)
• Saved registers (caller-saved or callee-saved, depending on the ABI)
• Local variables declared inside that function
• Any arguments passed on the stack (architecture-dependent)

Why is the Kernel-Mode Stack So Small? The Risks for Driver Developers

The kernel-mode stack is intentionally kept small. This is a deliberate design decision with important reasons:

• There can be thousands of kernel-mode stacks alive at any moment (one per thread). If each were large, the total memory footprint would be enormous.
• Kernel code is expected to be lean and efficient — deep recursion or large local arrays on the kernel stack are design mistakes, not features.
• A small stack encourages kernel developers to allocate large buffers on the heap (using kmalloc() or vmalloc()) instead of the stack.

Catching Stack Overflow Risk at Compile Time: CONFIG_FRAME_WARN

The Linux kernel build system includes a configurable option called CONFIG_FRAME_WARN that instructs GCC to emit a warning whenever a single function’s stack frame exceeds a specified size. This is an important safety net for kernel and driver developers.

# In your kernel .config or via menuconfig:
# Kernel hacking → Compile-time checks and compiler options → Warn for stack frames larger than (bytes)

CONFIG_FRAME_WARN=1024   # Warn if any stack frame exceeds 1024 bytes

When enabled, GCC will produce a build-time warning like:

drivers/mydriver/mydriver.c:128:1: warning: the frame size of 1280 bytes is larger than 1024 bytes [-Wframe-larger-than=]

What to Do When You Need Large Buffers in Kernel Code

/* ❌ Wrong — large buffer on kernel stack (dangerous!) */
static int my_driver_read(struct file *f, char __user *buf, size_t count, loff_t *pos)
{
    char tmp[4096];   /* 4 KB on a 16 KB stack — very risky */
    /* ... */
}

/* ✅ Correct — dynamically allocated on the kernel heap */
static int my_driver_read(struct file *f, char __user *buf, size_t count, loff_t *pos)
{
    char *tmp;

    tmp = kmalloc(4096, GFP_KERNEL);
    if (!tmp)
        return -ENOMEM;

    /* ... use tmp ... */

    kfree(tmp);
    return 0;
}

The Per-CPU Interrupt Stack — A Third Type of Stack in the Linux Kernel

Beyond user-mode stacks and kernel-mode stacks, modern Linux on architectures like x86_64 and ARM64 also maintains a per-CPU interrupt stack. This is a separate, dedicated stack used specifically when a hardware interrupt is being handled.

Why Does the Interrupt Stack Exist?

When a hardware interrupt fires — say, from a network card or a disk controller — the CPU’s interrupt mechanism takes control away from whatever thread was running and begins executing the interrupt service routine (ISR). If this ISR were to use the interrupted thread’s kernel-mode stack, it would put additional pressure on an already small 16 KB stack. In deeply nested interrupt scenarios, this could easily overflow.

The solution: each CPU has its own dedicated interrupt stack that is separate from any thread’s kernel-mode stack. When an interrupt fires, the CPU switches to this per-CPU interrupt stack for the duration of the interrupt handling.

Complete Summary: Stacks and Task Structures on a Linux System

Best Practices for Linux Kernel and Driver Stack Management

• Never allocate large local arrays on the kernel stack. Use kmalloc() or vmalloc() for buffers larger than a few hundred bytes.
• Avoid deep recursion in kernel code. Convert recursive algorithms to iterative ones if they will be called in kernel context.
• Enable CONFIG_FRAME_WARN in your kernel build configuration during development and fix all warnings before shipping a driver.
• Use CONFIG_KASAN (Kernel Address Sanitizer) during development to detect stack overflows and other memory errors early.
• Keep interrupt handlers short. If heavy processing is needed, defer it using workqueues or tasklets — which get their own kernel stacks — rather than doing it directly in the interrupt handler.
• Check /proc/<pid>/stack during debugging to see the current kernel-mode stack of any thread (requires root and appropriate kernel configuration).

Common Mistakes Made by Kernel and Driver Developers

Key Takeaways

Conclusion

Understanding how Linux manages kernel space stacks is foundational knowledge for anyone pursuing Linux kernel programming or Linux device driver development. The kernel-mode stack underpins every system call, every interrupt handler, and every line of code in your kernel module. Its small, fixed size is both a constraint and a discipline that pushes kernel developers toward clean, efficient code.

In the next article in this free Linux kernel development course, we will look at how to actually view and inspect both the kernel-mode and user-mode stacks of any running thread using the /proc filesystem and modern eBPF-based tools.

Frequently Asked Questions (FAQ)

References and Further Reading

• Linux Kernel Documentation: x86 Kernel Stacks
• Linux Kernel Core API Documentation
• struct task_struct definition — Elixir Linux Cross-Reference