Part of the Free Linux Kernel Programming Course at EmbeddedPathashala — updated for Linux 6.x

The Fundamental Split: Userspace and Kernel Space

Every Linux system divides memory and execution privilege into two distinct worlds: userspace and kernel space. Understanding this split is the single most important mental model shift you need to make when moving from userspace C programming to free Linux kernel programming.

In userspace, your program runs in an isolated virtual address space. If it crashes, the kernel kills the process and the rest of the system continues running. The operating system protects all other processes and the hardware from your buggy code. This safety net is provided by the CPU’s memory protection hardware (MMU) and enforced by the kernel.

In kernel space, there is no safety net. You are the safety net. A bug in kernel code can crash the entire system, corrupt the filesystem, expose security vulnerabilities, or silently corrupt data in other processes. This is why the discipline, tools, and mental model of free Linux kernel development are fundamentally different from userspace programming.

Side-by-Side Comparison — Userspace vs Kernel Space Programming

This table captures the most important practical differences you will encounter as you move from userspace C programming into free Linux kernel and device driver development.

No Standard C Library in the Kernel — What to Use Instead

This is the most immediately jarring difference for userspace developers entering kernel programming. All the functions you rely on daily — printf(), malloc(), memcpy(), sprintf(), strlen(), fopen() — either do not exist in the kernel or have different names and slightly different behavior.

The kernel provides its own implementations of commonly needed functions. Here is the mapping:

/* Practical example: common stdlib functions replaced with kernel equivalents */
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/slab.h>      /* kmalloc, kfree */
#include <linux/string.h>    /* strlen, memcpy, strscpy */
#include <linux/delay.h>     /* msleep, udelay */

MODULE_LICENSE("GPL");

static int __init stdlib_diff_init(void)
{
	char *buf;
	char msg[] = "hello from kernel";
	size_t len;

	/* kmalloc — like malloc, but must specify memory flags */
	/* GFP_KERNEL: normal allocation, allowed to sleep */
	buf = kmalloc(64, GFP_KERNEL);
	if (!buf) {
		pr_err("demo: kmalloc failed\n");
		return -ENOMEM;  /* not -1 like userspace; direct errno */
	}

	/* strlen — same name, kernel implementation */
	len = strlen(msg);
	pr_info("demo: message length = %zu\n", len);

	/* strscpy — safer than strcpy, always null-terminates */
	strscpy(buf, msg, 64);
	pr_info("demo: buf = '%s'\n", buf);

	/* snprintf — always bounded, never overflows */
	snprintf(buf, 64, "kernel version: %s", UTS_RELEASE);
	pr_info("demo: %s\n", buf);

	/* msleep — like sleep(), but in milliseconds, only in sleepable context */
	msleep(10);  /* sleep for 10 milliseconds */

	/* kfree — like free() */
	kfree(buf);
	buf = NULL;  /* good practice: null the pointer after freeing */

	return 0;
}

static void __exit stdlib_diff_exit(void) { }

module_init(stdlib_diff_init);
module_exit(stdlib_diff_exit);

The Kernel Stack — Tiny, Fixed, and Precious

In userspace, each thread gets a stack that starts at 8 MB by default on Linux and can be extended. You can declare large arrays on the stack, have deep call chains, and generally not worry about stack size. In the kernel, this luxury does not exist.

Each kernel thread and each interrupt handler gets a stack of either 4 KB or 8 KB (depending on architecture and configuration). This stack is fixed — it cannot grow. If your kernel code overflows the stack, the result is silent memory corruption or a kernel panic.

Stack Usage Rules for Kernel Code

• Never declare large local arrays on the kernel stack. A char buf[4096] local variable in a kernel function uses half the entire stack on a 32-bit system. Use kmalloc() for anything larger than a few hundred bytes.
• Avoid deep recursion. Each function call frame takes stack space. A recursion depth of 10–15 in kernel code can cause stack overflow. Use iterative algorithms instead.
• Pass structures by pointer, not by value. Passing a large struct by value copies it onto the stack. In the kernel, always pass pointers.
• Enable stack overflow detection during development. The kernel has CONFIG_VMAP_STACK (enabled by default in modern kernels) which uses guard pages to detect stack overflow. Also enable CONFIG_KASAN for catching stack corruption.

/* BAD: large local buffer uses too much stack */
static int my_func(void)
{
	char big_buf[2048];  /* WRONG in kernel — 2KB on a 4-8KB stack */
	/* ... */
	return 0;
}

/* GOOD: allocate large buffers dynamically */
static int my_func(void)
{
	char *big_buf;
	int ret = 0;

	big_buf = kmalloc(2048, GFP_KERNEL);
	if (!big_buf)
		return -ENOMEM;

	/* use big_buf ... */

	kfree(big_buf);
	return ret;
}

/* BAD: passing large struct by value */
struct my_large_struct my_func_bad(struct my_large_struct s) { ... }

/* GOOD: pass by pointer always */
int my_func_good(struct my_large_struct *s) { ... }

Kernel Memory Allocation — GFP Flags and Allocation Contexts

Kernel memory allocation is more nuanced than userspace malloc() because the kernel must handle requests that arrive from different contexts — some can sleep and wait for memory, others cannot. This is controlled through GFP flags (Get Free Pages flags) passed to kmalloc() and related functions.

/* Memory allocation examples showing different contexts */
#include <linux/slab.h>
#include <linux/interrupt.h>

/* In a normal init function — process context — can sleep */
static int __init alloc_demo_init(void)
{
	void *buf;

	/* GFP_KERNEL: can sleep while waiting for free memory */
	buf = kmalloc(1024, GFP_KERNEL);
	if (!buf)
		return -ENOMEM;

	/* kzalloc = kmalloc + memset(0) — prefer this for safety */
	buf = kzalloc(256, GFP_KERNEL);
	if (!buf)
		return -ENOMEM;

	kfree(buf);
	return 0;
}

/* In an interrupt handler — atomic context — cannot sleep */
static irqreturn_t my_irq_handler(int irq, void *dev_id)
{
	void *buf;

	/* MUST use GFP_ATOMIC here — no sleeping allowed in IRQ handler */
	buf = kmalloc(64, GFP_ATOMIC);
	if (!buf) {
		/* GFP_ATOMIC can fail more often than GFP_KERNEL */
		pr_err_ratelimited("irq: kmalloc failed\n");
		return IRQ_HANDLED;
	}

	/* use buf briefly */
	kfree(buf);
	return IRQ_HANDLED;
}

Concurrency in the Kernel — More Complex Than Userspace

Concurrency in userspace is already challenging — you deal with threads, mutexes, and race conditions. In the kernel, the situation is significantly more complex because there are additional sources of concurrent execution that do not exist in userspace.

Sleeping vs Non-sleeping Contexts

One of the most important rules in kernel programming is knowing whether your code is allowed to sleep. In userspace, you can call sleep(), mutex_lock(), or any blocking call from anywhere. In the kernel, sleeping is only safe in specific contexts.

You cannot sleep in any of these situations:

• Inside an interrupt handler (hard IRQ context)
• Inside a softirq or tasklet
• While holding a spinlock
• While interrupts are disabled (local_irq_disable() called)
• In an NMI handler

You can sleep in these situations:

• In process context (module init/exit functions, char driver read/write callbacks, work queue handlers)
• While holding a mutex (but not a spinlock)

/* Checking if you can sleep — use in_interrupt() */
#include <linux/hardirq.h>

static void my_kernel_function(void)
{
	if (in_interrupt()) {
		/*
		 * We are in interrupt context — cannot sleep.
		 * Use GFP_ATOMIC for memory allocation.
		 * Use spinlocks, not mutexes.
		 * Do NOT call msleep(), mutex_lock(), or any blocking API.
		 */
		pr_warn("in interrupt context — atomic operations only\n");
	} else {
		/*
		 * We are in process context — can sleep.
		 * GFP_KERNEL is fine.
		 * Mutexes and semaphores are fine.
		 * msleep() and schedule() are fine.
		 */
		pr_info("in process context — can sleep\n");
	}
}

Error Handling Conventions — Kernel vs Userspace

Userspace functions typically signal errors by returning -1 and setting the global errno variable. The kernel does not use this convention. Instead, kernel functions return negative error codes directly. This eliminates the need for a thread-local global variable and makes the error path explicit in the return value.

/* Userspace convention */
int fd = open("/dev/somedevice", O_RDWR);
if (fd == -1) {
	perror("open failed");  /* reads errno */
	return -1;
}

/* Kernel convention — return negative errno directly */
#include <linux/errno.h>  /* error codes */

static int my_device_open(struct inode *inode, struct file *filp)
{
	/* Check some condition */
	if (device_is_busy()) {
		pr_err("my_device: device busy\n");
		return -EBUSY;  /* not -1; direct negative errno */
	}

	if (!have_permission()) {
		return -EACCES;  /* permission denied */
	}

	/* Allocation failure */
	struct my_data *data = kzalloc(sizeof(*data), GFP_KERNEL);
	if (!data)
		return -ENOMEM;  /* out of memory */

	filp->private_data = data;
	return 0;  /* success */
}

Common Kernel Error Codes

Debugging Kernel Code vs Userspace — A Different Mindset

Debugging userspace programs is relatively straightforward: attach gdb, set breakpoints, inspect variables, use valgrind for memory bugs. Debugging kernel code requires a completely different toolkit and mindset.

/* Primary kernel debugging technique: pr_debug with dynamic debug */
#include <linux/module.h>
#include <linux/kernel.h>

/* pr_debug compiles to nothing in production (no DEBUG defined)
 * but can be enabled at runtime via /sys/kernel/debug/dynamic_debug/control */
static void process_data(int val)
{
	pr_debug("process_data: val=%d\n", val);  /* zero overhead in production */

	if (val < 0) {
		pr_warn("process_data: negative value %d, clamping to 0\n", val);
		val = 0;
	}

	pr_debug("process_data: processing value %d\n", val);
}

/* Enable specific debug messages at runtime without recompiling:
 *
 * echo "file my_module.c +p" > /sys/kernel/debug/dynamic_debug/control
 *
 * This enables all pr_debug() calls in my_module.c
 */

Portability Considerations Unique to Kernel Code

The Linux kernel runs on dozens of CPU architectures — x86, ARM32, ARM64, RISC-V, MIPS, PowerPC, and more. Kernel code must be written with this diversity in mind. Userspace POSIX code has some portability concerns, but kernel code has more:

• Data type sizes vary by architecture: int is 32-bit everywhere, but long is 32-bit on 32-bit architectures and 64-bit on 64-bit ones. Use explicit-width types from <linux/types.h>: u8, u16, u32, u64, s32, s64.
• Page size varies: Always use PAGE_SIZE and PAGE_SHIFT macros, never hardcode 4096.
• Endianness varies: Use the kernel’s endianness macros: cpu_to_le32(), le32_to_cpu(), cpu_to_be32() etc. Mark hardware-facing values with __le32 or __be32 annotations for sparse checking.
• Alignment requirements vary: Some architectures fault on unaligned memory accesses. Use get_unaligned() and put_unaligned() when accessing potentially unaligned hardware data.

#include <linux/types.h>    /* u8, u16, u32, u64, __le32, __be32 */
#include <asm/byteorder.h>  /* cpu_to_le32, le32_to_cpu, etc. */
#include <linux/unaligned.h> /* get_unaligned, put_unaligned */

/* PORTABLE: explicit width types */
u32 register_value;          /* always 32-bit */
u64 byte_count;              /* always 64-bit */

/* PORTABLE: page size */
void *buf = kmalloc(PAGE_SIZE, GFP_KERNEL);  /* not kmalloc(4096,...) */

/* PORTABLE: endianness conversion for hardware registers */
/* Device sends data in little-endian format */
__le32 hw_data;              /* annotate LE data from hardware */
u32 cpu_val = le32_to_cpu(hw_data);  /* convert to CPU byte order */

/* Send to device in big-endian format */
u32 value = 0x12345678;
__be32 hw_out = cpu_to_be32(value);  /* convert to big-endian for hardware */

/* PORTABLE: unaligned access (safe on all architectures) */
u8 *packet_buf = get_data_ptr();
u32 word = get_unaligned((u32 *)(packet_buf + 1));  /* safe, even if misaligned */

Common Mistakes When Coming from Userspace to Kernel Programming

• Using printf instead of pr_info: The most common first mistake. printf() does not exist in the kernel. The compiler error (“undefined reference to printf”) is confusing if you do not know why.
• Using malloc instead of kmalloc: Same issue — malloc() is a libc function. The kernel has no libc.
• Forgetting to check kmalloc return: Unlike in well-guarded userspace code, a NULL dereference in the kernel is immediately fatal. Always check allocation results.
• Sleeping in interrupt context: Calling msleep() or mutex_lock() from an interrupt handler causes an immediate kernel panic with “BUG: scheduling while atomic”.
• Large stack allocations: Declaring char buf[4096] as a local variable in a kernel function. Always allocate large buffers dynamically with kmalloc().
• Assuming int is always 32-bit for addresses: On 64-bit systems, pointers are 64-bit. Never cast a pointer to int — use uintptr_t or unsigned long if you must store a pointer as an integer.
• Ignoring concurrency: Userspace developers often underestimate the concurrency in the kernel. A global variable in a kernel module can be accessed simultaneously by multiple CPUs, interrupt handlers, and softirqs. Always protect shared state with appropriate locking.

Key Takeaways

Frequently Asked Questions

Authoritative References

• Linux Kernel Memory Allocation Guide (kernel.org)
• Linux Kernel Core API: String Functions and Helpers (kernel.org)
• Kernel Address Sanitizer — KASAN (kernel.org)
• Linux Kernel Locking Primitives (kernel.org)