A kernel module runs inside the kernel itself, not in a normal application process. It has access to every memory address, every hardware register, and every kernel data structure. If any user on the system could load a module freely, any malicious program could take complete control of the machine in seconds.

This is why Linux enforces a strict rule: only a process with the CAP_SYS_MODULE capability can load or unload kernel modules. On a typical system, only the root user (UID 0) has this capability by default.

Linux does not just have two modes — root and non-root. Modern Linux uses a system called POSIX Capabilities which splits the power of root into about 40 individual units. Each unit can be granted or taken away independently.

CAP_SYS_MODULE is one such unit. A process that has this capability can call init_module() or finit_module() (the system calls behind insmod) and delete_module() (behind rmmod). Without this capability, the system call returns -EPERM immediately.

You can inspect the capabilities of any process by reading /proc/<pid>/status and looking at the CapEff field. You can decode it with the capsh tool:

capsh --decode=$(grep CapEff /proc/self/status | awk '{print $2}')

Even if you have root access, the kernel has a one-way security switch at /proc/sys/kernel/modules_disabled. It is set to 0 by default, meaning module loading is allowed. If a system administrator writes 1 to it, module loading and unloading are both permanently disabled until the next reboot.

# Check current value
cat /proc/sys/kernel/modules_disabled

# Disable module operations permanently (until reboot)
echo 1 | sudo tee /proc/sys/kernel/modules_disabled

# Trying to set it back to 0 will fail
echo 0 | sudo tee /proc/sys/kernel/modules_disabled
# Output: tee: /proc/sys/kernel/modules_disabled: Invalid argument

This is a hardening technique used on production servers. After all required modules are loaded at boot, this switch is flipped to prevent any attacker from loading a malicious module even if they gain root access.

Modern Linux distributions go further than just checking for root. They enforce module signatures. Every loadable .ko file must be cryptographically signed, and the kernel verifies the signature before allowing the load.

You can check whether your kernel enforces signatures:

cat /sys/module/module/parameters/sig_enforce

If it prints Y, unsigned modules will be refused with an ENOKEY error. If it prints N, the kernel still prints a taint warning but allows loading.

From Linux 5.4 onward, the kernel also includes the Lockdown LSM. You can see the active lockdown level:

cat /sys/kernel/security/lockdown

The output shows three modes. The active one is in square brackets, for example [none] integrity confidentiality. Under integrity mode, unsigned modules are blocked even if you have root. This mode is automatically activated when UEFI Secure Boot is enabled on x86 and ARM64 machines.

The insmod command is the simplest way to load a kernel module. You give it the path to the .ko file and it loads that module into the running kernel.

sudo insmod ./helloworld_lkm.ko

If the command succeeds, it exits silently and returns exit code 0. You can verify with:

echo $?

If it fails, it returns a non-zero exit code. The actual error reason is printed to the kernel log and you can read it with dmesg.

When you type sudo insmod ./mod.ko, a whole chain of events takes place. Here is the step-by-step flow:

There are two kernel system calls for loading modules. Old books and older kernels mention init_module(), which takes a memory buffer. Modern insmod (from the kmod package) now uses finit_module() instead, which takes an open file descriptor to the .ko file.

The reason finit_module() is preferred is security: since it loads from a file path the kernel can verify the module’s signature from the actual file, not a potentially tampered memory copy.

When insmod fails, always read dmesg for the detailed reason. The error message from insmod itself is often too generic to be useful.

In your user-space programs you use printf() to print messages to the terminal. But inside the kernel there are no libraries, so printf() is not available. Instead, the kernel provides its own logging function called printk().

However, here is the key insight for modern kernel programming: you should almost never write raw printk() in your code. Instead, use the pr_info(), pr_err(), and other pr_*() wrapper macros. They are cleaner, shorter, and they automatically add your module name to the message.

Every message printed with printk() must have a log level. The log level tells the kernel how important the message is. There are 8 levels, numbered 0 (most critical) to 7 (least critical / debug). Think of it like this: lower number = higher urgency.

These levels are defined in <linux/kern_levels.h> and have not changed in kernel 6.x.

Here is the old way using raw printk(). It works but is not the recommended style in kernel 6.x:

/* Old style - works but not recommended */
printk(KERN_INFO "Hello from my module\n");
printk(KERN_ERR "Something went wrong: %d\n", error_code);

Here is the modern, correct way using pr_*() macros. This is how actual Linux kernel code is written today:

/* Add this at the very top of your .c file, before any #include */
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt

#include <linux/init.h>
#include <linux/module.h>
#include <linux/kernel.h>

MODULE_LICENSE("GPL");

static int __init mymod_init(void)
{
    pr_info("Module loaded successfully\n");
    pr_debug("Debug: init function called\n");
    return 0;
}

static void __exit mymod_exit(void)
{
    pr_info("Module unloaded\n");
}

module_init(mymod_init);
module_exit(mymod_exit);

With pr_fmt defined, every pr_*() message automatically gets prefixed with your module name, so in the kernel log you will see something like:

[  123.456789] mymod: Module loaded successfully

This makes it very easy to spot your module’s messages in a busy kernel log.

pr_debug() is special. By default, it is completely compiled out of your module — it generates zero machine code unless you explicitly enable it. This means debug messages have no performance cost in production builds.

There are two ways to enable pr_debug() output:

Method 1: Add -DDEBUG to your Makefile:

ccflags-y += -DDEBUG

Method 2: Use Dynamic Debug (if your kernel has CONFIG_DYNAMIC_DEBUG=y). This lets you enable/disable debug messages at runtime without recompiling:

# Enable pr_debug for a specific module at runtime
echo "module mymod +p" | sudo tee /sys/kernel/debug/dynamic_debug/control

# Disable it again
echo "module mymod -p" | sudo tee /sys/kernel/debug/dynamic_debug/control

Dynamic Debug is extremely powerful for debugging drivers in production without rebooting.

Imagine your driver detects a hardware error 1000 times per second. If you use pr_err() naively, you will flood the kernel log and potentially slow down the system. Kernel 6.x provides two solutions:

Rate-limited macros — print the message at most a few times per second:

/* Only prints at most once every 5 seconds */
pr_err_ratelimited("Hardware error detected: %d\n", err_code);

One-shot macros — print the message only the first time it is reached:

/* Prints exactly once per boot, no matter how many times this line runs */
pr_info_once("Driver initialized for the first time\n");

Use _ratelimited variants in interrupt handlers or hot paths. Use _once for initialization status messages that do not need to be repeated.

Unlike printf() which writes to the terminal, printk() does not have a single output destination. It writes to multiple places simultaneously:

The kernel ring buffer is a fixed-size circular buffer in RAM. It is always written first. Since it is in RAM, it is lost on reboot unless a persistent journal is configured. The size is controlled by the kernel config CONFIG_LOG_BUF_SHIFT.

The console only receives messages whose log level number is smaller than the current console_loglevel. Check and change it with:

# See four values: current, default, minimum, boot-time default
cat /proc/sys/kernel/printk
# Example output: 4  4  1  7
# This means only levels 0-3 (EMERG to ERR) go to console

# Temporarily print everything (level 0-7) to console
echo 8 | sudo tee /proc/sys/kernel/printk

When you are working in a graphical desktop session, you will not see kernel messages on screen even if they are urgent — you need to use dmesg or journalctl -k to read them.

The dmesg command reads the kernel ring buffer and prints it to your terminal. It is the primary tool for seeing what is happening inside the kernel — including all your pr_info() and pr_err() messages from your module.

On modern systems you usually need sudo to run dmesg, because kernel.dmesg_restrict=1 is the default security setting:

sudo dmesg

# See last 20 lines (most recent messages)
sudo dmesg | tail -n 20

# Show human-readable timestamps (VERY useful)
sudo dmesg -T

# Follow live like "tail -f" (watch kernel messages in real time)
sudo dmesg -w

# Filter only error and warning messages
sudo dmesg --level=err,warn

# Filter only messages from your module
sudo dmesg | grep mymod

# Clear the kernel log buffer
sudo dmesg -C

# Show with decoded log level labels
sudo dmesg -x

For kernel module development, this is the command sequence to use after each test:

sudo insmod ./mymod.ko && sudo dmesg -T | tail -n 10

By default, each dmesg line starts with a timestamp in square brackets. It looks like this:

[  412.880797] mymod: Module loaded successfully

The number 412.880797 means 412 seconds and 880797 microseconds have passed since the system was booted. This is a monotonic clock reading, not a wall-clock time.

To see actual wall-clock time (hour:minute:second format), use the -T flag:

sudo dmesg -T | tail -n 5
# Output example:
# [Wed Jun 18 10:32:15 2025] mymod: Module loaded successfully

Always use dmesg (which reads /dev/kmsg) for day-to-day kernel development. Avoid reading /proc/kmsg directly as it interferes with the system logger.

The dmesg ring buffer is lost on reboot. If your system crashes and you want to see what kernel messages appeared before the crash, you need a persistent log. On systems using systemd, the journal can be made persistent and then you can read previous boot kernel logs with:

# Show kernel messages from current boot (same as dmesg)
journalctl -k

# Show kernel messages from the previous boot (very useful after a crash)
journalctl -k -b -1

# Show kernel messages from two boots ago
journalctl -k -b -2

# Show kernel messages and follow live
journalctl -k -f

To enable persistent journal storage (if not already enabled):

sudo mkdir -p /var/log/journal
sudo systemctl restart systemd-journald

Once you load a module with insmod, the kernel keeps it in memory until you explicitly remove it. At any time you can check which modules are currently loaded.

The lsmod command lists all loaded kernel modules. It reads /proc/modules and formats it into three readable columns:

lsmod

Sample output:

Module                  Size  Used by
mymod                  16384  0
bluetooth             737280  31 btusb,bnep,btrtl
usbcore               262144  5 btusb,xhci_hcd,usbhid

The three columns mean:

To check if your specific module is loaded:

lsmod | grep mymod

The lsmod command is just a friendly viewer of /proc/modules. You can read it directly to see more columns:

cat /proc/modules

Each line has six fields:

mymod 16384 0 - Live 0x0000000000000000

The sysfs filesystem at /sys/module/ gives you detailed information about each loaded module. Each module gets its own directory:

ls /sys/module/mymod/

You will typically see entries like:

# Check if module is live
cat /sys/module/mymod/initstate

# Check reference count
cat /sys/module/mymod/refcnt

# Read module parameters (if any are defined)
ls /sys/module/mymod/parameters/
cat /sys/module/mymod/parameters/my_param

# Check which modules depend on this one
ls /sys/module/mymod/holders/

The parameters/ subdirectory is especially useful: some module parameters can be changed at runtime by writing to these files, without reloading the module.

To remove a module from the kernel, use rmmod with the module name (not the file path):

sudo rmmod mymod

This calls your module_exit() function and then frees the module’s memory. You can verify it is gone with lsmod | grep mymod.

If the module’s reference count (use count) is greater than zero, rmmod will refuse to remove it and return EWOULDBLOCK. This is a safety measure: it means something is still using the module. You must first stop or unload whatever is using it.

# Check reference count before removing
lsmod | grep mymod
# Module   Size  Used by
# mymod   16384  0       <-- 0 means safe to remove

# Then remove
sudo rmmod mymod && dmesg | tail -n 5

Here is the complete sequence to write, build, load, test, and remove a kernel module on Linux kernel 6.x. Go through this step by step in your VM.

Step 1: Write the module (helloworld.c)

#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt

#include <linux/init.h>
#include <linux/module.h>
#include <linux/kernel.h>

MODULE_LICENSE("GPL");
MODULE_AUTHOR("Your Name");
MODULE_DESCRIPTION("Hello World LKM for Linux 6.x");

static int __init helloworld_init(void)
{
    pr_info("Hello, World! Module loaded.\n");
    pr_info("Kernel version: %s\n", UTS_RELEASE);
    return 0;
}

static void __exit helloworld_exit(void)
{
    pr_info("Goodbye, World! Module unloaded.\n");
}

module_init(helloworld_init);
module_exit(helloworld_exit);

Step 2: Write the Makefile

obj-m := helloworld.o

all:
	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules

clean:
	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

Step 3: Build, load, verify, remove

# Build
make

# Load
sudo insmod ./helloworld.ko

# Verify it is loaded
lsmod | grep helloworld

# Read kernel messages
sudo dmesg -T | tail -n 5

# Inspect via sysfs
cat /sys/module/helloworld/initstate

# Remove
sudo rmmod helloworld

# Verify removal and see exit message
sudo dmesg -T | tail -n 5

Q1: What Linux capability is required to load a kernel module, and why?

Answer: CAP_SYS_MODULE is required. A kernel module runs at ring 0 (kernel privilege level), meaning it can access all memory and all hardware. If any unprivileged user could load modules, the system would have zero security. Even having root access (UID 0) is technically about having this capability assigned; the POSIX capabilities model makes this explicit.

Q2: What is the difference between insmod and modprobe?

Answer: insmod loads a single .ko file that you specify by file path. It does not resolve dependencies — if your module depends on another module that is not already loaded, insmod will fail with ENOENT or similar. modprobe is smarter: it reads /lib/modules/$(uname -r)/modules.dep to automatically load all dependencies first, then loads your module. For development, insmod is simpler. For deployment, modprobe is preferred.

Q3: What system call does modern insmod actually use — init_module or finit_module?

Answer: Modern insmod (from the kmod package) uses finit_module() by default. It opens the .ko file and passes the file descriptor to the kernel. It only falls back to the older init_module() (which takes a memory buffer) if finit_module() returns ENOSYS, meaning the running kernel is very old. finit_module() is preferred because it supports module signature verification from the file and in-kernel decompression.

Q4: What is the difference between printk() and pr_info()?

Answer: printk(KERN_INFO "message\n") and pr_info("message\n") produce identical output. The pr_info() macro is simply a wrapper that expands to the printk call with the right log level. The key advantage of pr_info() is that it also applies the pr_fmt() macro, which lets you define a prefix (like the module name) once at the top of the file and have it automatically added to every message. All modern Linux kernel code prefers pr_*() macros over raw printk().

Q5: Why is pr_debug() not visible by default, and how do you enable it?

Answer: pr_debug() expands to a no-op (empty statement) when the DEBUG</code > macro is not defined. This means debug messages have zero performance cost in production. You can enable them by adding ccflags-y += -DDEBUG in your Makefile (compile-time). Alternatively, if the kernel is built with CONFIG_DYNAMIC_DEBUG=y, you can enable or disable debug messages at runtime using the /sys/kernel/debug/dynamic_debug/control interface without recompiling.

Q6: What does “kernel taint” mean and when does loading a module cause it?

Answer: A “tainted” kernel is one that has been modified in a way the community cannot fully trust or support. Loading an out-of-tree (not part of the mainline kernel source) or unsigned module taints the kernel. The taint flag shows up in the kernel log as “loading out-of-tree module taints kernel” and can be seen in /proc/sys/kernel/tainted. A tainted state does not stop the system from working, but kernel developers may decline to help diagnose bugs on a tainted kernel because an untrusted module may have caused the problem.

Q7: What happens if your module_init() function returns a non-zero value?

Answer: If module_init() returns any non-zero value (a negative errno like -ENOMEM), the kernel aborts the load. The module is never made live, its memory is freed, and the negative value is returned to insmod as the error code. The module_exit() function is not called in this case. This is an important rule: your exit function must only clean up resources that were successfully allocated during init. If init failed halfway, you need to clean up only what was allocated before the failure point.

Q8: How do you check if a specific kernel module is loaded, and how do you see its parameters?

Answer: To check if a module is loaded: lsmod | grep module_name or cat /sys/module/module_name/initstate (returns “live” if loaded). To see module parameters: ls /sys/module/module_name/parameters/ lists all parameters, and each file in that directory holds the current value. Some parameters are read-only and some can be changed at runtime by writing to the file with root permission. You can also use modinfo module_name.ko to see what parameters a module accepts before loading it.

Q9: What is the Kernel Lockdown LSM and how does it affect module loading?

Answer: The Kernel Lockdown LSM (Linux Security Module) was introduced in Linux 5.4. It restricts what even root can do to protect kernel integrity. Under integrity lockdown mode, unsigned kernel modules cannot be loaded regardless of other settings. Under confidentiality mode, kernel memory cannot be read either. On x86 and ARM64 machines, lockdown is automatically set to integrity mode when UEFI Secure Boot is active. You can check the current level with cat /sys/kernel/security/lockdown.

Q10: What is the difference between dmesg and journalctl -k?

Answer: Both show kernel messages, but they differ in persistence. dmesg reads the in-memory kernel ring buffer, which is lost on reboot. journalctl -k reads kernel messages stored in the systemd journal, which can be made persistent across reboots. The most powerful feature of journalctl -k is journalctl -k -b -1, which shows kernel messages from the previous boot — essential for diagnosing kernel panics or crashes that caused a reboot.