How to Write an LKM to Inspect Kernel Memory? – Free Linux Device Driver Training

Kernel Segment Macros & Writing an LKM to Inspect Kernel Memory
Free Linux Kernel Development Course — EmbeddedPathashala
Level
Intermediate
Topic
Kernel Macros + LKM
Course
Free Linux Kernel

What You Will Learn

This is the hands-on companion to the previous lecture on kernel segment layout. In this free Linux device drivers course tutorial, you will go from theory to practice by learning the actual kernel macros that describe each memory region, and then writing a real kernel module (LKM) that reads and prints these values on your own machine. You will learn:

  • The key kernel macros and global variables that describe segment region boundaries
  • What PAGE_OFFSET, FIXADDR_START, MODULES_VADDR, VMALLOC_START and others mean
  • How architecture-dependent these values are and why
  • How to write, compile, insert, and inspect a kernel module that reads these macros
  • How to safely view kernel memory layout information from user space tools
  • Common pitfalls when working with kernel addresses in module code
  • Best practices for kernel memory safety in your modules
Prerequisites

You should have completed the previous lecture on kernel segment layout, and also:

Basic LKM (Hello World kernel module) printk() usage Kernel build environment set up Understanding of PAGE_OFFSET concept

Kernel Macros and Variables That Describe Segment Regions

In any free Linux kernel development course, you eventually need to know which macros actually encode the segment layout. The kernel does not hard-code address numbers everywhere — instead, it defines named macros and global variables that hold the boundary addresses of each region. Your module code should always use these names rather than hard-coded numbers, because the actual values change between architectures, kernel versions, and even boot sessions (due to KASLR).

Here is a reference table of the most important ones, organized from the highest KVA downward:

Key Kernel Macros and Variables — Kernel Segment Layout
Macro / Variable Region What It Represents
VECTORS_BASE Vector Table ARM-32 only. Start KVA of the 1-page interrupt/exception vector table.
FIXADDR_START Fix Map Start KVA of the fix map region. Size is FIXADDR_SIZE bytes. Compile-time virtual addresses are allocated here.
VMALLOC_START vmalloc Start KVA of the vmalloc/ioremap region. Dynamically allocated virtual memory for vmalloc() and ioremap() comes from here.
VMALLOC_END vmalloc End KVA of the vmalloc/ioremap region.
MODULES_VADDR Modules Start KVA of the kernel modules region. LKM text and data are loaded here.
MODULES_END Modules End KVA of the kernel modules region.
PAGE_OFFSET Lowmem Start KVA of the direct-mapped lowmem region. This is where physical RAM begins in kernel virtual space. Equal to 0xC0000000 on 32-bit 3:1 split.
high_memory Lowmem boundary Global variable (not a macro). Holds the KVA of the top of the lowmem region. Anything above this in physical RAM is in ZONE_HIGHMEM (32-bit only).
_text, _etext
_data, _edata
_bss_stop
Kernel image sections Linker-defined symbols marking the start and end of the kernel’s own text (code), data, and BSS sections within the lowmem region.
⚠ Architecture Dependency Warning

Not every macro listed above is defined on every architecture. For example, VECTORS_BASE exists only on ARM-32. MODULES_VADDR may not be defined on all platforms. Always guard your module code with proper #ifdef checks or check the kernel headers for your target platform before using a specific macro.

Writing a Kernel Module to Read Segment Information

Now let us build a practical kernel module. This module reads several of the macros above and prints them to the kernel ring buffer via printk(). You can then view the output with dmesg. This is one of the most instructive exercises in any free Linux kernel development course because it shows you the real numbers on your real hardware.

Step 1: Set Up Your Kernel Build Environment

Make sure you have the kernel headers installed for your running kernel:

# Ubuntu / Debian
sudo apt install linux-headers-$(uname -r) build-essential

# Fedora / RHEL
sudo dnf install kernel-devel kernel-headers

# Verify headers are in place
ls /lib/modules/$(uname -r)/build

Step 2: Create the Module Source File

Create a file called show_kernel_segment.c. The module reads key macros and prints them in a readable format. Note that we print sizes in MB to make the output human-friendly:

// show_kernel_segment.c
// EmbeddedPathashala - Free Linux Kernel Development Course
// Reads and prints key kernel segment region boundaries.
// Tested on Linux kernel 6.x (x86_64 and ARM64)

#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/mm.h>          // high_memory
#include <linux/vmalloc.h>     // VMALLOC_START, VMALLOC_END

MODULE_LICENSE("GPL");
MODULE_AUTHOR("EmbeddedPathashala");
MODULE_DESCRIPTION("Print Linux kernel segment region layout");
MODULE_VERSION("1.0");

// Helper: convert bytes to MB (unsigned long)
static inline unsigned long bytes_to_mb(unsigned long bytes)
{
    return bytes >> 20;
}

static int __init show_kernel_seg_init(void)
{
    pr_info("=== EmbeddedPathashala: Kernel Segment Layout ===\n");

    // PAGE_OFFSET: start of kernel segment (lowmem base)
    pr_info("PAGE_OFFSET (lowmem start) : 0x%016lx\n",
            (unsigned long)PAGE_OFFSET);

    // high_memory: top of the direct-mapped lowmem region
    pr_info("high_memory (lowmem end)   : 0x%016lx\n",
            (unsigned long)high_memory);

    pr_info("Lowmem size                : %lu MB\n",
            bytes_to_mb((unsigned long)high_memory - PAGE_OFFSET));

    // vmalloc region
#ifdef VMALLOC_START
    pr_info("VMALLOC_START              : 0x%016lx\n",
            (unsigned long)VMALLOC_START);
    pr_info("VMALLOC_END                : 0x%016lx\n",
            (unsigned long)VMALLOC_END);
    pr_info("vmalloc region size        : %lu MB\n",
            bytes_to_mb(VMALLOC_END - VMALLOC_START));
#endif

    // Kernel modules region
#ifdef MODULES_VADDR
    pr_info("MODULES_VADDR              : 0x%016lx\n",
            (unsigned long)MODULES_VADDR);
    pr_info("MODULES_END                : 0x%016lx\n",
            (unsigned long)MODULES_END);
    pr_info("Modules region size        : %lu MB\n",
            bytes_to_mb(MODULES_END - MODULES_VADDR));
#endif

    // Kernel image section symbols (defined by the linker script)
    pr_info("Kernel text  _text         : 0x%016lx\n",
            (unsigned long)_text);
    pr_info("Kernel text  _etext        : 0x%016lx\n",
            (unsigned long)_etext);
    pr_info("Kernel text size           : %lu kB\n",
            (unsigned long)(_etext - _text) / 1024);

    pr_info("Kernel data  _sdata        : 0x%016lx\n",
            (unsigned long)_sdata);
    pr_info("Kernel data  _edata        : 0x%016lx\n",
            (unsigned long)_edata);

    pr_info("Kernel BSS   __bss_start   : 0x%016lx\n",
            (unsigned long)__bss_start);
    pr_info("Kernel BSS   __bss_stop    : 0x%016lx\n",
            (unsigned long)__bss_stop);

    pr_info("=== End of kernel segment layout ===\n");
    return 0;   // 0 = success, module stays loaded
}

static void __exit show_kernel_seg_exit(void)
{
    pr_info("EmbeddedPathashala: show_kernel_segment module removed\n");
}

module_init(show_kernel_seg_init);
module_exit(show_kernel_seg_exit);

Step 3: Write the Makefile

Create a Makefile in the same directory:

# Makefile for show_kernel_segment module
obj-m += show_kernel_segment.o

# Point to the running kernel's build directory
KDIR := /lib/modules/$(shell uname -r)/build

all:
	make -C $(KDIR) M=$(PWD) modules

clean:
	make -C $(KDIR) M=$(PWD) clean

Step 4: Build and Insert the Module

# Build
make

# Insert the module
sudo insmod show_kernel_segment.ko

# View the output
dmesg | grep -A 30 "EmbeddedPathashala"

# Remove the module when done
sudo rmmod show_kernel_segment

Understanding the Output

On a modern 64-bit x86_64 Linux 6.x system, you will see something similar to this (exact addresses differ due to KASLR):

[  142.301234] === EmbeddedPathashala: Kernel Segment Layout ===
[  142.301240] PAGE_OFFSET (lowmem start) : 0xffff888000000000
[  142.301242] high_memory (lowmem end)   : 0xffff888200000000
[  142.301244] Lowmem size                : 8192 MB
[  142.301246] VMALLOC_START              : 0xffffc90000000000
[  142.301248] VMALLOC_END                : 0xffffe8ffffffffff
[  142.301250] vmalloc region size        : 32768 MB
[  142.301252] MODULES_VADDR              : 0xffffffffa0000000
[  142.301254] MODULES_END                : 0xfffffffffeffffff
[  142.301256] Modules region size        : 1504 MB
[  142.301258] Kernel text  _text         : 0xffffffff81000000
[  142.301260] Kernel text  _etext        : 0xffffffff82800000
[  142.301262] Kernel text size           : 24576 kB
...
✅ What These Numbers Tell You

Looking at the output, you can observe several things about your specific kernel and system: the total size of physical RAM that is direct-mapped into lowmem, how large the vmalloc pool is (important to know for driver work that uses vmalloc() or ioremap()), the size of the kernel modules area (affects how many large modules can be loaded at once), and the exact size of the compiled kernel’s text and data sections.

How the Layout Differs Across Architectures

One of the key lessons in a free Linux kernel development course is that the kernel is intentionally portable, but memory layout is very architecture-specific. Here is a comparison table to make the differences concrete:

Kernel Segment Layout Comparison Across Architectures
Property 32-bit x86 (3:1) 64-bit x86_64 64-bit ARM64
PAGE_OFFSET 0xC0000000 0xffff888000000000 Config-dependent
Kernel segment size 1 GB ~128 TB ~128 TB (48-bit VA)
ZONE_HIGHMEM Yes (RAM > ~768 MB) No No
Modules region location Just above user VAS (ARM-32 style) High in kernel segment Near kernel text
Vector table Yes (ARM-32) No (uses IDT) No (uses VBAR_EL1)
KASLR support Limited Full Full
64-bit x86_64 Kernel Segment — Detailed Region Map
0xffffffffffffffff
End of kernel segment
MODULES_END
Kernel Modules Region
~1.5 GB for loadable .ko files
FIXADDR_START
Fix Map Region
Compile-time reserved VAs
VMALLOC_END
vmalloc / ioremap Region
~32 TB for dynamic virtual mappings
VMALLOC_START
↑ vmalloc pool above
high_memory
↑ lowmem top (= amount of RAM)
variable
Lowmem (Direct-Mapped Region)
Physical RAM mapped 1:1 at PAGE_OFFSET
Kernel image, slab objects, page tables
PAGE_OFFSET
Kernel segment starts here ↑ / User space ends here ↓

Viewing Kernel Region Info from User Space

You do not always need a kernel module to explore the memory layout. Several user space interfaces expose this information safely (without revealing actual addresses on hardened kernels):

# View vmalloc allocations (safe on all kernels)
cat /proc/vmallocinfo

# View physical memory layout by zone
cat /proc/zoneinfo

# View overall memory statistics
cat /proc/meminfo

# View memory map for PID 1 (init/systemd) - shows kernel mappings too
sudo cat /proc/1/maps | grep -v "r--p\|---p" | head -30

# Kernel documentation: reference map for x86_64 (in kernel source)
# File: Documentation/x86/x86_64/mm.rst

The /proc/vmallocinfo file is particularly useful when you are writing kernel code that uses vmalloc() or ioremap(). It shows you every current vmalloc allocation: which caller requested it, the virtual address range, and what flags were used. This helps diagnose vmalloc space exhaustion, which is a real concern on older 32-bit systems.

Common Mistakes When Working with Kernel Addresses

Mistake 1: Printing Kernel Pointers with %lx Directly

If you use %lx or %p in a printk() to print a kernel address, modern kernels (5.x and above) will print a hashed value, not the real address. This is by design to prevent kernel address leaks. To print actual addresses during development, use %px (which bypasses hashing) or pass the no_hash_pointers boot parameter.

// Will print hashed address (safe for production logs):
pr_info("addr = %p\n", ptr);

// Will print real address (development only!):
pr_info("addr = %px\n", ptr);

// Or use %lx with explicit cast:
pr_info("addr = 0x%016lx\n", (unsigned long)ptr);
Mistake 2: Hard-Coding Kernel Addresses

Never write something like if (addr == 0xC0000000) in kernel code. The actual value of PAGE_OFFSET varies by architecture and configuration. Use the named macro: if (addr == PAGE_OFFSET). Your code must work on 32-bit ARM, 64-bit x86, and ARM64 without modification.

Mistake 3: Dereferencing an ioremap Address After iounmap

When a driver calls ioremap() to map device registers, it gets a KVA in the vmalloc region. After iounmap() is called, that KVA is no longer valid. Dereferencing it after unmapping causes silent data corruption or a kernel oops. Always set the pointer to NULL after unmapping and check for NULL before use.

Mistake 4: Using kmalloc Memory Above high_memory

Memory from kmalloc() and the slab allocator comes from the lowmem region and is DMA-safe on most architectures. However, if you try to use GFP_HIGHMEM on a 32-bit system and then dereference the pointer directly, you may be accessing memory that is not currently mapped. Always use kmap() / kunmap() around accesses to highmem pages on 32-bit platforms.

Best Practices for Kernel Memory Safety

  • Always use named macros (PAGE_OFFSET, VMALLOC_START, etc.) rather than hard-coded addresses.
  • Guard architecture-specific macros with #ifdef CONFIG_ARM, #ifdef CONFIG_X86, or similar guards.
  • Never dereference a virtual address without first verifying it is in the expected region using helper functions like virt_addr_valid().
  • Use access_ok() before reading or writing to any address that came from user space.
  • When using vmalloc()-allocated memory for DMA, do not assume the pages are physically contiguous — they are not. Use dma_alloc_coherent() instead.
  • On 64-bit kernels, prefer kmalloc() over vmalloc() for small to medium allocations (under 4 MB). kmalloc() returns physically contiguous memory and is faster to access.
  • Always call iounmap() and kfree() / vfree() in your module’s exit function to avoid kernel memory leaks.

Key Takeaways

  • The kernel provides named macros (PAGE_OFFSET, VMALLOC_START, MODULES_VADDR, etc.) to describe segment region boundaries. Always use these, never hard-coded numbers.
  • Macros are architecture-specific. Always use #ifdef guards for arch-specific macros.
  • Writing a simple kernel module to print these values is one of the best ways to understand kernel memory layout on your actual target system.
  • Modern kernels hash pointer values in printk() by default. Use %px for real addresses during development only.
  • /proc/vmallocinfo, /proc/zoneinfo, and /proc/meminfo are valuable user space tools for observing the kernel memory state without writing a module.
  • Kernel memory safety requires careful attention to which region an address belongs to and whether the mapping is currently active.

Frequently Asked Questions

Q: Why does my module compile fine but show different addresses than I expected?

Almost certainly because of KASLR. Every time you reboot, the kernel places itself at a different random base address inside the kernel segment. The values you see are correct for the current boot — they will be different after the next reboot. Disable KASLR only on development machines by adding nokaslr to your kernel boot parameters if you need reproducible addresses for testing.

Q: My module uses MODULES_VADDR but it does not compile. Why?

MODULES_VADDR is not defined on all architectures. On some platforms (like 32-bit x86), the modules region does not have a separate dedicated macro. Wrap your code in #ifdef MODULES_VADDR ... #endif to handle this portably. Also check arch/your-arch/include/asm/memory.h or equivalent for what is actually available on your target.

Q: What is the difference between virt_to_phys() and __pa()?

Both convert a kernel virtual address to a physical address, but they work only for lowmem (direct-mapped) addresses. Internally, __pa(addr) subtracts PAGE_OFFSET from the virtual address to get the physical one. virt_to_phys() is a wrapper around __pa() with some extra type checking. Neither works for vmalloc addresses or ioremap addresses — for those, you need vmalloc_to_page() followed by page_to_phys().

Q: How large is the vmalloc region on a typical 64-bit server?

On x86_64 with a 48-bit virtual address space (the most common configuration), the vmalloc region spans from 0xffffc90000000000 to 0xffffe8ffffffffff, which is approximately 32 TB. On kernels compiled with 5-level paging (57-bit VA), this region becomes even larger. For most practical purposes, you will never exhaust this pool. The concern only becomes real on embedded systems with small kernels (32-bit) or systems that are extremely heavy users of ioremap().

Q: Can two different kernel modules have their code at the same virtual address?

No. The kernel’s module loader allocates a unique virtual address range from the modules region for each module when it is inserted. These ranges never overlap for concurrently loaded modules. However, if you remove a module and then insert a different module, the new module may occupy the same virtual address range that the old module had — the kernel reuses freed virtual space. This is fine because the old module’s code is completely gone.

Q: How do I find what address my loaded module is at right now?

After loading a module, check /proc/modules or /sys/module/your_module_name/sections/.text for the address of the text section. You can also read /proc/kallsyms which lists every symbol in the kernel and all loaded modules along with their current addresses. Note that on hardened systems, non-root users see zeroed addresses in /proc/kallsyms.

# Find your module's load address
cat /proc/modules | grep show_kernel_segment

# Find specific symbols including from modules
sudo grep show_kernel_seg /proc/kallsyms

Conclusion

In this lecture of our free Linux kernel development course, you moved from theoretical understanding of kernel segment regions to practical hands-on exploration using real kernel macros and a working LKM. The kernel segment layout is not just academic knowledge — it directly affects how you write device drivers, how you allocate memory correctly, and how you debug kernel panics and memory corruption issues.

Knowing the difference between lowmem and vmalloc addresses, understanding why PAGE_OFFSET matters, and being able to inspect the live kernel segment using both module code and /proc interfaces are all skills that set apart a junior kernel developer from a confident one. In the next lecture, we will dig into the kernel’s memory allocation APIs — kmalloc(), vmalloc(), get_free_pages(), and their friends — and you will understand immediately why the region distinctions you learned here matter so much for choosing the right allocator.

Learn Linux Kernel Programming for Free

EmbeddedPathashala offers completely free courses on Linux kernel development, Linux device drivers, and embedded systems.

Visit EmbeddedPathashala

Leave a Reply

Your email address will not be published. Required fields are marked *