What You Will Learn

• Why a plain page table walk is too slow for every memory access
• What a TLB is and how it caches virtual-to-physical translations
• TLB lookup, TLB hit, and TLB miss explained step by step
• Software-managed vs hardware-managed TLB refill on modern CPUs
• How the Linux page fault handler is triggered on ARM64 and x86_64
• Writing an original kernel module that triggers TLB flush APIs

Prerequisites

This lecture builds directly on lddch11_4 (Linux page table structure: PGD/P4D/PUD/PMD/PTE). You should be comfortable with:

• Basic virtual memory and page table concepts
• Writing and loading a simple Linux kernel module (insmod/rmmod)
• A Linux machine or VM running kernel 6.x with headers installed

Recap: PTBR, TTBR0 and the Page Table Walk

In the previous lecture we saw that the MMU never stores addresses itself. Instead a dedicated CPU register — called the Page Table Base Register (PTBR), or TTBR0 on ARM64 — points to the base of the top-level page table (PGD) of the currently running process. This is exactly the field current->mm->pgd in struct mm_struct. On every context switch, the kernel reloads this register with the new process’s PGD, so the MMU always walks the correct process’s tables.

If every single memory access required a full walk through five page table levels, virtual memory would be many times slower than direct physical access, and the whole idea would defeat its own purpose. This is the exact performance problem the TLB was designed to solve.

What Is a TLB in the Linux Kernel?

The Translation Lookaside Buffer (TLB) is a small, extremely fast piece of associative memory built into the MMU. Think of it as a cache, but instead of caching data, it caches translations: the key is a virtual page number and the value is the matching physical frame number. Because it is content-addressable, the CPU can check “do I already know this translation?” in a single, near-instant lookup instead of walking the full table.

On a TLB hit, the CPU already has the physical frame number cached, so it calculates the target physical address immediately and continues execution — no page fault, no table walk, almost the same speed as a physical memory access. On a TLB miss, the requested virtual page is not currently cached, and the kernel or the MMU hardware must go find the translation the slow way, by walking the process page table.

TLB Miss Handling: Software vs Hardware

Historically, some architectures handled a TLB miss purely in software: the CPU raised a TLB-miss interrupt, the OS walked the page table by hand, found the PTE, and manually inserted the new entry into the TLB. Older MIPS-style designs worked this way. Modern mainstream Linux targets take a different, faster path.

On the ARM64 and x86_64 targets you will meet throughout this free Linux kernel development course, a routine TLB miss with a valid mapping is completely invisible to your driver code — the hardware silently refills the TLB. Your code only gets involved when the mapping genuinely does not exist, and that is when a page fault is raised.

The Linux Page Fault Handler Today

In both the software and hardware TLB-miss cases, if no valid entry can be found, control eventually reaches the architecture’s page fault handler. Older kernel documentation (matching the source PDF this lecture is modernized from) pointed to do_page_fault() in arch/arm/mm/fault.c for 32-bit ARM, which is still accurate for that architecture today.

On 64-bit targets the entry points have moved on. ARM64 still exposes a do_page_fault()-style handler inside arch/arm64/mm/fault.c, which decodes the fault syndrome register and eventually calls the common handle_mm_fault() path. On x86_64, the classic do_page_fault() name from older kernels was replaced by the modern IDT-entry style handler exc_page_fault(), also defined in arch/x86/mm/fault.c. Either way, the job is identical: figure out whether the address is a legitimate but unmapped part of the process’s virtual memory area (VMA), and if so, hand off to handle_mm_fault() to fault the page in; otherwise, deliver a fault signal to the process.

Original Demo: Triggering a TLB Flush From a Kernel Module

You will not call TLB-refill code directly — that lives deep in the MMU and low-level assembly entry code. But you absolutely can observe and exercise TLB-related kernel APIs from a normal kernel module, which is a safe and practical way to understand the TLB lifecycle. The demo below, ep_tlb_demo, allocates a page for a dummy per-process mapping style buffer, prints the associated virtual and physical addresses, and then explicitly calls the kernel’s TLB flush API to invalidate any TLB entries for that range — exactly the kind of call the kernel itself makes internally after unmapping memory.

// ep_tlb_demo.c
// Original demo module for the free Linux kernel development course
// Demonstrates flush_tlb_kernel_range() usage after modifying kernel mappings

#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/vmalloc.h>
#include <linux/mm.h>
#include <asm/tlbflush.h>

#define EP_TLB_SIZE (PAGE_SIZE)

static void *ep_tlb_area;

static int __init ep_tlb_demo_init(void)
{
    unsigned long start, end;

    ep_tlb_area = vmalloc(EP_TLB_SIZE);
    if (!ep_tlb_area) {
        pr_err("ep_tlb_demo: vmalloc failed\n");
        return -ENOMEM;
    }

    start = (unsigned long)ep_tlb_area;
    end   = start + EP_TLB_SIZE;

    pr_info("ep_tlb_demo: loaded\n");
    pr_info("ep_tlb_demo: kernel virtual addr = 0x%lx\n", start);
    pr_info("ep_tlb_demo: mapped size         = %lu bytes\n", (unsigned long)EP_TLB_SIZE);

    /* Touch the memory so a real translation gets cached in the TLB */
    memset(ep_tlb_area, 0xAA, EP_TLB_SIZE);
    pr_info("ep_tlb_demo: buffer touched, translation now cached in TLB\n");

    /* Explicitly flush this kernel virtual range from the TLB,
     * the same style of call the kernel makes after unmapping pages */
    flush_tlb_kernel_range(start, end);
    pr_info("ep_tlb_demo: flush_tlb_kernel_range() called for range [0x%lx - 0x%lx]\n",
            start, end);

    return 0;
}

static void __exit ep_tlb_demo_exit(void)
{
    if (ep_tlb_area) {
        unsigned long start = (unsigned long)ep_tlb_area;
        unsigned long end   = start + EP_TLB_SIZE;

        vfree(ep_tlb_area);
        flush_tlb_kernel_range(start, end);
        pr_info("ep_tlb_demo: unloaded, TLB range flushed again\n");
    }
}

module_init(ep_tlb_demo_init);
module_exit(ep_tlb_demo_exit);

MODULE_LICENSE("GPL");
MODULE_AUTHOR("EmbeddedPathashala");
MODULE_DESCRIPTION("Original TLB flush demo for free Linux kernel development course");

Build and Run

# Makefile
obj-m += ep_tlb_demo.o

all:
	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules

clean:
	make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

$ make
$ sudo insmod ep_tlb_demo.ko
$ dmesg | tail -6

Expected Output

[ 1234.567890] ep_tlb_demo: loaded
[ 1234.567892] ep_tlb_demo: kernel virtual addr = 0xffffabcd12340000
[ 1234.567893] ep_tlb_demo: mapped size         = 4096 bytes
[ 1234.567895] ep_tlb_demo: buffer touched, translation now cached in TLB
[ 1234.567897] ep_tlb_demo: flush_tlb_kernel_range() called for range [0xffffabcd12340000 - 0xffffabcd12341000]

$ sudo rmmod ep_tlb_demo
$ dmesg | tail -2
[ 1240.111222] ep_tlb_demo: unloaded, TLB range flushed again

Note: the exact hex addresses will differ on your machine since they depend on vmalloc’s current allocation state and KASLR. What matters is the pattern: allocate, touch (cache the translation), flush (invalidate the cached translation) — this is exactly what the kernel does internally whenever it changes a mapping and must guarantee the CPU stops using a stale TLB entry.

Common Mistakes

Best Practices

• Always flush the correct TLB range after changing a mapping, never flush more broadly than needed
• Prefer range-based flush APIs like flush_tlb_kernel_range() over flushing the entire TLB
• Understand your target’s TLB refill model before optimizing hot memory access paths

Performance Considerations

A TLB hit is essentially free, so keeping a process’s working set of pages small enough to fit in the TLB dramatically improves performance. This is one reason huge pages exist: fewer, larger translations mean far fewer TLB entries are needed to cover the same amount of memory, reducing TLB miss rate on memory-intensive workloads.

Security Considerations

Stale TLB entries after an unmap can be a serious security bug: if you free a page and reuse it for something else without flushing the TLB, another context could keep reading or writing through the old, now-incorrect translation. This is exactly why unmap paths in the kernel are always paired with a TLB flush call.

Summary and Key Takeaways

• A full page table walk costs one memory access per level, which is too slow for every access
• The TLB caches recent virtual-to-physical translations for near-instant lookup
• A TLB hit skips the table walk entirely; a TLB miss triggers either software or hardware refill
• x86_64 and ARM64 refill the TLB in hardware; the OS only steps in on a genuine page fault
• The modern page fault entry points are exc_page_fault() on x86_64 and do_page_fault() in arch/arm64/mm/fault.c on ARM64

Conclusion

The TLB is the quiet hero of virtual memory performance. Without it, every load and store would pay the full cost of a multi-level page table walk, and virtual memory would be impractical. By caching translations close to the CPU, the TLB keeps address translation nearly free on a hit, while hardware-managed refill on modern ARM64 and x86_64 systems keeps misses cheap too. In the next lecture of this free embedded Linux course, we move from address translation into the memory allocation mechanism itself — how the kernel actually hands out physical memory through kmalloc, vmalloc, the SLAB allocator, and the page allocator.

FAQ