01 What is Module stacking in linux kenrel modules?

When the Linux kernel loads a module, that module may declare functions or variables it wants to make available to other modules. When a second module is later loaded and calls those functions — consuming the symbols exported by the first — we say the two modules are stacked.

In plain terms: Module B sits on top of Module A. Module A exports a symbol; Module B uses it. This layered dependency is what gives the technique its name — module stacking.

┌──────────────────────────────────────────────┐
│            User Space (applications)         │
│           read() / write() / ioctl()         │
└────────────────────┬─────────────────────────┘
                     │  syscall
┌────────────────────▼─────────────────────────┐
│  LAYER 2 — Upper Module (mod2.ko)            │
│   • Registers /dev entry, exposes interface  │
│   • Calls symbols exported by lower layer    │
└────────────────────┬─────────────────────────┘
                     │  EXPORT_SYMBOL calls
┌────────────────────▼─────────────────────────┐
│  LAYER 1 — Base Module (mod1.ko)             │
│   • Hardware abstraction / core logic        │
│   • EXPORT_SYMBOL(myadd) → visible to all    │
└────────────────────┬─────────────────────────┘
                     │
┌────────────────────▼─────────────────────────┐
│            Linux Kernel Core                 │
│      (symbol table — /proc/kallsyms)         │
└──────────────────────────────────────────────┘

The kernel maintains a global symbol table. When a module calls EXPORT_SYMBOL(), its symbol lands in that table. Any subsequently loaded module can resolve its external references against that table at load time — this is the kernel’s version of dynamic linking.

02 Why Split into Layers?

Real-world drivers are complex. A USB audio device involves USB host controller logic, audio codec control, ALSA buffer management, and a /dev character interface — all interleaved if you wrote one monolithic driver. Module stacking separates concerns cleanly:

Benefits in Practice

• Reusability — A base module wrapping a GPIO controller can be consumed by an LED driver, a button driver, and a keypad driver simultaneously.
• Maintainability — Bugs in the hardware abstraction are fixed in one place; no upper module changes needed.
• Vendor BSP separation — SoC vendors ship a proprietary base module (closed source) while the upper interface module can be open and customised by OEMs.
• Faster iteration — Reload only the upper module during development; the base module stays loaded, keeping hardware state intact.

03 EXPORT_SYMBOL — Making Symbols Visible

By default, every function or variable defined in a kernel module is private to that module. The linker does not expose it to other modules. To make it part of the kernel’s global symbol table, you must explicitly export it.

There are three macros, each serving a different purpose:

What Happens Internally

When the kernel builds a module, EXPORT_SYMBOL(myadd) places an entry into a special ELF section: __ksymtab. This section contains the symbol’s address and name string. When the module is loaded via insmod, the kernel’s module loader walks __ksymtab and merges these entries into the global symbol table, making them resolvable by any future insmod call.

mod1.ko (ELF)
┌─────────────────────────────────────────┐
│  .text         │  myadd() function body │
│  __ksymtab     │  { &myadd, "myadd" }   │  ← EXPORT_SYMBOL adds this
│  __kstrtab     │  "myadd\0"             │
│  .modinfo      │  license, author, etc  │
└─────────────────────────────────────────┘

                     insmod mod1.ko

┌─────────────────────────────────────────┐
│  Kernel global symbol table             │
│  ...  │  myadd  │  0xffffffffcXXXXXXX  │
└─────────────────────────────────────────┘

                     insmod mod2.ko  →  linker resolves extern myadd here

04 The Linux Symbol Table (kallsyms)

/proc/kallsyms is a virtual file that exposes the entire kernel symbol table — both built-in kernel symbols and symbols exported by loaded modules — as a human-readable list. Each line has three fields:

# address          type  name          [module]
ffffffffc08a1000  T     myadd          [mod1]
ffffffff81234abc  T     printk
ffffffff81567def  t     some_private_fn

The type field follows nm conventions. The most common ones you will encounter:

As a developer, /proc/kallsyms is your go-to tool to confirm an export landed correctly before you try to load the dependent module.

05 Practical Example: myadd Across Two Modules

We will build the simplest possible stacked pair: mod1 exports a function myadd(), and mod2 calls it. This is intentionally trivial so every concept around the loading order, symbol resolution, and dependency enforcement stands out clearly.

mod1.c — The Exporting Module (Base Layer)

#include <linux/kernel.h>
#include <linux/module.h>

MODULE_LICENSE("GPL");
MODULE_AUTHOR("EmbeddedPathashala");
MODULE_DESCRIPTION("Base module: exports myadd symbol");

/*
 * myadd - exported addition function
 * @a: first operand
 * @b: second operand
 *
 * This function lives in the global kernel symbol table
 * after EXPORT_SYMBOL(). Any subsequently loaded module
 * can call it directly — no header or link-time trick needed.
 */
int myadd(int a, int b)
{
    pr_info("%s: sum of %d + %d = %d\n", __func__, a, b, a + b);
    return a + b;
}
EXPORT_SYMBOL(myadd);   /* ← publishes symbol into __ksymtab */

static int mod1_init(void)
{
    pr_info("%s: mod1 loaded, myadd now in symbol table\n", __func__);
    return 0;
}

static void mod1_exit(void)
{
    pr_info("%s: mod1 unloading — symbol myadd removed from table\n", __func__);
}

module_init(mod1_init);
module_exit(mod1_exit);

Key points in mod1.c

• MODULE_LICENSE("GPL") is mandatory. Without it the kernel will taint itself and may refuse to let dependent modules use your exported symbols.
• EXPORT_SYMBOL(myadd) must appear at file scope, after the function definition — not inside a function body.
• The init and exit callbacks are normal. Exporting a symbol has nothing to do with the module’s lifecycle hooks.

mod2.c — The Consuming Module (Upper Layer)

#include <linux/kernel.h>
#include <linux/module.h>

MODULE_LICENSE("GPL");
MODULE_AUTHOR("EmbeddedPathashala");
MODULE_DESCRIPTION("Stacked module: consumes myadd from mod1");

/*
 * Declare myadd as external — the linker will NOT find this
 * in mod2.ko itself. At insmod time, the kernel's module
 * loader resolves it from the global symbol table.
 * mod1 must already be loaded for this to succeed.
 */
extern int myadd(int a, int b);

static int mod2_init(void)
{
    int result;

    pr_info("%s: mod2 init — calling myadd from mod1\n", __func__);
    result = myadd(3, 5);
    pr_info("%s: myadd(3, 5) = %d\n", __func__, result);
    return 0;
}

static void mod2_exit(void)
{
    pr_info("%s: mod2 exit\n", __func__);
}

module_init(mod2_init);
module_exit(mod2_exit);

Key points in mod2.c

• extern int myadd(int a, int b) tells the compiler this symbol exists somewhere else. The compiler generates a relocatable reference; the kernel module loader fills in the real address at insmod time.
• No #include of a shared header is required — the declaration alone is sufficient for intra-module stacking. In production drivers, it is good practice to put the declaration in a shared header.
• mod2 has no knowledge of mod1’s implementation — only its interface. This is information hiding at the kernel level.

06 Build System — The Makefile

Building two stacked modules requires a Makefile that instructs the kernel build system (kbuild) to compile both objects. A standard out-of-tree Makefile looks like this:

# obj-m tells kbuild to build these as loadable modules
obj-m += mod1.o
obj-m += mod2.o

# Point to the running kernel's build tree
KDIR := /lib/modules/$(shell uname -r)/build

all:
	make -C $(KDIR) M=$(PWD) modules

clean:
	make -C $(KDIR) M=$(PWD) clean

Running make produces mod1.ko and mod2.ko. At this point mod2.ko contains an unresolved reference to myadd — it cannot be satisfied by the file system; it must be resolved at runtime against the live kernel symbol table.

07 Step-by-Step: Load, Verify, Unload

The Wrong Way First — Understand the Error

It is instructive to deliberately try loading mod2 before mod1. The kernel immediately fails with an Unknown symbol error:

raviteja@Inspiron:~/kernel_programming/Day7/export_module$ sudo insmod mod2.ko 

insmod: ERROR: could not insert module mod2.ko: Unknown symbol in module 

$ dmesg | tail -3 [ 8658.327642] mod2: Unknown symbol myadd (err -2)

err -2 is -ENOENT — no such entry in the symbol table. The kernel module loader tried to resolve myadd and found nothing. This is the enforcement mechanism for stacking order.

The Correct Loading Sequence

1 Load the base module (mod1) — this registers myadd in the kernel symbol table.

2 Verify the symbol is live — grep /proc/kallsyms for myadd. Confirm you see type T and [mod1].

3 Load the upper module (mod2) — the loader resolves myadd from the symbol table, increments mod1’s reference count, and executes mod2_init().

4 Check dmesg — confirm both init messages and the result of myadd(3, 5) = 8.

# Step 1: Load the base module $ sudo insmod mod1.ko 

# Step 2: Verify symbol is in the table $ cat /proc/kallsyms | grep myadd ffffffffc08a1000 T myadd [mod1] 

# Step 3: Load the upper module — myadd is now resolvable $ sudo insmod mod2.ko 

# Step 4: Check kernel log $ dmesg | tail -6 

[ 9120.441200] mod1_init: mod1 loaded, myadd now in symbol table 

[ 9134.885312] mod2_init: mod2 init — calling myadd from mod1

 [ 9134.885319] myadd: sum of 3 + 5 = 8 

[ 9134.885321] mod2_init: myadd(3, 5) = 8 

# Verify reference count: mod1 shows 1 user (mod2) $ cat /proc/modules | grep mod mod2 16384 0 - Live 0xffffffffc08b0000 mod1 16384 1 mod2 Live 0xffffffffc08a0000

Note the 1 in mod1's /proc/modules line — that is its use count. The kernel will refuse to let you rmmod mod1 while mod2 is loaded and holding a reference. This is the kernel's protection against use-after-free crashes.

Unloading — Reverse Dependency Order

Unloading must happen in the reverse of loading order. Trying to remove mod1 first will fail with EWOULDBLOCK:

# Wrong: mod1 still has a user (mod2) $ sudo rmmod mod1 rmmod: ERROR: Module mod1 is in use by: mod2 # Correct: remove upper layer first 

$ sudo rmmod mod2 

$ sudo rmmod mod1 

$ dmesg | tail -4 

[ 9201.113400] mod2_exit: mod2 exit 

[ 9210.774200] mod1_exit: mod1 unloading — symbol myadd removed from table # Confirm symbol is gone from the table 

$ cat /proc/kallsyms | grep myadd (no output)

08 Real-World Patterns

Character Driver Stacking

The most common pattern in BSP work: the base module owns hardware registers; the upper module registers a character device that user-space can open, read, and write.

┌──────────────────────────────────────────────┐
│  Upper Module: chardev_ui.ko                 │
│  • cdev_init(), cdev_add()                   │
│  • file_operations: .read → calls hw_read()  │
│  • file_operations: .write → calls hw_write()│
└────────────────────┬─────────────────────────┘
              EXPORT_SYMBOL calls
┌────────────────────▼─────────────────────────┐
│  Base Module: hw_abstraction.ko              │
│  • ioremap(), readl(), writel()              │
│  • EXPORT_SYMBOL(hw_read)                    │
│  • EXPORT_SYMBOL(hw_write)                   │
│  • EXPORT_SYMBOL(hw_init_device)             │
└──────────────────────────────────────────────┘

Bluetooth Stack Stacking (Real Kernel Example)

The BlueZ kernel stack is one of the most elaborate stacking examples in the mainline kernel. Each layer exports symbols consumed by the one above:

Stacking in SoC Vendor BSPs

SoC vendors like Qualcomm or MediaTek ship a platform module that exports chip-specific APIs (clock gating, power domains, DMA handles). OEM driver modules stack on top without needing — or being allowed — to touch silicon-level code. This is the primary commercial motivation for the pattern.

09 Common Errors and Fixes

10 Summary and Key Takeaways

Free learning to all !!

EmbeddedPathashala.