What You Will Learn

Prerequisites

Why Embedded Storage Is a Different Problem

On a laptop, if the power cuts mid-write, you get a filesystem check on the next boot and move on. An embedded device – a router, a payment terminal, a factory sensor – often can’t afford that. It has to survive a yanked power cord mid-flash-erase and still boot into a working state, possibly unattended, possibly for a decade. That single requirement drives almost every design decision in this chapter: which flash technology to pick, which filesystem to lay on top of it, and how in-field firmware updates are structured.

Solid-state storage has been the answer for embedded systems for over twenty years, evolving from simple ROM through several generations of flash memory. Understanding the properties of that flash – not just how big it is, but how it fails – is what lets you choose correctly.

The Flash Family: NOR, NAND, and Managed Flash

All flash memory stores bits by trapping charge in a cell, and all of it shares one quirk that trips up newcomers: you can only set bits from 1 to 0 by writing, but to go back from 0 to 1 you must erase – and erasing happens in large fixed-size blocks, never a single byte. Beyond that shared behavior, the three families diverge sharply:

Managed flash devices hide their internal NAND behind a controller that speaks a familiar block-device protocol, so the kernel just sees something like a small hard disk. Raw NOR and raw NAND, on the other hand, are exposed directly to the kernel – and that’s exactly the case the MTD subsystem exists for.

NOR Flash: Execute-in-Place Storage

NOR flash chips are organized into erase blocks – commonly in the range of tens to a few hundred kilobytes each. Erasing a block forces every bit in it to 1; programming then selectively clears individual bits to 0, one word at a time. Because each erase cycle stresses the oxide layer in the cell, a block eventually wears out – the datasheet will quote an endurance figure, typically in the hundreds of thousands of cycles.

What makes NOR special is that it can be wired directly into the CPU’s address space, just like RAM, and read one word at a time with no protocol overhead. That means the processor can fetch and execute instructions straight out of NOR without ever copying them into RAM first – a technique called eXecute In Place (XIP). This is precisely why NOR is still the go-to choice for first-stage bootloader storage on many SoCs: the reset vector can point directly at flash, and the chip needs no initialization sequence before the CPU can start fetching from it.

Nearly every modern NOR chip implements a standardized register-level interface called the Common Flash Interface (CFI), which lets a single generic Linux driver identify and operate chips from different vendors without per-chip code.

Enter the MTD Subsystem

Linux doesn’t want every filesystem and every bootloader talking to raw flash hardware with vendor-specific register sequences. The Memory Technology Device (MTD) subsystem sits between raw flash chip drivers and everything above them – filesystems like JFFS2 and UBIFS, and userspace tools – and exposes a uniform set of operations: erase, read, and write, plus geometry information like erase-block size and total size.

Every registered flash device shows up as a character device (/dev/mtd0, /dev/mtd1, and so on) for raw access, and optionally a read-only block device (/dev/mtdblock0) for cases where something wants to treat it like a disk. Userspace tools from the mtd-utils package – flash_erase, nanddump, nandwrite, mtdinfo – operate on these nodes.

The Current mtd_info API

At the driver level, a flash device is represented by a struct mtd_info, populated with the device’s geometry and a set of operation callbacks. On current mainline kernels, the essentials look like this:

struct mtd_info {
    const char *name;
    uint64_t size;          /* total device size in bytes */
    uint32_t erasesize;     /* size of one erase block */
    uint32_t writesize;     /* minimum write unit: 1 for NOR, page size for NAND */

    int (*_erase)(struct mtd_info *mtd, struct erase_info *instr);
    int (*_read)(struct mtd_info *mtd, loff_t from, size_t len,
                 size_t *retlen, u_char *buf);
    int (*_write)(struct mtd_info *mtd, loff_t to, size_t len,
                  size_t *retlen, const u_char *buf);
    /* ... */
};

int mtd_device_register(struct mtd_info *mtd,
                         const struct mtd_partition *parts,
                         int nr_parts);
void mtd_device_unregister(struct mtd_info *mtd);

A driver’s job is simply to fill in the geometry fields and implement the three callbacks against the real hardware, then call mtd_device_register(). Everything above – the char/block device nodes, the ioctl interface, partition parsing – is handled for you by the MTD core.

Hands-On: A RAM-Backed MTD Device

Real NOR hardware isn’t something everyone has on their desk, so we’ll build a tiny original driver, ep_mtdram, that emulates a flash chip using a plain kernel memory buffer. It behaves exactly like real flash from the MTD core’s point of view – erase sets bytes to 0xFF, write can only clear bits – which makes it a genuinely useful way to practice the API before touching real hardware.

// ep_mtdram.c - minimal RAM-backed MTD device for learning the MTD API
#include <linux/module.h>
#include <linux/mtd/mtd.h>
#include <linux/slab.h>
#include <linux/vmalloc.h>

#define EP_MTD_SIZE       (256 * 1024)
#define EP_MTD_ERASESIZE  (16 * 1024)

static struct mtd_info ep_mtd;
static u8 *ep_flash_mem;

static int ep_mtd_erase(struct mtd_info *mtd, struct erase_info *instr)
{
    memset(ep_flash_mem + instr->addr, 0xFF, instr->len);
    pr_info("ep_mtdram: erased 0x%llx len 0x%llx\n", instr->addr, instr->len);
    return 0;
}

static int ep_mtd_read(struct mtd_info *mtd, loff_t from, size_t len,
                        size_t *retlen, u_char *buf)
{
    memcpy(buf, ep_flash_mem + from, len);
    *retlen = len;
    return 0;
}

static int ep_mtd_write(struct mtd_info *mtd, loff_t to, size_t len,
                         size_t *retlen, const u_char *buf)
{
    size_t i;
    for (i = 0; i < len; i++)
        ep_flash_mem[to + i] &= buf[i]; /* can only clear bits, like real flash */
    *retlen = len;
    return 0;
}

static int __init ep_mtd_init(void)
{
    ep_flash_mem = vmalloc(EP_MTD_SIZE);
    if (!ep_flash_mem)
        return -ENOMEM;
    memset(ep_flash_mem, 0xFF, EP_MTD_SIZE);

    ep_mtd.name = "ep_mtdram0";
    ep_mtd.size = EP_MTD_SIZE;
    ep_mtd.erasesize = EP_MTD_ERASESIZE;
    ep_mtd.writesize = 1;
    ep_mtd.type = MTD_RAM;
    ep_mtd.flags = MTD_CAP_RAM;
    ep_mtd._erase = ep_mtd_erase;
    ep_mtd._read = ep_mtd_read;
    ep_mtd._write = ep_mtd_write;

    return mtd_device_register(&ep_mtd, NULL, 0);
}

static void __exit ep_mtd_exit(void)
{
    mtd_device_unregister(&ep_mtd);
    vfree(ep_flash_mem);
}

module_init(ep_mtd_init);
module_exit(ep_mtd_exit);
MODULE_LICENSE("GPL");

Build it with a standard out-of-tree Makefile, then load it and exercise it with mtd-utils:

$ make -C /lib/modules/$(uname -r)/build M=$PWD modules
$ sudo insmod ep_mtdram.ko
$ cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00040000 00004000 "ep_mtdram0"

$ sudo flash_erase /dev/mtd0 0 0
Erasing 16 Kibyte @ 0 -- 100 % complete

$ echo "hello flash" | sudo dd of=/dev/mtd0 bs=1 count=12
12+0 records in
12+0 records out

$ sudo dd if=/dev/mtd0 bs=1 count=12 2>/dev/null
hello flash

# dmesg output while exercising the device
[  912.114201] ep_mtdram: erased 0x0 len 0x4000
[  915.882330] mtd: device registered ep_mtdram0
[  921.556210] ep_mtdram: erased 0x0 len 0x4000

Common Mistakes and Troubleshooting

Best Practices

Summary and Key Takeaways

Embedded storage design starts with understanding the physical flash: NOR is byte-addressable and execute-in-place capable but low capacity and relatively costly per byte; NAND is cheap and dense but needs page-based access and can’t be memory-mapped. Linux abstracts both behind the MTD subsystem, giving drivers a uniform mtd_info structure with erase/read/write callbacks, and giving userspace uniform /dev/mtdX nodes. In the next lecture we go one level deeper into NAND flash – cell density, error correction, and the out-of-band area that makes NAND driver design genuinely tricky.

Conclusion

Whether you’re bringing up a new board or debugging why firmware updates keep bricking a device in the field, the MTD layer is where that investigation starts. Get comfortable with mtdinfo, /proc/mtd, and the erase/read/write model in this lecture – it’s the foundation every flash filesystem and update mechanism in this course builds on.

Frequently Asked Questions