What You Will Learn

• Why a read-only root filesystem is one of the highest-leverage reliability techniques in embedded Linux
• How to mount root read-only via the kernel command line versus using an inherently read-only filesystem
• How to handle the small set of files that traditionally expect to be writable
• How overlayfs gives you a writable illusion on top of an immutable base
• How to test that your root filesystem is actually immutable before shipping

Prerequisites

This lesson assumes you have completed the squashfs and tmpfs lessons earlier in this free embedded systems course — both are core building blocks for the design described here.

Why Read-Only Matters More Than Almost Anything Else

Most field failures in embedded Linux devices trace back to one root cause: something was being written to flash at the exact moment power was lost. A half-written inode table, a torn journal entry, a partially-flashed configuration file — any of these can leave a device unable to boot, sitting in a customer’s hands or, worse, bolted to a wall somewhere hard to reach. The single most effective defense is refusing to let the root filesystem be written to during normal operation in the first place.

There are two ways to get there, and it is worth understanding both. You can mount an otherwise-writable filesystem (ext4, F2FS) with the ro flag, or you can use a filesystem that is read-only by construction, like squashfs. The construction-level guarantee is strictly stronger: a mount flag can be changed at runtime by anything with the right privileges (mount -o remount,rw /), while squashfs has no write code path to re-enable at all.

Setting Root Read-Only Via The Kernel Command Line

If you are using ext4 or F2FS as your root filesystem and want it read-only, the simplest change is on the kernel command line, replacing rw with ro:

# in your bootloader's kernel command line
root=/dev/mmcblk0p2 rootfstype=ext4 ro

This is a good first step during development, but remember it is a mount option, not a structural guarantee — treat it as a safety net, not your primary defense, and prefer squashfs for the shipping production image where practical.

The Files That Insist On Being Writable

A handful of paths across a standard Linux system genuinely expect to be writable, and a naive read-only conversion breaks them immediately. You need a plan for each.

For the /var/lib case, a common boot-script pattern copies a baseline set of files onto a tmpfs mount and then bind-mounts it over the real path, so applications believe they are writing to persistent storage while nothing ever touches flash:

#!/bin/sh
# early boot script: seed /var/lib onto tmpfs
mount -t tmpfs -o size=8m tmpfs /run/varlib
cp -a /var/lib/. /run/varlib/
mount --bind /run/varlib /var/lib

Overlayfs: A Writable Illusion On An Immutable Base

Sometimes symlinking individual paths is too fine-grained for what you need — for example, during development you may want the whole root filesystem to appear writable so you can iterate quickly, while still shipping a truly immutable image in production. overlayfs solves this by layering a writable directory (usually tmpfs, or a small persistent partition) on top of a read-only lower filesystem, presenting a single merged, apparently-writable view.

# lowerdir = your immutable squashfs root
# upperdir/workdir = a writable tmpfs area for changes
mount -t overlay overlay \
  -o lowerdir=/mnt/rofs,upperdir=/run/overlay/upper,workdir=/run/overlay/work \
  /mnt/merged

Any file the application “modifies” is actually copied up into upperdir the first time it is touched (copy-on-write), leaving the original squashfs image completely untouched. If upperdir lives on tmpfs, all of those changes evaporate on reboot — giving you a fully writable-looking filesystem with the same reboot-to-known-state guarantee as a pure read-only root.

A Small Demo: Verifying Root Is Actually Read-Only

Here is a minimal original kernel module, ep_rofs_check, that inspects the superblock flags of the root mount at boot and logs a clear warning if root is not actually mounted read-only — a useful sanity check to bake into a factory test image so a misconfigured build never accidentally ships.

// ep_rofs_check.c - warns at boot if root is not mounted read-only
#include <linux/init.h>
#include <linux/module.h>
#include <linux/fs.h>
#include <linux/path.h>
#include <linux/namei.h>

static int __init ep_rofs_check_init(void)
{
    struct path root_path;
    int err;

    err = kern_path("/", LOOKUP_FOLLOW, &root_path);
    if (err) {
        pr_err("ep_rofs_check: could not resolve root path, err=%d\n", err);
        return err;
    }

    if (root_path.mnt->mnt_sb->s_flags & SB_RDONLY)
        pr_info("ep_rofs_check: root filesystem is read-only, as expected\n");
    else
        pr_warn("ep_rofs_check: WARNING root filesystem is WRITABLE\n");

    path_put(&root_path);
    return 0;
}

module_init(ep_rofs_check_init);
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("EmbeddedPathashala demo: confirm root filesystem is read-only at boot");

$ insmod ep_rofs_check.ko
$ dmesg | tail -1
[    2.204119] ep_rofs_check: root filesystem is read-only, as expected

Common Mistakes And Troubleshooting

• Relying on the ro mount flag alone for a security-sensitive product. It can be remounted read-write by anything with sufficient privilege; use a construction-level read-only filesystem like squashfs where tamper resistance genuinely matters.
• Forgetting a single writable path. One overlooked daemon that logs directly into the root filesystem is enough to reintroduce the exact corruption risk you were trying to eliminate — audit boot logs on a fresh image for any Read-only file system errors.
• Seeding overlayfs’s upperdir from the wrong source. If upperdir is accidentally left on persistent storage instead of tmpfs, “temporary” changes silently become permanent and eat into your flash budget.
• Not testing an actual power-cut scenario. A read-only root protects against write corruption, but you still need to verify your writable islands (tmpfs-seeded /var/lib, persistent config partitions) tolerate power loss on their own.

Best Practices

• Prefer squashfs (or another construction-level read-only filesystem) for shipping images; use the ro mount flag only as a development-time convenience or defense-in-depth layer.
• Keep the list of writable islands as small and explicit as possible — every one is a place corruption can still happen.
• Add an automated boot-time check (like the demo module above) to your factory test suite so a regression is caught before units ship.
• Document exactly which paths are persistent, which are tmpfs-backed, and which are truly immutable — future maintainers (including future you) will need this map.

Performance And Security Considerations

A read-only root filesystem has essentially no performance downside and often a performance upside, since there is no journal or write-back activity competing for I/O bandwidth on the root device. Security-wise, this is one of the most impactful hardening techniques available on embedded Linux: with no write path into the base image, a wide class of persistence techniques used by malware simply do not work, and any runtime compromise is wiped by the next reboot unless it manages to plant itself in one of your (hopefully small and audited) writable islands.

Summary And Key Takeaways

• A read-only root filesystem is one of the single highest-leverage reliability and security techniques in embedded Linux.
• Mount-flag read-only (ro) is a good development-time step; construction-level read-only filesystems like squashfs are the stronger production guarantee.
• A small, well-audited set of writable islands — via symlinks to tmpfs or a dedicated persistent partition, or via overlayfs — covers the files that genuinely need to change.
• Verify the result with an automated check rather than assuming your build configuration did what you intended.

Conclusion

This lesson closes out the storage strategy chapter of this free embedded systems course by bringing every earlier piece together: MTD and UBI give you reliable access to raw flash, F2FS and ext4 handle general-purpose writable storage, tmpfs absorbs everything volatile, and squashfs plus a disciplined set of writable islands gives you a root filesystem that a power failure simply cannot corrupt. That combination — not any single filesystem choice — is what a production-grade embedded storage strategy actually looks like.

FAQ