← Previous Lecture  |  Next Lecture →

What You Will Learn

• Why a naive register read-modify-write sequence is a data race
• Register bit numbering (LSB/MSB) and register banking basics
• Using ioread8() / iowrite8() safely with a lock
• Modern atomic bit-operation helpers the kernel provides for RMW-style updates
• An original, kernel 6.x driver example protecting a control register

Prerequisites

• Basic understanding of memory-mapped I/O (MMIO) and peripheral registers
• Familiarity with spinlocks (covered earlier in this Kernel Synchronization series)
• Comfortable reading simple C driver code

Register Basics: Bits, Bytes, and Register Banking

A byte is made up of 8 bits, numbered from bit 0 (the Least Significant Bit, LSB) to bit 7 (the Most Significant Bit, MSB). The Linux kernel formally defines this as BITS_PER_BYTE in include/linux/bits.h.

A hardware register is a small addressable piece of memory inside a peripheral device, commonly 8, 16, or 32 bits wide, used for control, status, and configuration. Chip vendors frequently place several related registers next to each other in memory — a pattern called register banking — so that a driver can reach any register using a base address plus a fixed offset. For example, a modern GPIO-style controller might expose its registers like this:

#define GPIO_REG_BASE   0x4801'0000UL
#define GPIO_STATUS_REG (GPIO_REG_BASE + 0x00)
#define GPIO_CTRL_REG   (GPIO_REG_BASE + 0x04)

Suppose the datasheet says: “set bit 7 of the control register to enable the peripheral.” The textbook RMW sequence every driver author learns is:

1. Read the register’s current value into a temporary variable.
2. Modify the variable to the desired value.
3. Write the variable back to the register.

The Naive (and Racy) RMW Implementation

A first attempt using the standard MMIO helpers ioread8() and iowrite8() might look like this:

static void ep_enable_device(void __iomem *ctrl_reg)
{
	u8 tmp;

	tmp = ioread8(ctrl_reg);   /* READ  current value  */
	tmp |= BIT(7);             /* MODIFY: set bit 7     */
	iowrite8(tmp, ctrl_reg);   /* WRITE  new value back */
}

This looks correct — and it is, as long as only one thread of execution ever touches this register. The moment two CPUs, two kernel threads, or a thread and an interrupt handler can call this function concurrently, it becomes a classic data race.

A hardware register is really just a shared, writable memory location — exactly like any shared variable in software. So it forms a critical section and must be protected the same way any other shared data structure would be.

Fixing It With a Spinlock

The most direct fix is to wrap the RMW sequence in a spinlock, as covered earlier in this Kernel Synchronization series:

static DEFINE_SPINLOCK(ep_ctrl_lock);

static void ep_enable_device(void __iomem *ctrl_reg)
{
	unsigned long flags;
	u8 tmp;

	spin_lock_irqsave(&ep_ctrl_lock, flags);

	tmp = ioread8(ctrl_reg);
	tmp |= BIT(7);
	iowrite8(tmp, ctrl_reg);

	spin_unlock_irqrestore(&ep_ctrl_lock, flags);
}

spin_lock_irqsave() is used here (instead of a plain spin_lock()) because register access like this is often also performed from interrupt handlers, so we must disable local interrupts while holding the lock to avoid a self-deadlock, as explained in the earlier spinlock lectures of this series.

A Lighter Alternative: Kernel Atomic Bit Operations

When the “register” you are protecting is really an in-memory bitmap or flags word (rather than a physical hardware register reached through MMIO), the kernel offers dedicated atomic bit-operation helpers that perform the RMW sequence for you, in a single indivisible step, with no explicit lock required:

Function	Effect
set_bit(nr, addr)	Atomically sets bit nr
clear_bit(nr, addr)	Atomically clears bit nr
change_bit(nr, addr)	Atomically toggles bit nr
test_and_set_bit(nr, addr)	Sets bit nr, returns its previous value
test_and_clear_bit(nr, addr)	Clears bit nr, returns its previous value

These are ideal for driver-internal flag words (for example, a “device busy” bitmap shared between a kthread and an interrupt handler), while direct MMIO peripheral registers still generally need the spinlock-protected ioread8()/iowrite8() pattern shown above, since the hardware itself has no concept of an atomic bit-set instruction over a bus.

Original Example: A Protected Control Register Driver

Here is a complete, original demo driver (not from any textbook) that safely enables and disables a simulated peripheral using the spinlock-protected RMW pattern, alongside an atomic in-memory status flag using test_and_set_bit():

#include <linux/module.h>
#include <linux/spinlock.h>
#include <linux/io.h>
#include <linux/bitops.h>

#define EP_ENABLE_BIT   7
#define EP_BUSY_BIT     0

static DEFINE_SPINLOCK(ep_ctrl_lock);
static unsigned long ep_driver_flags;

/* ctrl_reg points to a real or simulated 8-bit MMIO control register */
static void ep_set_enable(void __iomem *ctrl_reg, bool enable)
{
	unsigned long flags;
	u8 tmp;

	spin_lock_irqsave(&ep_ctrl_lock, flags);

	tmp = ioread8(ctrl_reg);
	if (enable)
		tmp |= BIT(EP_ENABLE_BIT);
	else
		tmp &= ~BIT(EP_ENABLE_BIT);
	iowrite8(tmp, ctrl_reg);

	spin_unlock_irqrestore(&ep_ctrl_lock, flags);
}

/* Returns true if some other context already marked the driver busy */
static bool ep_try_mark_busy(void)
{
	return test_and_set_bit(EP_BUSY_BIT, &ep_driver_flags);
}

static void ep_clear_busy(void)
{
	clear_bit(EP_BUSY_BIT, &ep_driver_flags);
}

MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("EmbeddedPathashala RMW register + atomic bitops demo");

Notice the two different protection strategies used side by side: the physical ctrl_reg MMIO register needs an explicit spinlock around its RMW sequence, while the purely in-memory ep_driver_flags word gets race-free updates directly from test_and_set_bit() / clear_bit() with no manual locking at all.

Performance Considerations

• Keep the critical section around a register RMW as short as possible — do the bit math outside the lock where feasible, and only wrap the actual read/modify/write.
• Prefer set_bit()/clear_bit() over a spinlock when you are only ever touching an in-memory flags word, since the atomic bitops avoid lock overhead entirely.
• Avoid unnecessary _irqsave variants on registers that are never touched from interrupt context — a plain spin_lock() is cheaper when it is safe to use.

Security Considerations

• An unprotected RMW on a security-relevant control register (for example, one that gates DMA access) can be exploited by a racing thread to leave the device in an unintended, more permissive state.
• Always double-check which bits your RMW sequence preserves — an overly broad write can accidentally clear security or lockdown bits set by another part of the kernel.

Common Mistakes

• Assuming a “quick” register update doesn’t need a lock because it “happens fast” — races don’t care how fast the code runs, only whether two contexts can interleave.
• Forgetting _irqsave/_irqrestore when the same register can also be touched from an interrupt handler, causing a self-deadlock.
• Using set_bit()/clear_bit() directly on MMIO memory expecting hardware-level atomicity — these are for in-memory bitmaps, not a substitute for proper register locking.

Summary / Key Takeaways

• A hardware register is shared writable memory — RMW access to it is a critical section like any other.
• A naive read-modify-write on a register can silently lose bit updates under concurrency.
• spin_lock_irqsave()/spin_unlock_irqrestore() is the standard fix for MMIO register RMW sequences touched from interrupt context.
• For purely in-memory flag words, set_bit(), clear_bit(), and test_and_set_bit() give you atomic RMW semantics with no explicit locking.

Frequently Asked Questions (FAQ)

Q1. Why is reading then writing a hardware register not enough on its own?
Because between the read and the write, another CPU, thread, or interrupt handler can modify the same register, causing one of the two updates to be silently overwritten.

Q2. Do I always need a spinlock around ioread8()/iowrite8()?
Yes, whenever more than one execution context (thread, interrupt handler, or CPU) can access the same register concurrently — which is the normal case for most real drivers.

Q3. What is the difference between set_bit() and a spinlock-protected register update?
set_bit() gives atomic RMW semantics for an in-memory word using CPU-level atomic instructions; a hardware register update generally still needs an explicit lock because the bus/register itself has no equivalent atomic instruction.

Q4. Why use spin_lock_irqsave() instead of spin_lock() for registers?
Because many registers are also accessed from interrupt handlers; spin_lock_irqsave() disables local interrupts while holding the lock, preventing a self-deadlock on the same CPU.

Q5. What is register banking?
A hardware design pattern where multiple related registers are placed sequentially in memory, addressed using a base address plus a fixed offset per register.

Q6. Can RMW register bugs cause security issues?
Yes — a race on a control register can leave a device in an unintended state, including one with weaker access restrictions than intended.