← PREV_LEC  |  NEXT_LEC →

Three Caching Strategies

Any system that caches writes has to pick one of three basic strategies, each with different trade-offs between safety, latency, and system load.

Strategy	Behaviour	Trade-off
Write-through	Every write updates both the cache and permanent storage immediately	Safest against data loss, but every write pays disk latency
Write-around	Writes go to storage and invalidate the cache entry immediately	Avoids polluting the cache with data that will not be re-read, but subsequent reads are slow and cache invalidation itself has overhead
Write-back	Writes update only the cache; storage is updated later, in bulk, at chosen intervals	Fastest for the writing process; carries a data-loss window if the system crashes before writeback occurs

Why Linux Uses Write-Back

Linux’s page cache uses write-back caching. A write updates the cached page and marks it dirty in the kernel’s tracking structures; the actual disk write happens later, at intervals or under specific conditions, rather than on every single write() call. This design gives three concrete benefits: reduced write latency for applications, fewer and larger physical I/O operations (storage devices are much more efficient at a small number of large operations than many small ones), and reduced wear on flash-based storage such as eMMC found in embedded systems.

You can see current dirty memory at any time with:

cat /proc/meminfo | grep Dirty

What Triggers a Writeback

Dirty pages are not written back arbitrarily. Three conditions trigger writeback in current kernels:

1. Memory pressure: when free memory drops below a threshold, the kernel writes back dirty pages so they can be reclaimed
2. Age: the oldest dirty data is written back once it has been dirty longer than a configured expiry time, so nothing stays dirty indefinitely
3. Explicit request: a process calling sync(), fsync(), or fdatasync() forces an on-demand writeback

Modernization Note: From pdflush to Per-BDI Flusher Threads

Older references describe a single global kernel thread family called pdflush responsible for all writeback across every backing device. That design was replaced years ago by per-BDI flusher threads (BDI stands for Backing Device Info): each storage device gets its own dedicated bdi_writeback worker, scheduled through the kernel workqueue via wb_workfn(), instead of one thread serving every device in the system. This change removed contention between unrelated devices and let writeback scale with the number of storage devices actually present.

You can list the backing devices currently registered and inspect their writeback statistics directly:

ls /sys/class/bdi/
cat /sys/class/bdi/<bdi-id>/stats

The three writeback triggers above map to specific sysctl knobs you can tune:

Sysctl (under /proc/sys/vm)	Meaning
dirty_writeback_centisecs	How often the periodic flusher wakes up, in hundredths of a second
dirty_expire_centisecs	How old a dirty page must be before it is eligible for the periodic flush
dirty_background_ratio	Percentage of memory that can be dirty before background writeback starts, without stalling writers
dirty_ratio	Percentage of memory that can be dirty before the kernel throttles the writing process directly

Original Demo: ep_dirty_watch

The following original user space program dirties a mapped file’s pages and reports how the system-wide Dirty counter in /proc/meminfo rises, then falls again once msync() forces writeback.

#include <stdio.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <unistd.h>
#include <string.h>

#define FILE_SIZE (8 * 1024 * 1024)

static void print_dirty(const char *label)
{
    char line[256];
    FILE *f = fopen("/proc/meminfo", "r");
    while (fgets(line, sizeof(line), f)) {
        if (strncmp(line, "Dirty:", 6) == 0) {
            printf("%s -> %s", label, line);
            break;
        }
    }
    fclose(f);
}

int main(void)
{
    int fd = open("ep_dirty_test.bin", O_RDWR | O_CREAT, 0644);
    ftruncate(fd, FILE_SIZE);

    char *map = mmap(NULL, FILE_SIZE, PROT_WRITE, MAP_SHARED, fd, 0);

    print_dirty("Before writes");

    memset(map, 0x42, FILE_SIZE);

    print_dirty("After writes (before msync)");

    msync(map, FILE_SIZE, MS_SYNC);

    print_dirty("After msync (forced writeback)");

    munmap(map, FILE_SIZE);
    close(fd);
    return 0;
}

Build and Run Steps

gcc ep_dirty_watch.c -o ep_dirty_watch
./ep_dirty_watch

Expected Output

Before writes -> Dirty:               412 kB
After writes (before msync) -> Dirty:            8604 kB
After msync (forced writeback) -> Dirty:           420 kB

The Dirty counter jumps by roughly the size of the file right after the memory writes, confirming the pages are cached and marked dirty rather than written immediately. After msync(MS_SYNC) forces an explicit writeback, the counter drops back close to its baseline, because the pages have now been flushed and marked clean. Exact numbers vary by system load and prior dirty memory.

Common Mistakes

Mistake	Consequence
Assuming write() means data is safely on disk	Data may sit dirty in the page cache for up to dirty_expire_centisecs before writeback
Tuning dirty_ratio too high on memory-constrained systems	Large writeback bursts can stall I/O for extended periods
Looking for a single “pdflush” thread in ps output on a modern kernel	It does not exist anymore; writeback now runs through per-BDI kernel workqueues instead

Best Practices

• Use fsync()/fdatasync() explicitly wherever durability matters, rather than assuming write-back timing
• On embedded systems writing to eMMC or flash, prefer the default write-back behaviour to reduce wear, and only shorten dirty_expire_centisecs if the data-loss window is unacceptable for your application
• Monitor /sys/class/bdi/<id>/stats when diagnosing unexpected I/O stalls tied to writeback

Performance and Security Considerations

Write-back caching trades a data-loss window for throughput and reduced storage wear. On systems where power loss or crashes are a real risk (many embedded devices fall into this category), consider a battery-backed write cache, a journaling filesystem, or explicit fsync() calls at safe checkpoints in your application logic.

Real World Use Cases

• Embedded data loggers that batch sensor writes and rely on write-back to reduce eMMC wear
• Databases that call fsync() at transaction commit points to guarantee durability despite write-back caching elsewhere
• Build systems and package managers that benefit from write-back’s reduced I/O overhead during large file operations

Summary and Key Takeaways

• Linux uses write-back caching: writes land in the page cache first and are marked dirty, then flushed later
• Writeback is triggered by memory pressure, dirty page age, or an explicit sync()/fsync() call
• Per-BDI flusher threads (bdi_writeback, driven by wb_workfn() through the kernel workqueue) replaced the old single global pdflush thread
• dirty_writeback_centisecs, dirty_expire_centisecs, dirty_background_ratio, and dirty_ratio are the key tuning knobs

Frequently Asked Questions

What is the difference between write-through and write-back caching?

Write-through updates storage on every write immediately. Write-back updates only the cache immediately and defers the storage update to a later time, trading a small risk window for much better performance.

Does pdflush still exist in the Linux kernel?

No. pdflush was replaced by per-BDI flusher threads years ago; each backing device now gets its own writeback worker scheduled through the kernel workqueue.

How do I check how much memory is currently dirty?

Run cat /proc/meminfo | grep Dirty to see the current system-wide dirty page total.

What triggers Linux to write dirty pages back to disk?

Three things: memory pressure falling below a threshold, dirty data exceeding its configured expiry age, or an explicit sync()/fsync()/fdatasync() call from a process.

Can I make writes safer without disabling write-back caching entirely?

Yes, call fsync() or fdatasync() at the specific points in your application where durability actually matters, instead of disabling caching system-wide.

Why does write-back caching help embedded storage like eMMC?

It merges many small writes into fewer, larger operations, which reduces wear leveling cycles and extends the storage device’s lifetime.

 

← PREV_LEC  |  NEXT_LEC →