← PREV_LEC  |  NEXT_LEC →

What Is a Cache?

A cache is a small, fast piece of memory that keeps copies of data pulled from a much larger and much slower store. The idea works because most programs repeatedly touch a small “working set” of data far more often than the rest. The first time a piece of data is requested it comes from the slow store and a copy is kept nearby; every later request for the same data is served from the fast copy instead, until the cache fills up and the least recently used entries are evicted to make room for new ones.

This single idea repeats at almost every layer of a computer: inside the CPU, inside the kernel, and inside applications such as web browsers.

CPU Caches: L1, L2, and L3

Modern CPUs have three levels of on-chip cache, ordered by size and speed:

Level	Typical size	Access latency	Scope
L1	32-64 KB	∼0.5-1 ns	Per core
L2	256 KB-2 MB	A few ns	Per core (usually)
L3	Several MB to tens of MB	Slower than L2, faster than RAM	Shared across all cores

Main memory access, by comparison, can take on the order of 100 ns, so a CPU that always had to wait on RAM would spend most of its time idle. The cache hierarchy hides most of that latency by keeping frequently used data as close to the execution units as possible.

The Linux Page Cache

The Linux page cache applies exactly the same idea to files. It is a cache of pages in RAM holding recently accessed file content, whether that content came from a regular filesystem file, a block device, or a memory-mapped file. On every read(), the kernel first checks whether the requested data is already sitting in the page cache; if it is, the data is returned immediately with no disk access at all. If not, the kernel reads from disk, returns the data, and keeps a copy cached for next time.

This is why the second time you read a large file it feels instant compared to the first time, even though your application code did not change at all.

Dirty Memory vs Clean Memory

When a process writes to a file, the write normally lands in the page cache first, not on disk. That cached page is now newer than the on-disk copy, and the kernel marks it dirty. A page whose content still matches disk is called clean. The process by which dirty pages are eventually written back to disk is called writeback, and it is the subject of the next lecture in this chapter.

You can see how many bytes of memory are currently dirty at any moment with:

cat /proc/meminfo | grep Dirty

Bypassing the Cache: O_SYNC and O_DIRECT

Sometimes an application needs different guarantees than “eventually written back.” Two open() flags change the default caching behaviour:

Flag	Guarantee
O_SYNC	write() does not return until data has actually reached the storage device
O_DIRECT	Bypasses the page cache for that file descriptor’s I/O; behaviour is filesystem-dependent and generally not recommended unless you have a specific reason, such as a database engine managing its own cache

Original Demo: ep_pagecache_demo

The following original user space program demonstrates the page cache effect directly by timing a cold read, a warm (cached) read, and an O_DIRECT read of the same file.

#define _GNU_SOURCE
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <time.h>
#include <stdlib.h>

#define BUF_SIZE (1 * 1024 * 1024)

static double read_once(const char *path, int extra_flags)
{
    struct timespec start, end;
    char *buf;
    int fd = open(path, O_RDONLY | extra_flags);
    if (fd < 0) { perror("open"); exit(1); }

    posix_memalign((void **)&buf, 4096, BUF_SIZE);

    clock_gettime(CLOCK_MONOTONIC, &start);
    ssize_t n = read(fd, buf, BUF_SIZE);
    clock_gettime(CLOCK_MONOTONIC, &end);

    free(buf);
    close(fd);

    if (n < 0) { perror("read"); exit(1); }

    return (end.tv_sec - start.tv_sec) * 1000.0 +
           (end.tv_nsec - start.tv_nsec) / 1e6;
}

int main(int argc, char **argv)
{
    if (argc < 2) {
        fprintf(stderr, "usage: %s <file>\n", argv[0]);
        return 1;
    }

    printf("First (cold) read:   %.3f ms\n", read_once(argv[1], 0));
    printf("Second (cached) read: %.3f ms\n", read_once(argv[1], 0));
    printf("O_DIRECT read:        %.3f ms\n", read_once(argv[1], O_DIRECT));
    return 0;
}

Build and Run Steps

gcc ep_pagecache_demo.c -o ep_pagecache_demo

# Create a 1 MB test file first
dd if=/dev/urandom of=testfile.bin bs=1M count=1

# Drop the page cache so the first read is truly cold (needs root)
echo 1 | sudo tee /proc/sys/vm/drop_caches

./ep_pagecache_demo testfile.bin

Expected Output

First (cold) read:   2.145 ms
Second (cached) read: 0.061 ms
O_DIRECT read:        2.310 ms

The cold read pays the cost of an actual disk (or SSD) access. The second read is served entirely from the page cache and is roughly 30-40 times faster in this run. The O_DIRECT read pays a similar cost to the cold read every time, because it deliberately bypasses the cache, confirming that the page cache, not some other factor, is responsible for the speedup.

Specialized Caches Outside the Kernel

The same caching idea appears in user space too. A web browser keeps a disk cache of recently visited pages and images so a second visit avoids a slow network round trip. Language runtime libraries and applications build their own in-memory caches for the same reason: predicting what will be needed again and keeping a copy close at hand.

Common Mistakes

Mistake	Consequence
Benchmarking file I/O without dropping caches first	Results measure the page cache, not the storage device
Using O_DIRECT casually for “faster” I/O	Usually slower for typical workloads, since it disables read-ahead and caching benefits
Assuming a completed write() means data is on disk	Data may still be dirty in the page cache; use fsync() if that guarantee is required

Best Practices

• Let the page cache do its job for ordinary application I/O; do not disable it without a measured reason
• Use O_DIRECT only for software that manages its own cache, such as database engines
• Call fsync() or fdatasync() explicitly whenever a write must be durable before continuing

Performance and Security Considerations

The page cache is shared system-wide, which is efficient but also means sensitive file contents can linger in RAM after a process closes a file. On systems handling confidential data, consider whether cache eviction policies or encrypted-at-rest storage are appropriate alongside normal page cache behaviour.

Real World Use Cases

• Web servers relying on the page cache to serve static files quickly after the first request
• Database engines using O_DIRECT to manage their own buffer pool instead of double-caching data
• Build systems that feel dramatically faster on a second run because source files are already cached

Summary and Key Takeaways

• Caching trades memory for speed at every layer, from CPU L1 cache to the Linux page cache
• The page cache serves repeated file reads from RAM instead of disk automatically
• Dirty pages are writes that have not yet reached disk; clean pages match the on-disk copy
• O_SYNC and O_DIRECT exist for the specific cases where the default caching behaviour is wrong for the application

Frequently Asked Questions

What is the difference between the page cache and the CPU cache?

The CPU cache (L1/L2/L3) caches arbitrary memory contents close to the processor cores. The page cache is a kernel subsystem that caches file data in RAM to avoid repeated disk access. They operate at completely different layers of the system.

Does the page cache ever get freed automatically?

Yes. The kernel reclaims clean page cache pages under memory pressure, since they can be re-read from disk if needed again. Dirty pages must be written back before they can be reclaimed.

Is it safe to drop caches with /proc/sys/vm/drop_caches?

It is safe on a running system but temporarily reduces performance, since subsequent reads must go back to disk. It is intended for benchmarking and testing, not routine use.

Why is my second read of a file so much faster?

Because it is served from the Linux page cache in RAM instead of the underlying storage device.

When should I actually use O_DIRECT?

Mainly in software like database engines that already implement their own buffer cache and want to avoid the overhead and duplication of also being cached by the kernel.

What does it mean for a page to be dirty?

A dirty page holds data that has been written in RAM but not yet flushed to the backing storage device, so the cached and on-disk copies are out of sync.

 

← PREV_LEC  |  NEXT_LEC →