PROT flags, O_RDONLY/O_RDWR/O_WRONLY, MAP_PRIVATE vs MAP_SHARED Memory Protection & File Access Mode

Linux system programming — June 15, 2026 · 0 Comment

 

Memory Protection & File Access Mode
Chapter 49 – Topic 2 | PROT flags, O_RDONLY/O_RDWR/O_WRONLY, MAP_PRIVATE vs MAP_SHARED

The Core Question

When you call mmap(), you pass two things that control access:

  • prot — the memory protection flags: PROT_READ, PROT_WRITE, PROT_EXEC, PROT_NONE
  • The mode the file was opened with — O_RDONLY, O_WRONLY, or O_RDWR

These two must be compatible. If they conflict, the kernel returns EACCES (permission denied). The rules are not always obvious, especially around O_WRONLY and on architectures where PROT_WRITE implies PROT_READ.

PROT Flags – What They Mean
Flag Meaning Typical Use
PROT_READ Pages can be read Read-only access to file data
PROT_WRITE Pages can be written Modify file data via mapping
PROT_EXEC Pages can be executed Load shared libraries (.so files)
PROT_NONE No access at all Guard pages, inaccessible regions

The Hardware Complication: PROT_WRITE Implies PROT_READ

On most hardware architectures (x86 is the classic example), the MMU does not support write-only pages. If a page can be written, it can also be read — the hardware just does not have a “write-only” protection mode. This means:

On write-implies-read architectures:
Specifying PROT_WRITE in mmap() automatically also grants PROT_READ, even if you did not ask for it. You cannot have a truly write-only mapping.

This has a direct consequence for O_WRONLY files (see below).

Compatibility Rules: PROT + Open Mode

File Open Mode MAP_SHARED MAP_PRIVATE Notes
O_RDWR ✔ All PROT combinations ✔ All PROT combinations Most flexible. Open with O_RDWR when you need PROT_WRITE.
O_WRONLY ✘ No combination works ✘ No combination works EACCES always. PROT_WRITE implies PROT_READ; O_WRONLY forbids reads.
O_RDONLY ✔ PROT_READ
✔ PROT_READ | PROT_EXEC
✘ PROT_WRITE (EACCES)		 ✔ Any PROT combination MAP_PRIVATE writes never go to the file, so O_RDONLY is fine even with PROT_WRITE.

Rule 1 – O_RDWR: The Easy Case

Opening a file with O_RDWR is compatible with every combination of PROT flags, for both MAP_SHARED and MAP_PRIVATE. This is the safest choice when you need write access to a mapped file.

Code Example 1 – O_RDWR with MAP_SHARED and PROT_READ|PROT_WRITE (the standard pattern)
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <unistd.h>

int main(void)
{
    int fd;
    char *addr;
    struct stat sb;

    /* O_RDWR is compatible with PROT_READ | PROT_WRITE, MAP_SHARED */
    fd = open("data.bin", O_RDWR);
    if (fd == -1) { perror("open"); exit(EXIT_FAILURE); }

    fstat(fd, &sb);

    addr = mmap(NULL, sb.st_size,
                PROT_READ | PROT_WRITE,   /* Needs O_RDWR or O_RDONLY+MAP_PRIVATE */
                MAP_SHARED,               /* Changes are written to file */
                fd, 0);
    if (addr == MAP_FAILED) { perror("mmap"); exit(EXIT_FAILURE); }

    close(fd);

    /* Read and modify file via memory */
    printf("First byte: %d\n", (unsigned char)addr[0]);
    addr[0] = 0xFF;  /* This change will be written to the file on disk */
    printf("Modified first byte to: %d\n", (unsigned char)addr[0]);

    munmap(addr, sb.st_size);
    return 0;
}

Rule 2 – O_WRONLY: Why It Always Fails with mmap()

You might think: “I only want to write to the file through the mapping, so O_WRONLY should work.” Unfortunately, it does not — and for a subtle hardware reason:

Why PROT_WRITE + O_WRONLY = EACCES
What you want:

Open file write-only (O_WRONLY)
Map it with PROT_WRITE
Write to the mapping
✘ Forbidden

 Why the kernel blocks it:

Hardware: PROT_WRITE implies PROT_READ
So the mapping would actually have PROT_READ too
But file is O_WRONLY — reading is not allowed
Conflict → EACCES
Solution: Always use O_RDWR when you need PROT_WRITE in a mapping.
Code Example 2 – Demonstrating O_WRONLY failure (expected EACCES)
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <errno.h>
#include <string.h>

int main(void)
{
    int fd;
    char *addr;

    /* Open file WRITE-ONLY */
    fd = open("data.bin", O_WRONLY);
    if (fd == -1) { perror("open"); exit(EXIT_FAILURE); }

    /* Try to mmap with PROT_WRITE | MAP_SHARED.
     * This WILL fail with EACCES on architectures where
     * PROT_WRITE implies PROT_READ (e.g., x86, ARM).
     */
    addr = mmap(NULL, 4096,
                PROT_WRITE,    /* On x86, this also implicitly includes PROT_READ */
                MAP_SHARED,
                fd, 0);

    if (addr == MAP_FAILED) {
        /* Expected: errno = EACCES */
        printf("mmap failed as expected: %s (errno=%d)\n",
               strerror(errno), errno);
        /* Output: mmap failed as expected: Permission denied (errno=13) */
    } else {
        printf("mmap unexpectedly succeeded (not on x86?)\n");
        munmap(addr, 4096);
    }

    close(fd);

    /* FIX: use O_RDWR instead */
    fd = open("data.bin", O_RDWR);
    addr = mmap(NULL, 4096, PROT_WRITE, MAP_SHARED, fd, 0);
    if (addr == MAP_FAILED) {
        perror("mmap with O_RDWR");
    } else {
        printf("mmap with O_RDWR and PROT_WRITE succeeded.\n");
        munmap(addr, 4096);
    }
    close(fd);

    return 0;
}

Rule 3 – O_RDONLY: Different for MAP_SHARED vs MAP_PRIVATE

O_RDONLY has different rules depending on whether you use MAP_SHARED or MAP_PRIVATE:

O_RDONLY + MAP_SHARED
  • ✔ PROT_READ
  • ✔ PROT_READ | PROT_EXEC
  • ✘ PROT_WRITE → EACCES
  • ✘ PROT_WRITE | PROT_READ → EACCES
MAP_SHARED writes go to the file. O_RDONLY forbids writing the file. Conflict → EACCES.

O_RDONLY + MAP_PRIVATE
  • ✔ PROT_READ
  • ✔ PROT_WRITE
  • ✔ PROT_READ | PROT_WRITE
  • ✔ PROT_EXEC
  • ✔ Any combination
MAP_PRIVATE writes NEVER reach the file (copy-on-write). So read-only file is fine.

The reason MAP_PRIVATE + O_RDONLY + PROT_WRITE is allowed is the copy-on-write (COW) mechanism: when you write to a private mapping, the kernel creates a private copy of that page for your process. The original file is never touched. There is no conflict with the read-only file descriptor.

Code Example 3 – O_RDONLY + MAP_PRIVATE + PROT_WRITE (Copy-on-Write)
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <unistd.h>

int main(void)
{
    int fd;
    char *addr;
    struct stat sb;

    /* Open file READ-ONLY */
    fd = open("data.bin", O_RDONLY);
    if (fd == -1) { perror("open"); exit(EXIT_FAILURE); }

    fstat(fd, &sb);

    /* MAP_PRIVATE + O_RDONLY + PROT_WRITE is VALID.
     * Writes create a private (COW) copy in the process; the file is untouched.
     */
    addr = mmap(NULL, sb.st_size,
                PROT_READ | PROT_WRITE,
                MAP_PRIVATE,              /* Copy-on-write: writes stay private */
                fd, 0);
    if (addr == MAP_FAILED) { perror("mmap"); exit(EXIT_FAILURE); }

    close(fd);

    printf("Original first byte: %d\n", (unsigned char)addr[0]);

    /* This write creates a private page copy. The file on disk is unchanged. */
    addr[0] = 0xAB;
    printf("Modified in-memory: %d\n", (unsigned char)addr[0]);
    printf("File on disk is still unchanged (verify with xxd data.bin).\n");

    munmap(addr, sb.st_size);
    return 0;
}

Code Example 4 – Handling EACCES from incompatible prot/mode combination
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <errno.h>
#include <string.h>
#include <sys/stat.h>

/* A helper that explains why mmap() failed */
void explain_mmap_error(int err)
{
    switch (err) {
    case EACCES:
        printf("EACCES: prot/flags incompatible with file open mode.\n"
               "  - PROT_WRITE + MAP_SHARED requires O_RDWR\n"
               "  - O_WRONLY never works with mmap()\n");
        break;
    case EINVAL:
        printf("EINVAL: bad argument (e.g., offset not page-aligned, bad flags)\n");
        break;
    case ENOMEM:
        printf("ENOMEM: not enough virtual address space\n");
        break;
    default:
        printf("mmap error: %s\n", strerror(err));
    }
}

int main(void)
{
    int fd;
    void *addr;
    struct stat sb;

    /* --- Test 1: O_RDONLY + MAP_SHARED + PROT_WRITE → EACCES --- */
    fd = open("data.bin", O_RDONLY);
    fstat(fd, &sb);

    addr = mmap(NULL, sb.st_size,
                PROT_READ | PROT_WRITE,
                MAP_SHARED,    /* Will fail: can't write through MAP_SHARED to O_RDONLY */
                fd, 0);
    if (addr == MAP_FAILED) {
        printf("Test 1 (O_RDONLY + MAP_SHARED + PROT_WRITE): ");
        explain_mmap_error(errno);
    }
    close(fd);

    /* --- Test 2: O_RDONLY + MAP_PRIVATE + PROT_WRITE → succeeds --- */
    fd = open("data.bin", O_RDONLY);
    addr = mmap(NULL, sb.st_size,
                PROT_READ | PROT_WRITE,
                MAP_PRIVATE,   /* OK: writes are COW, never reach the file */
                fd, 0);
    if (addr == MAP_FAILED) {
        printf("Test 2 failed unexpectedly: %s\n", strerror(errno));
    } else {
        printf("Test 2 (O_RDONLY + MAP_PRIVATE + PROT_WRITE): OK\n");
        munmap(addr, sb.st_size);
    }
    close(fd);

    return 0;
}

Code Example 5 – PROT_EXEC mapping (loading executable code)
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <unistd.h>

/* This is essentially what the dynamic linker (ld.so) does when loading .so files.
 * It maps the .text (code) section with PROT_READ | PROT_EXEC.
 * O_RDONLY is sufficient because PROT_EXEC does not include PROT_WRITE.
 */
int main(int argc, char *argv[])
{
    int fd;
    void *code_map;
    struct stat sb;

    if (argc < 2) {
        fprintf(stderr, "Usage: %s <shared-library.so>\n", argv[0]);
        exit(EXIT_FAILURE);
    }

    /* Open the shared library read-only */
    fd = open(argv[1], O_RDONLY);
    if (fd == -1) { perror("open"); exit(EXIT_FAILURE); }

    fstat(fd, &sb);

    /* Map the .so file as executable (read + execute, no write) */
    code_map = mmap(NULL, sb.st_size,
                    PROT_READ | PROT_EXEC,   /* Needs O_RDONLY or O_RDWR */
                    MAP_SHARED,
                    fd, 0);
    if (code_map == MAP_FAILED) {
        perror("mmap PROT_EXEC");
        exit(EXIT_FAILURE);
    }

    close(fd);
    printf("Mapped %s at %p (%ld bytes) as executable.\n",
           argv[1], code_map, (long)sb.st_size);

    /* Note: actually calling into code_map requires knowing the exact
     * function offsets (ELF parsing). This is for illustration only.
     */

    munmap(code_map, sb.st_size);
    return 0;
}

Complete Compatibility Summary
Open Mode Flags PROT Requested Result
O_RDWR MAP_SHARED PROT_READ|PROT_WRITE ✔ OK
O_RDWR MAP_PRIVATE PROT_READ|PROT_WRITE ✔ OK
O_RDONLY MAP_SHARED PROT_READ ✔ OK
O_RDONLY MAP_SHARED PROT_READ|PROT_EXEC ✔ OK
O_RDONLY MAP_SHARED PROT_WRITE ✘ EACCES
O_RDONLY MAP_PRIVATE PROT_WRITE ✔ OK (COW)
O_WRONLY MAP_SHARED Any ✘ EACCES always
O_WRONLY MAP_PRIVATE Any ✘ EACCES always

Interview Questions – Memory Protection & File Access Mode
Q1. Why does mmap() fail with EACCES when the file is opened O_WRONLY?
On most hardware, PROT_WRITE implicitly includes PROT_READ because the MMU cannot create a write-only page. So any mapping with PROT_WRITE would allow reading the page content. But the file was opened O_WRONLY, which explicitly forbids reading the file’s original content. This contradiction causes the kernel to return EACCES for every combination of PROT flags.
Q2. Can you use PROT_WRITE with a file opened O_RDONLY? When?
Yes — but only with MAP_PRIVATE. Private mappings use copy-on-write: when you first write to a page, the kernel copies that page into private memory for your process. The original file page is never modified. Since the file itself is never written, opening it O_RDONLY is not a conflict. This is how the dynamic linker loads writeable data sections from read-only executable files.
Q3. What does PROT_EXEC do and what open mode does it require?
PROT_EXEC marks pages as executable — the CPU can fetch and execute instructions from them. It requires the file to be opened O_RDONLY or O_RDWR. It is typically combined with PROT_READ (as PROT_READ | PROT_EXEC) and used with MAP_SHARED for loading shared libraries and executable segments.
Q4. What is the difference between MAP_SHARED and MAP_PRIVATE in terms of file writes?
MAP_SHARED: Writes to the mapping are eventually propagated back to the underlying file. All processes that map the same file with MAP_SHARED see each other’s changes.
MAP_PRIVATE: Writes use copy-on-write. Each process gets its own private copy of modified pages. The file on disk is never changed. Other processes mapping the same file do not see the changes.
Q5. What error does mmap() return when the prot/flag combination is incompatible with the file open mode?
EACCES (errno 13, “Permission denied”). This is distinct from EINVAL (which is for bad arguments like an unaligned offset) and ENOMEM (insufficient address space).
Q6. When would you use PROT_NONE?
PROT_NONE creates a region that cannot be read, written, or executed — any access raises SIGSEGV. Common uses: (1) Guard pages between stack and other memory to catch stack overflows. (2) Reserving virtual address space for future use without consuming physical memory. (3) Implementing memory-safe languages that need hard boundaries. You can later use mprotect() to change PROT_NONE pages to accessible ones.

Leave a Reply

Your email address will not be published. Required fields are marked *

© 2026 embeddedpathashala.com - WordPress Video Theme by WPEnjoy