Shared Memory in Virtual Memory Virtual Address Layout and /proc/PID/maps

 

Shared Memory in Virtual Memory
Virtual Address Layout and /proc/PID/maps · Chapter 48 | EmbeddedPathashala

Where Does Shared Memory Live in Your Process?

Understanding where the kernel places a shared memory segment in a process’s virtual address space helps you avoid addressing bugs and understand memory layout tools. On x86-32 Linux, the kernel places shared memory segments, mmap() mappings, and shared libraries all in the same “middle region” of the virtual address space between the heap and the stack.

The constant TASK_UNMAPPED_BASE (0x40000000 on x86-32, one-third of the address space on most architectures) is where the kernel starts looking for space to place these mappings. The actual location can vary based on kernel version and the process’s RLIMIT_STACK setting.

x86-32 Process Virtual Memory Layout

Here is the standard layout from TLPI Chapter 48, Figure 48-2:

0xC0000000 Kernel space
Stack
grows downward ↓
argv, environ argv, environ
⋮ (unallocated)
← Mapped here Shared memory segments
mmap() mappings
Shared libraries (.so)
(placed starting at TASK_UNMAPPED_BASE)
⋮ (reserved for heap growth)
program break → Heap
grows upward ↑
bss Uninitialized data (bss)
Initialized data
0x08048000 Text (program code)
Reserved (unmapped)
0x00000000 NULL address

x86-32 virtual address space (3 GB user, 1 GB kernel)
TASK_UNMAPPED_BASE = 0x40000000

The region starting at TASK_UNMAPPED_BASE (0x40000000) is where the kernel places:

  • System V shared memory segments (attached via shmat())
  • Anonymous and file-backed mmap() mappings
  • Shared libraries (.so files, loaded at runtime)

This region grows from 0x40000000 upward. The heap grows from below and the stack comes down from 0xC0000000, leaving a large gap in between for all these mappings.

/proc/PID/maps — Inspecting Memory Mappings

Linux provides a virtual file /proc/<PID>/maps that shows every memory mapping in the process’s address space. This is invaluable for debugging shared memory and understanding the layout.

Reading /proc/PID/maps in a running program:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/shm.h>

void print_maps(void)
{
    char path[64];
    char line[256];
    FILE *f;

    snprintf(path, sizeof(path), "/proc/%d/maps", getpid());
    f = fopen(path, "r");
    if (!f) { perror("fopen /proc/PID/maps"); return; }

    printf("\n=== /proc/%d/maps ===\n", getpid());
    while (fgets(line, sizeof(line), f))
        fputs(line, stdout);

    fclose(f);
}

int main(void)
{
    /* Create and attach two shared memory segments */
    int shmid1 = shmget(IPC_PRIVATE, 102400,   IPC_CREAT | 0660);
    int shmid2 = shmget(IPC_PRIVATE, 3276800,  IPC_CREAT | 0660);

    void *p1 = shmat(shmid1, NULL, 0);
    void *p2 = shmat(shmid2, NULL, 0);

    printf("shmid1=%d attached at %p\n", shmid1, p1);
    printf("shmid2=%d attached at %p\n", shmid2, p2);

    /* Print the maps to see where they ended up */
    print_maps();

    shmdt(p1); shmdt(p2);
    shmctl(shmid1, IPC_RMID, NULL);
    shmctl(shmid2, IPC_RMID, NULL);
    return 0;
}

Typical /proc/PID/maps output (from TLPI, Listing 48-4):

08048000-0804a000 r-xp 00000000 08:05 5526989  /home/mtk/svshm_attach
0804a000-0804b000 r--p 00001000 08:05 5526989  /home/mtk/svshm_attach
0804b000-0804c000 rw-p 00002000 08:05 5526989  /home/mtk/svshm_attach

b7bed000-b7f0d000 rw-s 00000000 00:09 9666565  /SYSV00000000 (deleted)
b7f0d000-b7f26000 rw-s 00000000 00:09 9633796  /SYSV00000000 (deleted)

b7f27000-b8064000 r-xp 00000000 08:06 122031   /lib/libc-2.8.so
b8064000-b8066000 r--p 0013d000 08:06 122031   /lib/libc-2.8.so
b8066000-b8067000 rw-p 0013f000 08:06 122031   /lib/libc-2.8.so

b8083000-b809e000 r-xp 00000000 08:06 122125   /lib/ld-2.8.so
b809e000-b809f000 r--p 0001a000 08:06 122125   /lib/ld-2.8.so
b809f000-b80a0000 rw-p 0001b000 08:06 122125   /lib/ld-2.8.so

bfd8a000-bfda0000 rw-p bffea000 00:00 0        [stack]
ffffe000-fffff000 r-xp 00000000 00:00 0        [vdso]

Understanding the /proc/PID/maps Format

Each line has 6 fields:

Field Example Meaning
Address range b7bed000-b7f0d000 Start and end virtual address of mapping
Permissions rw-s r=read, w=write, x=execute, p=private, s=shared. Shared memory shows ‘s’.
Offset 00000000 Offset into the file/segment. For shared memory, always 0.
Device 00:09 Major:minor device. Shared memory is on device 00:09 (tmpfs).
Inode 9666565 Inode number. For System V shm, this is the shmid!
Pathname /SYSV00000000 (deleted) System V shared memory shows as “/SYSV<key_hex> (deleted)”. “(deleted)” means IPC_RMID was already called.

Key insight: The inode field for a System V shared memory segment is the shmid. So you can correlate /proc/PID/maps entries with ipcs -m output by matching the inode number to the shmid column.

Examining Shared Memory with Shell Tools
# Step 1: Create two shared memory segments
./svshm_create -p 102400    # Creates ~100KB segment
9633796                      # shmid printed by program
./svshm_create -p 3276800   # Creates ~3.2MB segment
9666565

# Step 2: Attach them in a process (which goes to sleep)
./svshm_attach 9633796:0 9666565:0 &
# Output: SHMLBA = 4096 (0x1000), PID = 9903
#         1: 9633796:0 ==> 0xb7f0d000
#         2: 9666565:0 ==> 0xb7bed000

# Step 3: While it sleeps, read its maps file
cat /proc/9903/maps | grep SYSV
# b7bed000-b7f0d000 rw-s 00000000 00:09 9666565  /SYSV00000000 (deleted)
# b7f0d000-b7f26000 rw-s 00000000 00:09 9633796  /SYSV00000000 (deleted)

# The sizes match:
# b7f0d000 - b7bed000 = 0x320000 = 3276800 bytes  ✓ (shmid 9666565)
# b7f26000 - b7f0d000 = 0x19000  = 102400 bytes   ✓ (shmid 9633796)

# Step 4: Check with ipcs
ipcs -m
# key      shmid  owner  perms  bytes    nattch  status
# 0x0000   9633796  root  660  102400    1
# 0x0000   9666565  root  660  3276800   1

# Step 5: Check /proc/PID/smaps for more detail (since kernel 2.6.14)
cat /proc/9903/smaps | grep -A 8 "b7bed000"

/proc/PID/smaps — Detailed Memory Statistics

Since Linux kernel 2.6.14, /proc/PID/smaps provides additional memory statistics per mapping — more detail than /proc/PID/maps:

/* Reading smaps in a program */
#include <stdio.h>
#include <string.h>
#include <unistd.h>

void print_smaps_for_shm(void)
{
    char path[64];
    char line[256];
    int in_shm_region = 0;
    FILE *f;

    snprintf(path, sizeof(path), "/proc/%d/smaps", getpid());
    f = fopen(path, "r");
    if (!f) return;

    while (fgets(line, sizeof(line), f)) {
        /* Detect SYSV shared memory regions */
        if (strstr(line, "SYSV")) {
            in_shm_region = 1;
            printf("--- SHM Region ---\n");
        } else if (in_shm_region && line[0] != ' ' && isxdigit(line[0])) {
            in_shm_region = 0;  /* Next region started */
        }
        if (in_shm_region)
            fputs(line, stdout);
    }
    fclose(f);
}

/* Typical smaps output for a shared memory segment:
b7bed000-b7f0d000 rw-s 00000000 00:09 9666565  /SYSV00000000 (deleted)
Size:           3200 kB       <-- virtual size
Rss:            3200 kB       <-- resident (in RAM)
Pss:            1600 kB       <-- proportional (split among 2 processes)
Shared_Clean:      0 kB
Shared_Dirty:   3200 kB       <-- modified shared pages
Private_Clean:     0 kB
Private_Dirty:     0 kB
Referenced:     3200 kB
Swap:              0 kB
*/

Key smaps fields explained:

  • Size: Virtual size of the mapping
  • Rss: Resident Set Size — pages actually in RAM now
  • Pss: Proportional Share Size — Rss divided equally among all processes sharing the pages
  • Shared_Dirty: Modified pages shared with other processes
  • Swap: Pages currently swapped out (0 if SHM_LOCK is used)

TASK_UNMAPPED_BASE — The Starting Point for Mappings

TASK_UNMAPPED_BASE is a kernel constant that defines where the kernel starts looking for free virtual address space when placing shared memory segments and mmap() mappings (when no address is specified by the caller).

Architecture TASK_UNMAPPED_BASE Total User Space
x86-32 0x40000000 (1 GB) 3 GB (0x0 to 0xBFFFFFFF)
x86-64 ASLR-based, typically 1/3 of address space 128 TB (47-bit addresses)
ARM (32-bit) 0x40000000 3 GB
/* You can change TASK_UNMAPPED_BASE by recompiling the kernel.
   On a running system, check it via /proc:

   $ cat /proc/sys/vm/mmap_min_addr
   65536

   This is the minimum allowed mmap address, not TASK_UNMAPPED_BASE.
   TASK_UNMAPPED_BASE is a compile-time kernel constant.
*/

/* In practice on x86-64 with ASLR:
   shmat() places segments at randomized addresses above TASK_UNMAPPED_BASE.
   That's why addresses change on every run when ASLR is enabled.

   To see ASLR setting: */
#include <stdio.h>
#include <stdlib.h>

int main(void)
{
    FILE *f = fopen("/proc/sys/kernel/randomize_va_space", "r");
    if (f) {
        int val;
        fscanf(f, "%d", &val);
        fclose(f);
        /* 0 = ASLR disabled
           1 = stack, mmap ASLR
           2 = stack, mmap, heap ASLR (default) */
        printf("ASLR setting: %d\n", val);
    }
    return 0;
}

Attaching at a Specific Address — Why It’s Tricky

If you want all processes to attach a segment at the same virtual address (so absolute pointers stored in the segment remain valid), you can pass a non-NULL address to shmat(). But this is fragile because:

  • The address might already be in use (by a library or another mapping)
  • Address Space Layout Randomization (ASLR) means library load addresses change each run
  • Different machines may have different memory layouts
/* Risky — specifying an explicit attach address */
#include <stdio.h>
#include <sys/shm.h>

#define WANTED_ADDR  ((void *) 0x50000000)

int main(void)
{
    int   shmid = shmget(IPC_PRIVATE, 4096, IPC_CREAT | 0660);
    void *shmp;

    /* Try to attach at a specific address */
    shmp = shmat(shmid, WANTED_ADDR, SHM_RND);
    if (shmp == (void *) -1) {
        perror("shmat at specific address failed");
        /* Fall back to kernel-chosen address */
        shmp = shmat(shmid, NULL, 0);
        if (shmp == (void *) -1) { perror("shmat NULL"); return 1; }
    }
    printf("Attached at: %p (wanted: %p)\n", shmp, WANTED_ADDR);

    shmdt(shmp);
    shmctl(shmid, IPC_RMID, NULL);
    return 0;
}

Best practice: Always let the kernel choose the address (pass NULL to shmat). If you need to share pointers between processes, store offsets from the segment base, not absolute addresses.

Chapter 48 — Complete Tutorial Series Index

File 1: Introduction to System V Shared Memory

What is shared memory Why fastest IPC API overview

File 2: shmget() — Creating and Opening Segments

shmget() signature IPC keys ftok() IPC_CREAT IPC_EXCL

File 3: shmat() and shmdt() — Attach and Detach

shmat() flags SHM_RDONLY Pointer danger

File 4: shmctl() and shmid_ds — Control Operations

IPC_STAT IPC_SET IPC_RMID shmid_ds struct SHM_LOCK

File 5: Data Transfer — Writer and Reader Programs

Complete transfer example Semaphore ping-pong svshm_xfr programs

File 6: Virtual Memory Layout and /proc/PID/maps ← You are here

Memory layout TASK_UNMAPPED_BASE /proc/PID/maps /proc/PID/smaps

Chapter 48 Complete!

← Previous: Data Transfer Example ↑ Back to Introduction

File 6 of 6 | EmbeddedPathashala.com — Free Embedded Systems Education

Interview Questions — Virtual Memory and /proc

Q1. Where does the kernel place System V shared memory segments in a process’s virtual address space?

Starting at TASK_UNMAPPED_BASE (0x40000000 on x86-32), in the unmapped region between the heap and the stack. Shared memory segments, mmap() mappings, and shared libraries all share this region. The exact location depends on ASLR settings and what other mappings already exist.

Q2. How can you identify System V shared memory segments in /proc/PID/maps?

They appear as entries with the pathname /SYSV<key_hex> and permission field containing ‘s’ (shared). The inode field equals the shmid. If IPC_RMID was called, the pathname ends with “(deleted)” — but the mapping is still valid for currently attached processes.

Q3. What is the difference between /proc/PID/maps and /proc/PID/smaps?

/proc/PID/maps lists each mapping with its address range, permissions, and name — one line per mapping. /proc/PID/smaps (available since kernel 2.6.14) expands each mapping with detailed memory statistics: RSS, PSS, shared/private clean/dirty pages, and swap usage. smaps is much more useful for memory profiling and leak detection.

Q4. What does TASK_UNMAPPED_BASE mean and can it be changed?

It is a compile-time kernel constant defining the lowest virtual address at which the kernel will place mmap()/shmat() mappings when no address is specified. On x86-32 it is 0x40000000. It can be changed by rebuilding the kernel with a different value. It cannot be changed at runtime on a running kernel.

Q5. How do you figure out which shared memory segment corresponds to an entry in /proc/PID/maps?

The inode field in /proc/PID/maps equals the shmid. Cross-reference with ipcs -m output — find the shmid column that matches the inode number. The size of the entry (end address minus start address) should match the segment size in ipcs -m.

Q6. An embedded engineer says “I want to place shared memory at address 0x40000000 in all my processes so I can use absolute pointers.” What is the problem and what is the better approach?

The problem: on modern Linux with ASLR, libraries and other mappings may already occupy or conflict with 0x40000000; also, specifying an explicit address is non-portable and fragile. Better approach: let the kernel choose the address and store offsets relative to the segment’s base address (returned by shmat) rather than absolute pointers. Calculate absolute addresses at runtime by adding the offset to the base: abs_ptr = shm_base + offset.

Leave a Reply

Your email address will not be published. Required fields are marked *