System V Shared Memory Summary, Best Practices & Full Examples

 

System V Shared Memory
Chapter 48 · Part 3 of 3 — Summary, Best Practices & Full Examples
Topic
Best Practices
Level
Intermediate
Series
TLPI Ch 48

The Big Picture

Shared memory is the fastest form of IPC on Linux because once a segment is mapped into two processes’ address spaces, data transfer requires zero kernel involvement — one process writes, the other reads, and no copy happens.

That speed comes with two responsibilities:

1. Synchronization — you need a semaphore (or other mechanism) to prevent simultaneous writes and torn reads.
2. Relative addressing — the same segment may be mapped at different virtual addresses in different processes, so you must use offsets, not absolute pointers, inside the segment.

Key Terms

shmat() shmdt() shmget() shmctl() IPC_RMID Relative Offsets Semaphore Sync Producer-Consumer SHM_LOCK SHM_UNLOCK

1. How Shared Memory Works — Zero-Copy IPC

When two processes attach the same segment, the kernel maps the same physical pages into both virtual address spaces. A write by process A instantly appears to process B — no write(), no read(), no pipe, no socket.

Process A
Virtual Memory

code
stack
shm @ 0x7f000
Physical RAM
Kernel Pages

Shared Pages
single copy
no kernel call
needed to share
Process B
Virtual Memory

code
stack
shm @ 0x6b000
↑ Same physical pages, different virtual addresses in each process

Notice the virtual addresses differ (0x7f000 vs 0x6b000). This is why storing absolute pointers inside the shared segment is wrong.

2. Relative Addressing — The Most Important Rule

If process A stores a pointer that points to something inside the shared segment, that pointer value (a virtual address from A’s perspective) is meaningless in process B because the segment is mapped at a different base address.

The solution: store offsets from the start of the segment instead of absolute pointers. Convert to a real pointer only when you need to dereference.

❌ Wrong — absolute pointer ✔ Correct — relative offset
char *p = (char *)shm_addr + 100;
header->next = p; /* stores virtual address */

Process B sees a wrong address — crash!

header->next_offset = 100; /* bytes from base */
/* to use in Process B: */
char *p = (char *)shm_addr + header->next_offset;
Works regardless of mapped address.
/* Demonstrating relative offsets in shared memory */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/ipc.h>
#include <sys/shm.h>

/* Layout of our shared segment */
typedef struct {
    int   count;            /* number of items stored */
    off_t first_offset;     /* offset (from seg base) to first item */
} ShmHeader;

typedef struct {
    off_t next_offset;  /* offset to next node, or -1 if none */
    int   value;
} ShmNode;

int main(void)
{
    key_t key = ftok("/tmp", 'Z');
    int shmid = shmget(key, 4096, IPC_CREAT | 0660);
    if (shmid == -1) { perror("shmget"); exit(1); }

    char *base = shmat(shmid, NULL, 0);
    if (base == (void *)-1) { perror("shmat"); exit(1); }

    /* Writer: build a linked list using offsets */
    ShmHeader *hdr = (ShmHeader *)base;
    hdr->count = 3;

    /* Place three nodes sequentially after the header */
    size_t node_start = sizeof(ShmHeader);
    for (int i = 0; i < 3; i++) {
        ShmNode *n = (ShmNode *)(base + node_start + i * sizeof(ShmNode));
        n->value = (i + 1) * 10;
        /* next_offset points to next node, or -1 for last */
        n->next_offset = (i < 2)
            ? (off_t)(node_start + (i + 1) * sizeof(ShmNode))
            : -1;
    }
    hdr->first_offset = (off_t)node_start;

    /* Reader: traverse using offsets (simulates different process) */
    printf("Reading linked list from shared memory:\n");
    off_t cur = hdr->first_offset;
    while (cur != -1) {
        ShmNode *n = (ShmNode *)(base + cur);
        printf("  value = %d\n", n->value);
        cur = n->next_offset;
    }

    shmdt(base);
    shmctl(shmid, IPC_RMID, NULL);
    return 0;
}

3. Synchronization — Why You Cannot Skip It

Shared memory has no built-in synchronization. If two processes write at the same time, the result is undefined (data corruption). If one process reads while another writes, it may see a partial update.

Common synchronization mechanisms used with shared memory:

Mechanism Header Best For Notes
System V Semaphore <sys/sem.h> Multi-process, same machine Classic pairing with SysV SHM
POSIX Semaphore (named) <semaphore.h> Multi-process Simpler API; link with -lpthread
pthread mutex (process-shared) <pthread.h> Multi-process Must set PTHREAD_PROCESS_SHARED; store mutex inside the segment
Atomic operations <stdatomic.h> Simple counters/flags No blocking; lock-free
/* Process-shared pthread mutex stored inside the shared segment */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <pthread.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#include <sys/wait.h>

#define BUF_SZ 256

typedef struct {
    pthread_mutex_t lock;   /* stored inside the segment */
    int             counter;
    char            message[BUF_SZ];
} SharedData;

int main(void)
{
    key_t key = ftok("/tmp", 'M');
    int shmid = shmget(key, sizeof(SharedData), IPC_CREAT | 0660);
    if (shmid == -1) { perror("shmget"); exit(1); }

    SharedData *sd = shmat(shmid, NULL, 0);
    if (sd == (void *)-1) { perror("shmat"); exit(1); }

    /* Initialize process-shared mutex (done once by creator) */
    pthread_mutexattr_t attr;
    pthread_mutexattr_init(&attr);
    pthread_mutexattr_setpshared(&attr, PTHREAD_PROCESS_SHARED);
    pthread_mutex_init(&sd->lock, &attr);
    pthread_mutexattr_destroy(&attr);

    sd->counter = 0;

    pid_t child = fork();
    if (child == 0) {
        /* child process: increment counter 100 times */
        SharedData *csd = shmat(shmid, NULL, 0);
        for (int i = 0; i < 100; i++) {
            pthread_mutex_lock(&csd->lock);
            csd->counter++;
            pthread_mutex_unlock(&csd->lock);
        }
        shmdt(csd);
        exit(0);
    }

    /* parent: also increment 100 times */
    for (int i = 0; i < 100; i++) {
        pthread_mutex_lock(&sd->lock);
        sd->counter++;
        pthread_mutex_unlock(&sd->lock);
    }

    wait(NULL);
    printf("Final counter = %d (expected 200)\n", sd->counter);

    pthread_mutex_destroy(&sd->lock);
    shmdt(sd);
    shmctl(shmid, IPC_RMID, NULL);
    return 0;
}
/* Compile: gcc -o demo demo.c -lpthread */

4. Locking Segments in RAM — SHM_LOCK / SHM_UNLOCK

In real-time or low-latency embedded systems, you may not want shared memory pages to be swapped out to disk — a page fault at the wrong moment adds unpredictable latency. shmctl(shmid, SHM_LOCK, NULL) pins the segment in physical RAM. SHM_UNLOCK releases the pin. Requires CAP_IPC_LOCK capability (root, or set via ulimit -l).

When locked, the SHM_LOCKED bit is set in shm_perm.mode.

/* Lock a shared memory segment into physical RAM */
#include <stdio.h>
#include <stdlib.h>
#include <sys/ipc.h>
#include <sys/shm.h>

int main(void)
{
    key_t key  = ftok("/tmp", 'L');
    int shmid  = shmget(key, 4096, IPC_CREAT | 0660);
    if (shmid == -1) { perror("shmget"); exit(1); }

    if (shmctl(shmid, SHM_LOCK, NULL) == -1) {
        perror("SHM_LOCK (need CAP_IPC_LOCK / root)");
    } else {
        printf("Segment locked into RAM\n");

        struct shmid_ds info;
        shmctl(shmid, IPC_STAT, &info);
        printf("SHM_LOCKED flag set: %s\n",
               (info.shm_perm.mode & SHM_LOCKED) ? "YES" : "NO");

        shmctl(shmid, SHM_UNLOCK, NULL);
        printf("Segment unlocked\n");
    }

    shmctl(shmid, IPC_RMID, NULL);
    return 0;
}

5. Full Producer-Consumer with System V Semaphore

This is the classic pattern recommended by TLPI. A writer fills a buffer in shared memory and signals a semaphore; the reader waits on the semaphore, reads, and signals back. A cnt == 0 message signals end-of-data.

/* Shared header — put in shm_common.h */
#ifndef SHM_COMMON_H
#define SHM_COMMON_H

#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#include <sys/sem.h>

#define SHM_KEY  0x1234
#define SEM_KEY  0x5678
#define BUF_SIZE 1024

/* Two semaphores: index 0 = WRITE_SEM, index 1 = READ_SEM */
#define WRITE_SEM 0
#define READ_SEM  1

typedef struct {
    int  cnt;               /* bytes written into buf, 0 = done */
    char buf[BUF_SIZE];
} SharedBuf;

/* Helper: P operation (decrement/wait) */
static inline void sem_wait_op(int semid, int semnum) {
    struct sembuf sb = { semnum, -1, 0 };
    semop(semid, &sb, 1);
}

/* Helper: V operation (increment/signal) */
static inline void sem_signal_op(int semid, int semnum) {
    struct sembuf sb = { semnum, +1, 0 };
    semop(semid, &sb, 1);
}

#endif
/* writer.c — Producer */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include "shm_common.h"

int main(void)
{
    /* Create shared memory */
    int shmid = shmget(SHM_KEY, sizeof(SharedBuf), IPC_CREAT | 0660);
    if (shmid == -1) { perror("shmget"); exit(1); }
    SharedBuf *shmp = shmat(shmid, NULL, 0);

    /* Create semaphore set: sem[WRITE]=1 (writer goes first), sem[READ]=0 */
    int semid = semget(SEM_KEY, 2, IPC_CREAT | 0660);
    semctl(semid, WRITE_SEM, SETVAL, 1);
    semctl(semid, READ_SEM,  SETVAL, 0);

    /* Read from stdin and send chunks */
    while (1) {
        sem_wait_op(semid, WRITE_SEM);   /* wait for write turn */

        shmp->cnt = read(STDIN_FILENO, shmp->buf, BUF_SIZE);
        if (shmp->cnt == -1) { perror("read"); exit(1); }

        sem_signal_op(semid, READ_SEM);  /* tell reader data is ready */

        if (shmp->cnt == 0) break;       /* EOF */
    }

    /* Wait for reader to finish, then cleanup */
    sem_wait_op(semid, WRITE_SEM);
    shmdt(shmp);
    shmctl(shmid, IPC_RMID, NULL);
    semctl(semid, 0, IPC_RMID);
    printf("Writer: done\n");
    return 0;
}
/* reader.c — Consumer */
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include "shm_common.h"

int main(void)
{
    int shmid = shmget(SHM_KEY, sizeof(SharedBuf), 0);
    if (shmid == -1) { perror("shmget"); exit(1); }
    SharedBuf *shmp = shmat(shmid, NULL, 0);

    int semid = semget(SEM_KEY, 2, 0);
    if (semid == -1) { perror("semget"); exit(1); }

    long xfrs = 0, bytes = 0;

    while (1) {
        sem_wait_op(semid, READ_SEM);    /* wait for data */

        if (shmp->cnt == 0) break;       /* writer signalled EOF */

        if (write(STDOUT_FILENO, shmp->buf, shmp->cnt) != shmp->cnt) {
            perror("write"); exit(1);
        }
        xfrs++;
        bytes += shmp->cnt;

        sem_signal_op(semid, WRITE_SEM); /* give writer next turn */
    }

    sem_signal_op(semid, WRITE_SEM);     /* let writer clean up */
    shmdt(shmp);

    fprintf(stderr, "Reader: %ld transfers, %ld bytes\n", xfrs, bytes);
    return 0;
}
# Build and run
gcc -o writer writer.c
gcc -o reader reader.c

# In one terminal:
./writer < /etc/passwd

# In another terminal:
./reader > /tmp/out.txt
diff /etc/passwd /tmp/out.txt   # should be identical

6. Complete Shared Memory Lifecycle

Every shared memory program follows the same six steps. Missing any step leads to resource leaks or crashes.

1. shmget()
Create or open segment
2. shmat()
Attach to address space (kernel chooses address)
3. Read / Write data
Use semaphores for synchronization; use offsets not pointers
4. shmdt()
Detach — shm_nattch decremented
5. shmctl(IPC_RMID)
Mark for deletion (or actually delete if nattch=0)
Kernel removes pages when last process detaches

7. Let the Kernel Choose the Attach Address

When calling shmat(shmid, addr, flags):

If you pass NULL as addr, the kernel picks a suitable virtual address. This is the recommended approach because the kernel knows which addresses are free in the process’s virtual map.

Passing a specific address is fragile — the address may be occupied by libraries or stack in some processes and not others.

/* Recommended: NULL address — let kernel choose */
void *addr = shmat(shmid, NULL, 0);
if (addr == (void *)-1) { perror("shmat"); exit(1); }

/*
 * Do NOT do this for portable code:
 * void *addr = shmat(shmid, (void *)0x50000000, SHM_RND);
 * That address may conflict with ASLR-placed libraries.
 */

/* Read-only attach (pass SHM_RDONLY flag) */
void *ro_addr = shmat(shmid, NULL, SHM_RDONLY);

8. Common Pitfalls and How to Avoid Them
Pitfall Consequence Fix
Storing absolute pointers inside segment Crash or data corruption in other process Use offsets from segment base
No synchronization Race conditions, torn reads/writes Use semaphore or mutex
Not calling IPC_RMID Segment persists after all processes exit — wastes RAM Always call IPC_RMID in cleanup path; use atexit()
Forgetting to shmdt() shm_nattch never reaches 0; IPC_RMID delayed Call shmdt() before process exits
Exceeding SHMMAX shmget() returns EINVAL Check /proc/sys/kernel/shmmax; increase if needed
Orphaned segments from crashed process SHMMNI or SHMALL exhausted over time Use ipcs -m; write cleanup utilities

9. Quick API Reference
#include <sys/ipc.h>
#include <sys/shm.h>

/* 1. Create or open a segment */
int shmget(key_t key, size_t size, int shmflg);
/*   Returns shmid on success, -1 on error          */
/*   IPC_CREAT | 0660  — create with rw-rw---- perms */
/*   IPC_CREAT | IPC_EXCL  — fail if already exists  */

/* 2. Attach to address space */
void *shmat(int shmid, const void *shmaddr, int shmflg);
/*   shmaddr = NULL → kernel chooses address         */
/*   shmflg = SHM_RDONLY → read-only attach          */
/*   Returns attached address, or (void *)-1 on error */

/* 3. Detach */
int shmdt(const void *shmaddr);
/*   shmaddr = address returned by shmat()           */

/* 4. Control operations */
int shmctl(int shmid, int cmd, struct shmid_ds *buf);
/*   IPC_STAT   — read shmid_ds                      */
/*   IPC_SET    — write uid/gid/mode                 */
/*   IPC_RMID   — mark segment for deletion          */
/*   SHM_LOCK   — lock pages in RAM (root)           */
/*   SHM_UNLOCK — unlock pages                       */
/*   IPC_INFO   — read system limits (shminfo)       */
/*   SHM_INFO   — read current usage (shm_info)      */
/*   SHM_STAT   — IPC_STAT but by table index        */

Interview Questions & Answers

Q1. Why is shared memory the fastest IPC mechanism?
Once both processes map the segment, data exchange requires no system call. The writer updates memory directly; the reader reads it directly. There is no copy, no kernel buffer, and no context switch for the data transfer itself.

Q2. Why should you not store absolute pointers inside a shared memory segment?
The same physical pages are mapped at different virtual addresses in different processes. An absolute pointer from process A is a virtual address valid only in A’s address space. When process B reads that pointer and dereferences it, it reads from a completely different (and likely invalid) location. Use offsets from the base of the segment instead.

Q3. What is the recommended value for the shmaddr argument of shmat()?
NULL. This tells the kernel to choose a suitable virtual address automatically. Hard-coding an address is fragile because ASLR or other mappings may occupy that address in some processes.

Q4. What happens when shmctl(IPC_RMID) is called while processes still have the segment attached?
The segment is not immediately destroyed. The kernel sets the SHM_DEST flag in shm_perm.mode. The segment is physically removed only after the last process calls shmdt() (i.e., when shm_nattch drops to 0).

Q5. What synchronization mechanisms can be used with shared memory?
System V semaphores (classic pairing), POSIX named semaphores, process-shared pthread mutexes stored inside the segment, or lock-free atomics for simple flags/counters. Shared memory itself has no built-in locking.

Q6. What is the difference between shmdt() and shmctl(IPC_RMID)?
shmdt() only unmaps the segment from the calling process’s address space — the segment still exists in the kernel. IPC_RMID schedules the segment for deletion from the kernel’s IPC table.

Q7. How many times can a process attach the same segment?
Multiple times — each shmat() call returns a different virtual address mapping the same physical pages. Each mapping must be separately detached with shmdt(), and each successful shmat() increments shm_nattch.

Q8. What is the SHM_LOCK operation used for in embedded/real-time systems?
It pins the shared memory pages in physical RAM, preventing the kernel from swapping them to disk. This guarantees deterministic access latency — critical in hard real-time systems where a page fault would cause an unacceptable delay.

Q9. How does the System V shared memory lifecycle differ from POSIX shared memory?
System V: created by shmget() with an integer key; attached with shmat(); persists until explicitly deleted with IPC_RMID or system reboot. POSIX: created with shm_open() using a filename-like string; mapped with mmap(); unlinked with shm_unlink(). POSIX has a cleaner API but both persist beyond process exit.

Q10. Write the sequence of calls for a minimal shared memory writer-reader.
Writer: shmget(key, size, IPC_CREAT|0660)shmat(id, NULL, 0) → write data → signal semaphore → shmdt()shmctl(IPC_RMID). Reader: shmget(key, size, 0)shmat(id, NULL, 0) → wait semaphore → read data → shmdt().

Q11. Can shared memory be used between unrelated processes (not parent-child)?
Yes. Any two processes that know the same key_t (e.g. generated with ftok() on a common pathname and id, or a hard-coded key) can call shmget() to obtain the same shmid and then attach.

Chapter 48 Complete!

You have covered shmid_ds, all SHM limits, /proc, IPC_INFO, SHM_INFO, and best practices.

← Part 1: shmid_ds ← Part 2: Limits EmbeddedPathashala

Leave a Reply

Your email address will not be published. Required fields are marked *