System V Semaphores โ€“ Part 8 Semaphore Limits, /proc Tuning & IPC_INFO | TLPI Chapter

 

System V Semaphores โ€“ Part 8
Semaphore Limits, /proc Tuning & IPC_INFO | TLPI Chapter 47.10
๐Ÿ“˜ TLPI Chapter 47.10
โš™๏ธ Kernel Limits
๐Ÿ”ง /proc Tuning
๐ŸŽฏ Interview Q&A

Why Do Semaphore Limits Exist?

The Linux kernel manages System V semaphores as shared kernel resources. Without limits, a runaway or malicious process could create millions of semaphores and exhaust kernel memory, causing the entire system to crash or become unresponsive. These limits act as a safety fence.

Understanding these limits is critical for embedded and systems engineers because: applications in production hit these limits silently โ€” semget() just returns ENOSPC or EINVAL with no obvious explanation. Knowing what each limit means helps you diagnose and fix such failures quickly.

Keywords in This Chapter

SEMMNI SEMMSL SEMMNS SEMOPM SEMVMX SEMAEM SEMMNU SEMUME /proc/sys/kernel/sem IPC_INFO seminfo semctl()

1. Complete List of System V Semaphore Limits

The table below covers every limit โ€” what it controls, what error it generates when exceeded, and the default/ceiling values on Linux x86-32.

Limit Name What It Controls Affected Syscall Error Default (Linux 2.6) Ceiling (x86-32)
SEMMNI Max number of semaphore sets (identifiers) system-wide semget() ENOSPC 128 32768 (IPCMNI)
SEMMSL Max semaphores per semaphore set semget() EINVAL 250 65536
SEMMNS Max total semaphores across all sets, system-wide semget() ENOSPC 32000 (= 128 ร— 250) 2147483647 (INT_MAX)
SEMOPM Max operations per semop() call semop() E2BIG 32 ~1000
SEMVMX Maximum value a semaphore can hold semop() ERANGE 32767 (fixed) 32767 (cannot change)
SEMAEM Max value in a semadj (SEM_UNDO) total. Same as SEMVMX. semop() ERANGE 32767 (fixed) 32767 (cannot change)
SEMMNU Max undo structures system-wide (non-Linux only) semop() ENOSPC N/A on Linux N/A on Linux
SEMUME Max undo entries per undo structure (non-Linux only) semop() EINVAL N/A on Linux N/A on Linux

Relationship Between SEMMNI, SEMMSL, and SEMMNS

SEMMNI
Max semaphore SETS
128
ร—
SEMMSL
Max sems per SET
250
= default
SEMMNS
Max total sems
32000

The default SEMMNS (32000) is exactly SEMMNI (128) ร— SEMMSL (250). The kernel enforces both the per-set limit (SEMMSL) and the global total (SEMMNS) independently.

2. Reading and Modifying Limits via /proc

On Linux, four of the semaphore limits live in a single file: /proc/sys/kernel/sem. This file contains exactly four numbers separated by spaces in the order: SEMMSL, SEMMNS, SEMOPM, SEMMNI.

/* Reading the current semaphore limits from /proc */

$ cat /proc/sys/kernel/sem
250  32000  32  128
 ^     ^    ^    ^
 |     |    |    |
SEMMSL SEMMNS SEMOPM SEMMNI

Notice: SEMVMX and SEMAEM are NOT in this file โ€” they are hardcoded at 32767 and cannot be changed.

2.1 Temporary Change (lost on reboot)

# Increase SEMMNI to 512, SEMMNS to 128000, SEMOPM to 64, SEMMSL to 500
# Format: SEMMSL SEMMNS SEMOPM SEMMNI
$ echo "500 128000 64 512" > /proc/sys/kernel/sem

# Verify the change
$ cat /proc/sys/kernel/sem
500  128000  64  512

2.2 Permanent Change (survives reboot)

# Edit /etc/sysctl.conf and add:
kernel.sem = 500 128000 64 512

# Apply immediately without reboot:
$ sysctl -p

# Or apply specific key:
$ sysctl -w kernel.sem="500 128000 64 512"

2.3 Read limits programmatically in C

/* read_sem_limits.c โ€” Read semaphore limits from /proc */

#include <stdio.h>
#include <stdlib.h>

int main(void)
{
    FILE *fp;
    int semmsl, semmns, semopm, semmni;

    fp = fopen("/proc/sys/kernel/sem", "r");
    if (fp == NULL) {
        perror("fopen /proc/sys/kernel/sem");
        exit(EXIT_FAILURE);
    }

    /* File contains: SEMMSL SEMMNS SEMOPM SEMMNI */
    if (fscanf(fp, "%d %d %d %d",
               &semmsl, &semmns, &semopm, &semmni) != 4) {
        fprintf(stderr, "Failed to parse /proc/sys/kernel/sem\n");
        fclose(fp);
        exit(EXIT_FAILURE);
    }
    fclose(fp);

    printf("Semaphore Limits (from /proc/sys/kernel/sem):\n");
    printf("  SEMMSL (max sems per set)    : %d\n", semmsl);
    printf("  SEMMNS (total sems, system)  : %d\n", semmns);
    printf("  SEMOPM (ops per semop call)  : %d\n", semopm);
    printf("  SEMMNI (max semaphore sets)  : %d\n", semmni);
    printf("  SEMVMX (max sem value)       : 32767 [fixed]\n");
    printf("  SEMAEM (max semadj value)    : 32767 [fixed]\n");

    return 0;
}
Note on /proc format inconsistency: Message queues and shared memory each have separate files per limit (e.g., /proc/sys/kernel/msgmax). Semaphores are different โ€” all four limits share a single file. This is a historical accident in the Linux kernel and has been kept for backward compatibility.

3. Reading Limits Programmatically with IPC_INFO

Linux provides a special semctl() operation called IPC_INFO that returns all semaphore limits in a struct seminfo structure. This is the clean, portable way to read limits in C code without parsing /proc files.

/* ipc_info_demo.c โ€” Read all semaphore limits using IPC_INFO
   Compile: gcc ipc_info_demo.c -o ipc_info_demo
   Run:     ./ipc_info_demo
*/

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/sem.h>
#include "semun.h"   /* provides union semun and struct seminfo */

/* Note: struct seminfo is Linux-specific.
   If your semun.h does not include it, add this:

   struct seminfo {
       int semmap;   // not used on Linux
       int semmni;   // max semaphore identifiers (sets)
       int semmns;   // max total semaphores
       int semmnu;   // max undo structures (not used on Linux)
       int semmsl;   // max semaphores per set
       int semopm;   // max ops per semop()
       int semume;   // max undo entries per struct (not Linux)
       int semusz;   // size of struct sem_undo
       int semvmx;   // max semaphore value
       int semaem;   // max semadj value (= semvmx)
   };
*/

int main(void)
{
    union semun arg;
    struct seminfo buf;

    arg.__buf = &buf;   /* Point to our seminfo buffer */

    /*
     * semctl(0, 0, IPC_INFO, arg):
     *   First 0  = semid (ignored for IPC_INFO)
     *   Second 0 = semnum (ignored for IPC_INFO)
     *   IPC_INFO = operation code
     * Returns: index of highest used semaphore set entry on success
     */
    int maxidx = semctl(0, 0, IPC_INFO, arg);
    if (maxidx == -1) {
        perror("semctl IPC_INFO");
        exit(EXIT_FAILURE);
    }

    printf("=== System V Semaphore Limits (via IPC_INFO) ===\n");
    printf("  SEMMNI (max semaphore sets)     : %d\n", buf.semmni);
    printf("  SEMMSL (max sems per set)       : %d\n", buf.semmsl);
    printf("  SEMMNS (max total semaphores)   : %d\n", buf.semmns);
    printf("  SEMOPM (max ops per semop call) : %d\n", buf.semopm);
    printf("  SEMVMX (max semaphore value)    : %d\n", buf.semvmx);
    printf("  SEMAEM (max semadj value)       : %d\n", buf.semaem);
    printf("  Highest used set index          : %d\n", maxidx);

    return 0;
}
โš  Linux-specific: IPC_INFO and struct seminfo are Linux extensions, not part of POSIX. On other UNIX systems (Solaris, AIX, macOS), use the sysctl or sysconf interfaces instead.

4. Diagnosing Limit-Related Errors in Production

When an application hits a semaphore limit, the system call returns -1 with a specific errno. Here is how to diagnose each one:

/* limit_diagnosis.c โ€” Detect which limit was hit */

#include <stdio.h>
#include <errno.h>
#include <sys/sem.h>

int create_semaphore_set(key_t key, int nsems)
{
    int semId = semget(key, nsems, IPC_CREAT | IPC_EXCL | 0600);

    if (semId == -1) {
        switch (errno) {
        case ENOSPC:
            fprintf(stderr,
                "semget() ENOSPC: Hit SEMMNI (max semaphore sets) or\n"
                "                 SEMMNS (max total semaphores) limit.\n"
                "Check: cat /proc/sys/kernel/sem\n"
                "Fix:   echo 'kernel.sem = 500 128000 64 512' >> /etc/sysctl.conf\n"
                "       sysctl -p\n");
            break;

        case EINVAL:
            if (nsems <= 0) {
                fprintf(stderr,
                    "semget() EINVAL: nsems=%d is invalid (must be > 0)\n", nsems);
            } else {
                fprintf(stderr,
                    "semget() EINVAL: nsems=%d exceeds SEMMSL (%d max per set)\n",
                    nsems, /* read SEMMSL here */ 250);
            }
            break;

        case EEXIST:
            fprintf(stderr, "semget() EEXIST: Key already exists (use IPC_EXCL removal first)\n");
            break;

        default:
            perror("semget");
            break;
        }
        return -1;
    }
    return semId;
}

int perform_semop(int semId, struct sembuf *sops, size_t nsops)
{
    if (semop(semId, sops, nsops) == -1) {
        switch (errno) {
        case E2BIG:
            fprintf(stderr,
                "semop() E2BIG: nsops=%zu exceeds SEMOPM limit.\n"
                "Default SEMOPM = 32. Reduce batch size or raise limit.\n",
                nsops);
            return -1;

        case ERANGE:
            fprintf(stderr,
                "semop() ERANGE: Operation would push semaphore value above SEMVMX (32767)\n"
                "or semadj total above SEMAEM (32767).\n");
            return -1;

        default:
            perror("semop");
            return -1;
        }
    }
    return 0;
}

Shell Command to Show Current Usage vs Limits

# Check current semaphore usage
$ ipcs -s          # List all semaphore sets
$ ipcs -s -l       # Show limits alongside usage

# Count semaphore sets currently allocated
$ ipcs -s | grep -c "^0x"

# Show limits
$ cat /proc/sys/kernel/sem

5. Special Notes and Gotchas About Limits
SEMMSL and Large Sets

While you CAN raise SEMMSL above 65536 and create large semaphore sets, semop() can only operate on the first 65536 semaphores in the set. In practice, the TLPI book recommends keeping semaphore sets under 8000 semaphores due to internal kernel implementation limitations.

SEMMNS Practical Ceiling

The theoretical ceiling for SEMMNS is INT_MAX (~2 billion), but the real practical limit is your system’s available RAM. Each semaphore consumes kernel memory. Setting SEMMNS to INT_MAX is meaningless โ€” you will run out of RAM long before you create 2 billion semaphores.

SEMOPM Practical Maximum

The kernel documentation says SEMOPM can be raised to around 1000. However, in practice it is very rarely useful to perform more than a few operations in a single semop() call. More than 5-10 operations per call usually indicates a design problem. The default of 32 is adequate for virtually all real-world applications.

SEMVMX and SEMAEM Are Fixed at 32767

These two limits are hardcoded in the kernel โ€” they cannot be changed at runtime. If your application needs semaphore values higher than 32767, you must redesign the solution (for example, use counting semaphores differently, or switch to POSIX semaphores which have larger value ranges).

/* Demonstrating SEMVMX limit */

#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <sys/sem.h>
#include "semun.h"

#define SEMVMX  32767   /* Hardcoded maximum semaphore value */

int main(void)
{
    int semId;
    union semun arg;
    struct sembuf sops;

    semId = semget(IPC_PRIVATE, 1, IPC_CREAT | 0600);
    if (semId == -1) { perror("semget"); exit(1); }

    /* Set semaphore to SEMVMX (maximum allowed) */
    arg.val = SEMVMX;
    if (semctl(semId, 0, SETVAL, arg) == -1) { perror("setval"); exit(1); }

    printf("Semaphore value set to %d (SEMVMX) โ€” OK\n", SEMVMX);

    /* Try to increment by 1 โ€” this will fail with ERANGE */
    sops.sem_num = 0;
    sops.sem_op  = 1;    /* +1 would make it 32768 > SEMVMX */
    sops.sem_flg = 0;

    if (semop(semId, &sops, 1) == -1) {
        if (errno == ERANGE) {
            printf("semop(+1) failed: ERANGE โ€” cannot exceed SEMVMX=%d\n", SEMVMX);
        } else {
            perror("semop");
        }
    }

    semctl(semId, 0, IPC_RMID);
    return 0;
}

6. Complete Example: Check and Adjust Limits Before Creating Semaphores

This is a real-world pattern โ€” check limits before creating semaphores, and emit a helpful error if insufficient:

/* check_limits_before_create.c
   Best practice: check limits before semget() to give useful error messages.
   Compile: gcc check_limits_before_create.c -o check_limits
*/

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/sem.h>
#include "semun.h"

/* Read semaphore limits via IPC_INFO */
static int get_sem_limits(struct seminfo *buf)
{
    union semun arg;
    arg.__buf = buf;
    return semctl(0, 0, IPC_INFO, arg);
}

/* Count currently allocated semaphore sets using SEM_INFO */
static int get_sem_usage(struct seminfo *buf)
{
    union semun arg;
    arg.__buf = buf;
    return semctl(0, 0, SEM_INFO, arg);
    /* Returns index of highest used entry (not the count directly) */
}

int safe_semget(key_t key, int nsems, int flags)
{
    struct seminfo limits, usage;
    int maxidx, semId;

    /* Step 1: Read limits */
    if (get_sem_limits(&limits) == -1) {
        perror("IPC_INFO");
        return -1;
    }

    /* Step 2: Check nsems against SEMMSL */
    if (nsems > limits.semmsl) {
        fprintf(stderr,
            "ERROR: Requested %d semaphores per set, but SEMMSL limit is %d.\n"
            "  Fix: echo 'kernel.sem = %d %d %d %d' | sudo tee /proc/sys/kernel/sem\n",
            nsems, limits.semmsl,
            nsems + 10,          /* new SEMMSL suggestion */
            limits.semmns,
            limits.semopm,
            limits.semmni);
        return -1;
    }

    /* Step 3: Get current usage */
    maxidx = get_sem_usage(&usage);
    if (maxidx == -1) {
        perror("SEM_INFO");
        return -1;
    }

    printf("Semaphore system status:\n");
    printf("  Sets allocated / SEMMNI limit  : %d / %d\n",
           usage.semusz,   /* current number of sets */
           limits.semmni);
    printf("  Sems allocated / SEMMNS limit  : %d / %d\n",
           usage.semaem,   /* overloaded: current total sems in SEM_INFO */
           limits.semmns);
    printf("  Sems per set allowed (SEMMSL)  : %d\n", limits.semmsl);
    printf("  Max ops per call (SEMOPM)      : %d\n", limits.semopm);
    printf("  Max semaphore value (SEMVMX)   : %d\n", limits.semvmx);

    /* Step 4: Create semaphore set */
    semId = semget(key, nsems, flags);
    if (semId == -1) {
        perror("semget");
        return -1;
    }

    printf("Semaphore set created successfully. ID = %d\n", semId);
    return semId;
}

int main(void)
{
    int semId = safe_semget(IPC_PRIVATE, 5, IPC_CREAT | 0600);
    if (semId == -1) {
        fprintf(stderr, "Failed to create semaphore set\n");
        exit(EXIT_FAILURE);
    }

    /* Use the semaphores... */
    printf("Using semaphore set %d...\n", semId);

    /* Cleanup */
    semctl(semId, 0, IPC_RMID);
    return 0;
}

7. Interview Questions and Answers
Q1. What error does semget() return when SEMMNI is exceeded?

semget() returns -1 with errno = ENOSPC. The same error is returned when the total number of individual semaphores in all sets would exceed SEMMNS. To distinguish the two cases, check the current usage with SEM_INFO before calling semget().

Q2. What is the format of /proc/sys/kernel/sem and what does each value represent?

The file contains four space-separated integers in this order: SEMMSL SEMMNS SEMOPM SEMMNI. A sample output of 250 32000 32 128 means: max 250 semaphores per set, 32000 total semaphores across the system, max 32 operations per semop() call, and max 128 semaphore sets. SEMVMX and SEMAEM are not in this file because they are fixed at 32767.

Q3. Which semaphore limits cannot be changed on Linux and why?

SEMVMX (max semaphore value) and SEMAEM (max semadj value) are fixed at 32767 on Linux. They are defined as compile-time constants in the kernel source. Changing them would require recompiling the kernel. All other limits (SEMMNI, SEMMSL, SEMMNS, SEMOPM) can be changed at runtime via /proc/sys/kernel/sem.

Q4. What is IPC_INFO and how does it differ from SEM_INFO?

Both are Linux-specific semctl() operations that return a struct seminfo. The difference is: IPC_INFO returns the configured limits (what the system allows). SEM_INFO returns the current usage (how many are actually in use). The return value is also different: IPC_INFO returns the index of the highest used semaphore table entry, while SEM_INFO returns the same but with the seminfo fields overloaded to show current counts rather than limits.

Q5. What error does semop() return if you try to perform too many operations at once?

semop() returns -1 with errno = E2BIG if the nsops argument (number of operations in the sembuf array) exceeds SEMOPM. The default Linux SEMOPM is 32. This limit exists because semop() performs all operations atomically, and the kernel must allocate temporary memory proportional to the number of operations.

Q6. Why does Linux use a single /proc file for all semaphore limits, unlike message queues?

This is a historical accident. When the semaphore limits interface was added to the Linux kernel, all four values were crammed into one file instead of separate files like /proc/sys/kernel/msgmax for message queues. By the time this inconsistency was noticed, many scripts and tools depended on the existing format, so it was kept as-is for backward compatibility. The TLPI book explicitly notes this as a historical accident that is difficult to rectify.

Q7. Write a shell one-liner to find out how many semaphore sets are currently in use.

# Count semaphore sets (each line starting with 0x is one set)
$ ipcs -s | grep -c "^0x"

# OR: use ipcs -s -u for summary
$ ipcs -s -u

# Output example:
# ------ Semaphore Status --------
# used arrays = 3
# allocated semaphores = 9
Q8. An application fails with “semget: No space left on device”. What do you check?

ENOSPC from semget means either SEMMNI (max semaphore sets) or SEMMNS (max total semaphores) was exceeded. Steps to diagnose: (1) Run ipcs -s -u to see current usage. (2) Run cat /proc/sys/kernel/sem to see limits. (3) Run ipcs -s to see if stale semaphore sets were left by crashed processes โ€” remove them with ipcrm -s <semid>. (4) If legitimate demand exceeds limits, increase SEMMNI and/or SEMMNS via sysctl -w kernel.sem="...".

8. Quick Reference Card: Linux Default vs Ceiling Values
Limit Default Ceiling (x86-32) Changeable? In /proc?
SEMMNI 128 32768 โœ“ Yes โœ“ Yes (4th field)
SEMMSL 250 65536 โœ“ Yes โœ“ Yes (1st field)
SEMMNS 32000 INT_MAX โœ“ Yes โœ“ Yes (2nd field)
SEMOPM 32 ~1000 โœ“ Yes โœ“ Yes (3rd field)
SEMVMX 32767 32767 โœ— Fixed โœ— No
SEMAEM 32767 32767 โœ— Fixed โœ— No

Chapter 47 Complete!
You have mastered System V Semaphores โ€” from basics to binary semaphores to limits

โ† Previous: Binary Semaphores Back to Home

Leave a Reply

Your email address will not be published. Required fields are marked *