System V Message Queues Disadvantages & Alternatives

 

System V Message Queues
Part 2 — Disadvantages & Alternatives | Chapter 46 · TLPI
46.9
Section
4
Disadvantages
3
Alternatives

Why Are Disadvantages Important to Know?

System V message queues were created in the 1980s as part of the original UNIX IPC suite. By today’s standards, their design has several rough edges compared to more modern IPC mechanisms. Understanding these limitations helps you make the right choice of IPC tool for new projects — and is a common topic in embedded Linux and systems programming interviews.

The book (Section 46.9) lists four main disadvantages. We’ll explore each one with a clear explanation, a practical example of the problem it causes, and how modern alternatives solve it.

Key Terms in This Section
file descriptor select() / poll() epoll ftok() IPC_PRIVATE connectionless reference counting POSIX MQ FIFO UNIX domain socket ipcs ipcrm

Disadvantage 1 — Not File Descriptors
Problem: Can’t use select(), poll(), or epoll()

System V message queues are identified by a numeric queue ID, not a file descriptor. Almost all other UNIX I/O mechanisms (pipes, FIFOs, sockets, files) use file descriptors. This matters because the kernel’s I/O multiplexing calls — select(), poll(), and epoll() — only work on file descriptors.

If your program needs to simultaneously monitor a message queue AND a socket (e.g., waiting for either a network packet or a local IPC message), you can’t do it with a simple select() call. You would need two separate threads (one blocking on each), which adds complexity.

FD-based IPC (works with poll)
fd[0] = pipe read end
fd[1] = socket
fd[2] = FIFO
→ poll(fds, 3, -1) ✓
SysV MQ (can’t use poll)
mqid = 12345 (not an fd)
mqid = 12346
→ poll() doesn’t accept ✗
/* Example: Trying to monitor both a pipe and a SysV MQ simultaneously */

/* This works fine for pipes/sockets/FIFOs: */
struct pollfd fds[2];
fds[0].fd = pipe_fd;      /* pipe read end - OK */
fds[0].events = POLLIN;

/* You CANNOT do this for a SysV message queue: */
/* fds[1].fd = mqid;      <-- mqid is NOT an fd, this is WRONG */

/* The workaround: use a separate thread that blocks on msgrcv() */
/* and signals the main thread via a pipe when a message arrives */

/* Thread approach (complex): */
void *mq_watcher(void *arg) {
    int   mqid   = *(int *)arg;
    int   pipefd = ((int *)arg)[1];   /* write end of notification pipe */
    char  buf[512];
    char  notify = 1;
    /* Block waiting for any message */
    while (msgrcv(mqid, buf, sizeof(buf), 0, 0) != -1) {
        /* Wake up main thread via pipe */
        write(pipefd, &notify, 1);
    }
    return NULL;
}
/* Now main thread can poll() on pipe_fd instead */
POSIX message queues solve this! mq_open() returns a descriptor that can be used with select()/poll() on Linux, making it far more convenient in event-driven programs.

Disadvantage 2 — Keys Instead of Filenames
Problem: Complex identification using keys and ftok()

To use a named SysV message queue, two processes must agree on the same numeric key. The standard way to generate this is ftok(), which derives a key from a filename and a project ID number.

The problem: ftok() uses the inode number and device number of the file. If the file is deleted and recreated (even with the same name), it gets a new inode number — and ftok() returns a different key. This causes subtle bugs where two processes think they are using the same queue but are not.

#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/msg.h>
#include <stdio.h>

int main(void)
{
    key_t key;
    int   mqid;

    /* Generate a key from a file path and a project number (1-255) */
    key = ftok("/tmp/myapp_ipc", 'A');
    if (key == -1) {
        perror("ftok");
        return 1;
    }

    printf("Generated key: 0x%x\n", (unsigned)key);

    /* Problem: if /tmp/myapp_ipc is deleted and recreated,
       ftok() returns a DIFFERENT key even for the same path! */

    /* Using IPC_PRIVATE avoids this but requires sharing the ID
       out-of-band (e.g., via a file, environment variable, or
       inheritance through fork): */
    mqid = msgget(IPC_PRIVATE, 0600);
    if (mqid == -1) {
        perror("msgget");
        return 1;
    }
    printf("Private queue ID: %d\n", mqid);
    /* Now you must somehow tell other processes this mqid...
       e.g., write it to a well-known file:             */
    FILE *fp = fopen("/tmp/myapp.mqid", "w");
    fprintf(fp, "%d\n", mqid);
    fclose(fp);

    /* To list all SysV IPC objects: run 'ipcs -q' in shell */
    /* To delete: run 'ipcrm -q <mqid>' or call msgctl()   */
    return 0;
}
Named pipes (FIFOs) and UNIX domain sockets use regular filesystem paths (e.g. /tmp/myfifo). This means you can use ls, rm, and standard file permissions — much simpler than ipcs/ipcrm.

Disadvantage 3 — Connectionless (No Reference Counting)
Problem: Hard to know when it’s safe to delete a queue

Pipes, FIFOs, and sockets are reference-counted by the kernel. When the last process closes its end of a pipe, the kernel automatically destroys it. Message queues have no such mechanism. The kernel does not track how many processes are currently using a queue.

This creates two hard problems for the programmer:

  • When to delete: If you delete the queue too early, other processes that haven’t read their messages yet lose data immediately.
  • Leaked queues: If your server crashes before calling msgctl(IPC_RMID), the queue stays in the kernel permanently (until reboot or manual cleanup with ipcrm).

Pipe (reference counted)
Process A opens pipe → refcount=1
Process B opens pipe → refcount=2
Process A exits → refcount=1
Process B exits → refcount=0 → kernel auto-destroys ✓
SysV MQ (NOT reference counted)
Server creates queue
Clients send/receive messages
Server crashes → queue stays!
Nobody cleans up → memory leak until reboot ✗
#include <sys/msg.h>
#include <signal.h>
#include <stdlib.h>

static int g_mqid = -1;

/* Signal handler to clean up queue on crash/termination */
void cleanup_handler(int sig)
{
    if (g_mqid != -1) {
        msgctl(g_mqid, IPC_RMID, NULL);
        g_mqid = -1;
    }
    /* Re-raise to get default behavior (core dump, etc.) */
    signal(sig, SIG_DFL);
    raise(sig);
}

int main(void)
{
    /* Register cleanup on common termination signals */
    signal(SIGTERM, cleanup_handler);
    signal(SIGINT,  cleanup_handler);
    signal(SIGHUP,  cleanup_handler);
    /* Note: SIGKILL and SIGSTOP cannot be caught — queue may leak */

    g_mqid = msgget(IPC_PRIVATE, 0600);
    if (g_mqid == -1) { perror("msgget"); return 1; }

    /* ... use the queue ... */

    /* Normal exit: clean up manually */
    msgctl(g_mqid, IPC_RMID, NULL);
    return 0;
}

/* To see leaked queues from the shell:
   $ ipcs -q
   To clean up manually:
   $ ipcrm -q <msqid>  */

Disadvantage 4 — Fixed Kernel Limits
Problem: Hard limits on number of queues, message size, and queue capacity

The kernel enforces limits on System V message queues. If your application needs more than the default limits, you must reconfigure the system — which is extra work when deploying to a new machine, and requires root privileges.

Limit Controlled by Typical default
Max message queues system-wide /proc/sys/kernel/msgmni 32000
Max bytes per message /proc/sys/kernel/msgmax 8192 bytes
Max bytes in a single queue /proc/sys/kernel/msgmnb 16384 bytes
Max message queues per msgget() call MSGMNI compile-time constant varies
/* Check current system limits from a program */
#include <stdio.h>

int main(void)
{
    FILE *fp;
    long  val;

    /* Maximum number of queues */
    fp = fopen("/proc/sys/kernel/msgmni", "r");
    fscanf(fp, "%ld", &val);
    fclose(fp);
    printf("Max message queues: %ld\n", val);

    /* Maximum single message size */
    fp = fopen("/proc/sys/kernel/msgmax", "r");
    fscanf(fp, "%ld", &val);
    fclose(fp);
    printf("Max message size  : %ld bytes\n", val);

    /* Maximum total bytes in one queue */
    fp = fopen("/proc/sys/kernel/msgmnb", "r");
    fscanf(fp, "%ld", &val);
    fclose(fp);
    printf("Max queue capacity: %ld bytes\n", val);

    return 0;
}

/* To raise the limit (as root):
   echo 65536 > /proc/sys/kernel/msgmnb
   Or permanently in /etc/sysctl.conf:
   kernel.msgmnb = 65536               */
Why does this matter in embedded systems? In embedded Linux (e.g., devices running on ARM SoCs), the default limits may be much lower to conserve kernel memory. If your IPC design silently hits a limit, msgsnd() blocks (or fails with EAGAIN if IPC_NOWAIT is set) — which can be very hard to debug.

Better Alternatives (from Section 46.9)

The book recommends these alternatives when you need message-based IPC:

Mechanism File Descriptor? Named by Reference Counted? Message Types?
SysV MQ No ftok() key No Yes (numeric)
POSIX MQ Yes (on Linux) /name path Yes Yes (priority)
FIFO (named pipe) Yes filesystem path Yes No (byte stream)
UNIX Domain Socket Yes filesystem path Yes Datagram mode

Alternative 1: POSIX Message Queues (mqueue)

POSIX MQs are the direct replacement for SysV MQs. They support message priorities (similar to type filtering), are identified by /name style paths, and return a file descriptor on Linux — enabling use with select()/poll().

#include <mqueue.h>
#include <stdio.h>
#include <string.h>

int main(void)
{
    mqd_t mq;
    struct mq_attr attr;
    char  buf[256];
    unsigned int prio;

    attr.mq_flags   = 0;
    attr.mq_maxmsg  = 10;
    attr.mq_msgsize = 256;
    attr.mq_curmsgs = 0;

    /* Create/open by name - like a file path */
    mq = mq_open("/myqueue", O_CREAT | O_RDWR, 0600, &attr);
    if (mq == (mqd_t)-1) { perror("mq_open"); return 1; }

    /* Send with a priority (higher number = higher priority) */
    mq_send(mq, "hello", 5, 10);   /* priority 10 */
    mq_send(mq, "urgent", 6, 20);  /* priority 20 - received FIRST */

    /* Receive: always gets highest priority message first */
    mq_receive(mq, buf, 256, &prio);
    printf("Got: '%s' (prio=%u)\n", buf, prio);  /* urgent, 20 */

    mq_close(mq);
    mq_unlink("/myqueue");   /* like rm for FIFOs */
    return 0;
}
/* Compile: gcc prog.c -lrt  */

Alternative 2: Two FIFOs for Priority Simulation

If you need “normal” vs “priority” message categories (like using mtype=1 vs mtype=2), you can use two separate FIFOs — one per priority level. Then use select() or poll() to monitor both FDs and always drain the high-priority FIFO first.

#include <stdio.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <poll.h>
#include <unistd.h>

#define FIFO_NORMAL   "/tmp/fifo_normal"
#define FIFO_PRIORITY "/tmp/fifo_priority"

int main(void)
{
    int fd_normal, fd_prio;
    struct pollfd fds[2];
    char buf[256];
    int  ready;

    /* Create two FIFOs */
    mkfifo(FIFO_NORMAL,   0600);
    mkfifo(FIFO_PRIORITY, 0600);

    /* Open both (non-blocking so open() doesn't block waiting
       for the other end) */
    fd_prio   = open(FIFO_PRIORITY, O_RDONLY | O_NONBLOCK);
    fd_normal = open(FIFO_NORMAL,   O_RDONLY | O_NONBLOCK);

    fds[0].fd = fd_prio;    /* index 0 = priority channel */
    fds[0].events = POLLIN;
    fds[1].fd = fd_normal;  /* index 1 = normal channel   */
    fds[1].events = POLLIN;

    /* Wait for input on either FIFO */
    ready = poll(fds, 2, 5000);  /* 5-second timeout */

    if (ready > 0) {
        /* Always check priority channel first */
        if (fds[0].revents & POLLIN) {
            read(fd_prio, buf, sizeof(buf));
            printf("Priority message: %s\n", buf);
        } else if (fds[1].revents & POLLIN) {
            read(fd_normal, buf, sizeof(buf));
            printf("Normal message: %s\n", buf);
        }
    }

    close(fd_prio); close(fd_normal);
    unlink(FIFO_PRIORITY); unlink(FIFO_NORMAL);
    return 0;
}
/* Advantage: works with select/poll, named by path, reference-counted */

Quick Decision Guide

Which IPC to Choose?
Need message boundaries? (not a byte stream)
Need select/poll?
✓ Use POSIX MQ or
UNIX Domain Datagram Socket
Legacy code / type filtering?
⚠ SysV MQ (know the limits)
Simple byte stream?
✓ Use Pipe or FIFO

Interview Questions
Q1. Why can’t you use epoll() with System V message queues?
epoll() (and select()/poll()) only operate on file descriptors. System V message queues are identified by an integer queue ID, not a file descriptor, so the kernel has no fd to monitor for readability. POSIX message queues on Linux return an mqd_t that is a file descriptor and can be used with select()/poll().
Q2. What is ftok() and what is its main weakness?
ftok(pathname, proj_id) converts a filename and project ID into a System V IPC key. Its main weakness is that it uses the file’s inode number and device number to compute the key. If the file is deleted and recreated — even with the same name — it gets a new inode, so ftok() returns a different key. Two processes may then refer to different queues while believing they’re using the same one.
Q3. What does “connectionless” mean in the context of SysV message queues?
It means the kernel does not maintain a count of how many processes are currently using a queue. Compare this with a pipe: the kernel has a reference count, and when all write ends are closed, readers get EOF and the pipe is eventually freed. For SysV MQs, there is no such tracking — the queue stays alive until explicitly deleted with msgctl(IPC_RMID) or a system reboot, regardless of whether any process is using it.
Q4. How would you simulate SysV MQ message types using FIFOs?
Use one FIFO per message type. For example, if you need “normal” (type 1) and “priority” (type 2) messages, create two FIFOs: /tmp/mq_normal and /tmp/mq_priority. The sender writes to the appropriate FIFO based on message type. The receiver uses poll() or select() on both FIFOs, always draining the priority FIFO first. This approach also gains all the fd-based benefits: select() support and automatic cleanup when both ends are closed.
Q5. How can you check and modify System V IPC limits on Linux?
Use the /proc/sys/kernel/ files: msgmni (max queues), msgmax (max message size), msgmnb (max queue capacity). Read them with cat or from code using fopen(). Modify temporarily with echo value > /proc/sys/kernel/msgmnb (as root), or permanently via /etc/sysctl.conf with entries like kernel.msgmnb = 65536 and then sysctl -p. List existing queues with ipcs -q and remove them with ipcrm -q mqid.
Q6. What is the main advantage of POSIX MQ over SysV MQ?
POSIX message queues (mq_open()) fix nearly all SysV MQ disadvantages: (1) They return a file descriptor on Linux — usable with select()/poll(). (2) They are named with a POSIX path (/name) instead of ftok() keys. (3) They support message priorities (like type filtering) natively. (4) They support asynchronous notification via mq_notify() (signal or thread). The main limitation is that POSIX MQs are not supported on all UNIX systems (they are a Linux-specific addition in the real-time extensions).

Leave a Reply

Your email address will not be published. Required fields are marked *