Client-Server with a Single Message Queue IPC Design Patterns

 

Client-Server with a Single Message Queue
Chapter 46 – Section 46.7 (Part A)  |  IPC Design Patterns
46.7
TLPI Section
1 Queue
Both Directions
mtype
Routing Key

Overview: Client-Server IPC Designs

When building a client-server application using System V message queues, a fundamental design decision is: how many queues do we need? Two main approaches exist:

  1. Single shared queue — Both client requests and server responses flow through one queue. The mtype field routes messages to the right reader.
  2. Separate queues per client — The server has one queue for requests. Each client creates its own private queue for receiving responses.

This file covers the single queue design in depth — how it works, how routing is done, and what its limitations are.

Key Concepts

mtype field Message routing Process ID as mtype mtype = 1 for server Selective receive Deadlock risk Queue capacity limit Malicious client

1. How a Single-Queue Design Works

With a single shared message queue, both requests (client → server) and responses (server → client) travel through the same queue. To avoid message mix-ups, the mtype field in each message acts as a routing tag. The rules are:

Message Direction mtype Value Who Reads It
Client → Server (request) 1 (init’s PID, never used as client PID) Server reads with msgtyp=1
Server → Client (response) Client’s PID (e.g., 4321) Client reads with msgtyp=own PID

The client includes its own PID in the message body when sending a request. The server reads this PID, then uses it as the mtype for the response. The client waits for a message with mtype equal to its own PID — this way each client only picks up its own responses.

Message flow in a single-queue design:

CLIENT
PID = 4321
1. msgsnd(mtype=1, body includes pid=4321)

4. msgrcv(msgtyp=4321) → wait for own response

SHARED QUEUE
mtype=1 → [request from pid=4321]
mtype=1 → [request from pid=5678]
mtype=4321 → [response for client 4321]
mtype=5678 → [response for client 5678]

SERVER
2. msgrcv(msgtyp=1) → read all requests

3. Process request, msgsnd(mtype=client_pid)

Why mtype=1 for requests?
Process ID 1 belongs to the init process (or systemd) which is permanently running. Therefore, no regular client process can ever have PID=1. Using mtype=1 for all requests creates a safe, unique routing value that the server can select on without any conflict with client PIDs.

2. Limitations and Problems of the Single-Queue Design

While simple, the single-queue approach has two significant weaknesses:

⚠ Problem 1: Deadlock from Queue Overflow

Message queues have a limited capacity (controlled by msg_qbytes, default 16KB on Linux). If many clients send requests simultaneously, they can fill the queue. Now the server cannot write responses because the queue is full, and clients are also blocked trying to send more requests. Nobody can proceed — this is a deadlock.

Using two queues (one for requests, one for responses) prevents this specific deadlock but still doesn’t fully solve capacity issues.

⚠ Problem 2: Malicious / Crashed Client Blocks Others

Suppose a client crashes after sending a request but before reading the server’s response. The response message with mtype = dead_client_pid stays in the queue forever, consuming space. A malicious client could intentionally do this to fill the queue, effectively denying service to all other clients. No cleanup happens automatically.

Summary: The single-queue design is suitable for simple, low-traffic scenarios with small messages and trusted clients. For production systems, per-client queues are preferred.

Code Example 1: Message Structures for Single-Queue Client-Server

Define the message formats used by both client and server:

#include <sys/types.h>
#include <sys/msg.h>
#include <limits.h>

/* ----------------------------------------------------------------
 * All System V messages MUST start with a long mtype field.
 * The mtype controls routing — who reads this message.
 * ---------------------------------------------------------------- */

/* Message sent FROM client TO server */
struct clientRequest {
    long mtype;           /* Always set to 1 (server's well-known type) */
    pid_t clientPid;      /* Client's PID — server uses this as response mtype */
    char  requestData[256]; /* The actual request payload */
};

/* Message sent FROM server TO client */
struct serverResponse {
    long mtype;           /* Set to clientPid — only that client will read it */
    int  statusCode;      /* 0 = success, -1 = error */
    char responseData[256]; /* The actual response payload */
};

/* ----------------------------------------------------------------
 * KEY RULE:
 *   - Client sends with mtype = 1
 *   - Client receives with msgtyp = getpid()
 *   - Server receives with msgtyp = 1
 *   - Server sends with mtype = req.clientPid
 * ---------------------------------------------------------------- */

Code Example 2: Server Loop — Single Queue

The server reads requests with mtype=1 and responds using the client’s PID as mtype:

#define _GNU_SOURCE
#include <stdio.h>
#include <sys/types.h>
#include <sys/msg.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>

#define SERVER_KEY 0x12345678

struct clientRequest {
    long  mtype;
    pid_t clientPid;
    char  requestData[256];
};

struct serverResponse {
    long mtype;
    int  statusCode;
    char responseData[256];
};

int main(void) {
    int qid;
    struct clientRequest req;
    struct serverResponse resp;

    /* Create the shared message queue (or open if it already exists) */
    qid = msgget(SERVER_KEY, IPC_CREAT | 0666);
    if (qid == -1) { perror("msgget"); exit(1); }

    printf("Server started. Queue ID = %d\n", qid);
    printf("Waiting for requests (mtype=1)...\n");

    while (1) {
        /* Receive ONLY messages with mtype = 1 (client requests) */
        ssize_t n = msgrcv(qid,
                           &req,
                           sizeof(req) - sizeof(long),  /* mtext size */
                           1,                            /* msgtyp = 1 */
                           0);                           /* flags */

        if (n == -1) {
            if (errno == EINTR) continue;  /* Interrupted by signal, retry */
            perror("msgrcv");
            break;
        }

        printf("Request from PID %d: '%s'\n", req.clientPid, req.requestData);

        /* Build and send response back to that specific client */
        resp.mtype      = req.clientPid;  /* Route to the requesting client */
        resp.statusCode = 0;
        snprintf(resp.responseData, sizeof(resp.responseData),
                 "Processed: %s", req.requestData);

        if (msgsnd(qid, &resp, sizeof(resp) - sizeof(long), 0) == -1) {
            perror("msgsnd response");
        }
    }

    /* Cleanup */
    msgctl(qid, IPC_RMID, NULL);
    return 0;
}
msgrcv() parameters explained:
msgrcv(qid, &buf, msgsz, msgtyp, flags)
msgsz = sizeof(struct) – sizeof(long mtype) — do NOT include the mtype in the size
msgtyp = 1 — only receive messages of type 1
flags = 0 — block until a message arrives

Code Example 3: Client Side — Single Queue

The client sends with mtype=1 and waits for a response with mtype=own PID:

#define _GNU_SOURCE
#include <stdio.h>
#include <sys/types.h>
#include <sys/msg.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

#define SERVER_KEY 0x12345678

struct clientRequest {
    long  mtype;
    pid_t clientPid;
    char  requestData[256];
};

struct serverResponse {
    long mtype;
    int  statusCode;
    char responseData[256];
};

int main(int argc, char *argv[]) {
    int qid;
    struct clientRequest  req;
    struct serverResponse resp;

    if (argc != 2) {
        fprintf(stderr, "Usage: %s <request-string>\n", argv[0]);
        exit(1);
    }

    /* Open the existing shared queue (server must create it first) */
    qid = msgget(SERVER_KEY, 0);
    if (qid == -1) { perror("msgget"); exit(1); }

    /* Build the request */
    req.mtype     = 1;           /* All requests use mtype = 1 */
    req.clientPid = getpid();    /* Tell server our PID for routing response */
    strncpy(req.requestData, argv[1], sizeof(req.requestData) - 1);

    /* Send request to server */
    if (msgsnd(qid, &req, sizeof(req) - sizeof(long), 0) == -1) {
        perror("msgsnd");
        exit(1);
    }

    printf("Request sent. Waiting for response (mtype=%d)...\n", (int)getpid());

    /* Wait for response addressed specifically to us (mtype = our PID) */
    if (msgrcv(qid, &resp,
               sizeof(resp) - sizeof(long),
               getpid(),          /* msgtyp = our own PID */
               0) == -1) {
        perror("msgrcv");
        exit(1);
    }

    printf("Response: status=%d, data='%s'\n", resp.statusCode, resp.responseData);
    return 0;
}
Race condition awareness: In this design, if two processes happen to have the same PID (impossible in practice, but worth noting theoretically), they would read each other’s responses. In reality, PIDs are unique at any given moment, so this is safe.

3. When to Use the Single-Queue Design
Scenario Single Queue Suitable? Reason
Small number of clients (1–5) ✓ Yes Low chance of queue overflow
Small messages (<1KB each) ✓ Yes Queue fills slowly
Trusted clients only ✓ Yes No risk of deliberate clogging
Many simultaneous clients ✗ No Deadlock risk from queue overflow
Large response messages ✗ No Queue fills quickly
Public-facing service ✗ No Denial-of-service by malicious clients
Clients that may crash ⚠ Risky Orphaned response messages accumulate

Interview Questions — Single Queue Client-Server Design

Q1. Why is mtype=1 used for client-to-server requests in a single-queue design?
PID 1 is permanently assigned to the init (or systemd) process and can never be the PID of an ordinary client process. Therefore, using mtype=1 for all client requests creates a unique, stable routing value. The server calls msgrcv(qid, &req, size, 1, 0) to selectively receive only request messages without accidentally picking up response messages (which have mtype = a client PID, always > 1). Using the server’s own PID as an alternative is impractical because clients would need a way to discover it.
Q2. How does a client ensure it receives only its own response in a shared queue?
The client sends its PID in the message body as part of the request. The server reads this PID and sets resp.mtype = clientPid before sending the response. The client then calls msgrcv(qid, &resp, size, getpid(), 0) — the msgtyp = getpid() argument tells the kernel to return only messages whose mtype equals the client’s PID. Messages for other clients remain in the queue untouched.
Q3. Describe a deadlock scenario in the single-queue client-server design.
Suppose 10 clients each send multiple large requests, filling the queue to capacity. The server is blocked in msgsnd() trying to send a response — it cannot because the queue is full. Meanwhile, each client is also blocked in msgsnd() trying to send its next request. None can proceed: the server needs to write before clients can drain the queue, but clients are blocking the server from writing. This circular wait is a deadlock. The root cause is that both request and response messages compete for the same limited queue space.
Q4. How does a malicious client attack a single-queue server?
A malicious client sends a request to the server but then never calls msgrcv() to collect the server’s response. The response message with mtype = malicious_client_pid sits in the queue forever. If many such requests are sent, unread responses pile up until the queue is full (msg_cbytes = msg_qbytes). At that point, the server’s msgsnd() blocks, and legitimate clients cannot receive responses either — a denial-of-service attack achieved without any special privileges.
Q5. What happens if the server process crashes while clients are waiting for responses?
If the server crashes, waiting clients will block indefinitely in msgrcv() because no new messages with their PID as mtype will ever arrive. The message queue itself persists even after the server dies — it is an IPC object owned by the kernel, not a process resource. Clients would need a timeout (using IPC_NOWAIT or alarm signals) to detect the server failure. In the single-queue design, there is no built-in mechanism to notify clients that the server is gone.
Q6. Can two servers read from the same queue simultaneously? What are the implications?
Yes, multiple processes can call msgrcv() on the same queue simultaneously. The kernel guarantees that each individual message is delivered to exactly one reader — there is no duplication. However, with two servers both reading mtype=1 requests, there is no control over which server handles which request. The first server to call msgrcv() when a message arrives gets it. This can be used for simple load balancing, but it makes it hard to maintain per-client state in the server, since the same client may be handled by different servers for different requests.
Q7. What is the msgsz argument in msgrcv(), and what is a common mistake when setting it?
msgsz specifies the maximum size of the message body to receive — it does NOT include the long mtype field at the start of the struct. The correct formula is sizeof(struct myMessage) - sizeof(long). A common mistake is passing sizeof(struct myMessage) (including the mtype size), which works in practice but is technically incorrect. Another mistake is passing too small a size, which by default causes msgrcv() to fail with E2BIG (unless MSG_NOERROR flag is used to silently truncate).

Leave a Reply

Your email address will not be published. Required fields are marked *