Overview: Client-Server IPC Designs
When building a client-server application using System V message queues, a fundamental design decision is: how many queues do we need? Two main approaches exist:
- Single shared queue — Both client requests and server responses flow through one queue. The
mtypefield routes messages to the right reader. - Separate queues per client — The server has one queue for requests. Each client creates its own private queue for receiving responses.
This file covers the single queue design in depth — how it works, how routing is done, and what its limitations are.
With a single shared message queue, both requests (client → server) and responses (server → client) travel through the same queue. To avoid message mix-ups, the mtype field in each message acts as a routing tag. The rules are:
| Message Direction | mtype Value | Who Reads It |
|---|---|---|
| Client → Server (request) | 1 (init’s PID, never used as client PID) | Server reads with msgtyp=1 |
| Server → Client (response) | Client’s PID (e.g., 4321) | Client reads with msgtyp=own PID |
The client includes its own PID in the message body when sending a request. The server reads this PID, then uses it as the mtype for the response. The client waits for a message with mtype equal to its own PID — this way each client only picks up its own responses.
1. msgsnd(mtype=1, body includes pid=4321)
4. msgrcv(msgtyp=4321) → wait for own response
3. Process request, msgsnd(mtype=client_pid)
Process ID 1 belongs to the
init process (or systemd) which is permanently running. Therefore, no regular client process can ever have PID=1. Using mtype=1 for all requests creates a safe, unique routing value that the server can select on without any conflict with client PIDs.While simple, the single-queue approach has two significant weaknesses:
Message queues have a limited capacity (controlled by msg_qbytes, default 16KB on Linux). If many clients send requests simultaneously, they can fill the queue. Now the server cannot write responses because the queue is full, and clients are also blocked trying to send more requests. Nobody can proceed — this is a deadlock.
Using two queues (one for requests, one for responses) prevents this specific deadlock but still doesn’t fully solve capacity issues.
Suppose a client crashes after sending a request but before reading the server’s response. The response message with mtype = dead_client_pid stays in the queue forever, consuming space. A malicious client could intentionally do this to fill the queue, effectively denying service to all other clients. No cleanup happens automatically.
Define the message formats used by both client and server:
#include <sys/types.h>
#include <sys/msg.h>
#include <limits.h>
/* ----------------------------------------------------------------
* All System V messages MUST start with a long mtype field.
* The mtype controls routing — who reads this message.
* ---------------------------------------------------------------- */
/* Message sent FROM client TO server */
struct clientRequest {
long mtype; /* Always set to 1 (server's well-known type) */
pid_t clientPid; /* Client's PID — server uses this as response mtype */
char requestData[256]; /* The actual request payload */
};
/* Message sent FROM server TO client */
struct serverResponse {
long mtype; /* Set to clientPid — only that client will read it */
int statusCode; /* 0 = success, -1 = error */
char responseData[256]; /* The actual response payload */
};
/* ----------------------------------------------------------------
* KEY RULE:
* - Client sends with mtype = 1
* - Client receives with msgtyp = getpid()
* - Server receives with msgtyp = 1
* - Server sends with mtype = req.clientPid
* ---------------------------------------------------------------- */
The server reads requests with mtype=1 and responds using the client’s PID as mtype:
#define _GNU_SOURCE
#include <stdio.h>
#include <sys/types.h>
#include <sys/msg.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#define SERVER_KEY 0x12345678
struct clientRequest {
long mtype;
pid_t clientPid;
char requestData[256];
};
struct serverResponse {
long mtype;
int statusCode;
char responseData[256];
};
int main(void) {
int qid;
struct clientRequest req;
struct serverResponse resp;
/* Create the shared message queue (or open if it already exists) */
qid = msgget(SERVER_KEY, IPC_CREAT | 0666);
if (qid == -1) { perror("msgget"); exit(1); }
printf("Server started. Queue ID = %d\n", qid);
printf("Waiting for requests (mtype=1)...\n");
while (1) {
/* Receive ONLY messages with mtype = 1 (client requests) */
ssize_t n = msgrcv(qid,
&req,
sizeof(req) - sizeof(long), /* mtext size */
1, /* msgtyp = 1 */
0); /* flags */
if (n == -1) {
if (errno == EINTR) continue; /* Interrupted by signal, retry */
perror("msgrcv");
break;
}
printf("Request from PID %d: '%s'\n", req.clientPid, req.requestData);
/* Build and send response back to that specific client */
resp.mtype = req.clientPid; /* Route to the requesting client */
resp.statusCode = 0;
snprintf(resp.responseData, sizeof(resp.responseData),
"Processed: %s", req.requestData);
if (msgsnd(qid, &resp, sizeof(resp) - sizeof(long), 0) == -1) {
perror("msgsnd response");
}
}
/* Cleanup */
msgctl(qid, IPC_RMID, NULL);
return 0;
}
msgrcv(qid, &buf, msgsz, msgtyp, flags)msgsz = sizeof(struct) – sizeof(long mtype) — do NOT include the mtype in the size
msgtyp = 1 — only receive messages of type 1
flags = 0 — block until a message arrives
The client sends with mtype=1 and waits for a response with mtype=own PID:
#define _GNU_SOURCE
#include <stdio.h>
#include <sys/types.h>
#include <sys/msg.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#define SERVER_KEY 0x12345678
struct clientRequest {
long mtype;
pid_t clientPid;
char requestData[256];
};
struct serverResponse {
long mtype;
int statusCode;
char responseData[256];
};
int main(int argc, char *argv[]) {
int qid;
struct clientRequest req;
struct serverResponse resp;
if (argc != 2) {
fprintf(stderr, "Usage: %s <request-string>\n", argv[0]);
exit(1);
}
/* Open the existing shared queue (server must create it first) */
qid = msgget(SERVER_KEY, 0);
if (qid == -1) { perror("msgget"); exit(1); }
/* Build the request */
req.mtype = 1; /* All requests use mtype = 1 */
req.clientPid = getpid(); /* Tell server our PID for routing response */
strncpy(req.requestData, argv[1], sizeof(req.requestData) - 1);
/* Send request to server */
if (msgsnd(qid, &req, sizeof(req) - sizeof(long), 0) == -1) {
perror("msgsnd");
exit(1);
}
printf("Request sent. Waiting for response (mtype=%d)...\n", (int)getpid());
/* Wait for response addressed specifically to us (mtype = our PID) */
if (msgrcv(qid, &resp,
sizeof(resp) - sizeof(long),
getpid(), /* msgtyp = our own PID */
0) == -1) {
perror("msgrcv");
exit(1);
}
printf("Response: status=%d, data='%s'\n", resp.statusCode, resp.responseData);
return 0;
}
| Scenario | Single Queue Suitable? | Reason |
|---|---|---|
| Small number of clients (1–5) | ✓ Yes | Low chance of queue overflow |
| Small messages (<1KB each) | ✓ Yes | Queue fills slowly |
| Trusted clients only | ✓ Yes | No risk of deliberate clogging |
| Many simultaneous clients | ✗ No | Deadlock risk from queue overflow |
| Large response messages | ✗ No | Queue fills quickly |
| Public-facing service | ✗ No | Denial-of-service by malicious clients |
| Clients that may crash | ⚠ Risky | Orphaned response messages accumulate |
Interview Questions — Single Queue Client-Server Design
init (or systemd) process and can never be the PID of an ordinary client process. Therefore, using mtype=1 for all client requests creates a unique, stable routing value. The server calls msgrcv(qid, &req, size, 1, 0) to selectively receive only request messages without accidentally picking up response messages (which have mtype = a client PID, always > 1). Using the server’s own PID as an alternative is impractical because clients would need a way to discover it.resp.mtype = clientPid before sending the response. The client then calls msgrcv(qid, &resp, size, getpid(), 0) — the msgtyp = getpid() argument tells the kernel to return only messages whose mtype equals the client’s PID. Messages for other clients remain in the queue untouched.msgsnd() trying to send a response — it cannot because the queue is full. Meanwhile, each client is also blocked in msgsnd() trying to send its next request. None can proceed: the server needs to write before clients can drain the queue, but clients are blocking the server from writing. This circular wait is a deadlock. The root cause is that both request and response messages compete for the same limited queue space.msgrcv() to collect the server’s response. The response message with mtype = malicious_client_pid sits in the queue forever. If many such requests are sent, unread responses pile up until the queue is full (msg_cbytes = msg_qbytes). At that point, the server’s msgsnd() blocks, and legitimate clients cannot receive responses either — a denial-of-service attack achieved without any special privileges.msgrcv() because no new messages with their PID as mtype will ever arrive. The message queue itself persists even after the server dies — it is an IPC object owned by the kernel, not a process resource. Clients would need a timeout (using IPC_NOWAIT or alarm signals) to detect the server failure. In the single-queue design, there is no built-in mechanism to notify clients that the server is gone.msgrcv() on the same queue simultaneously. The kernel guarantees that each individual message is delivered to exactly one reader — there is no duplication. However, with two servers both reading mtype=1 requests, there is no control over which server handles which request. The first server to call msgrcv() when a message arrives gets it. This can be used for simple load balancing, but it makes it hard to maintain per-client state in the server, since the same client may be handled by different servers for different requests.msgsz specifies the maximum size of the message body to receive — it does NOT include the long mtype field at the start of the struct. The correct formula is sizeof(struct myMessage) - sizeof(long). A common mistake is passing sizeof(struct myMessage) (including the mtype size), which works in practice but is technically incorrect. Another mistake is passing too small a size, which by default causes msgrcv() to fail with E2BIG (unless MSG_NOERROR flag is used to silently truncate).← Previous: Listing Queues Next: Per-Client Queue Design → EmbeddedPathashala Home
