Why TCP Needs Concurrency
TCP is different from UDP. TCP creates a persistent connection between client and server. A client may send many messages over that connection and the server must respond to each. If you used an iterative server with TCP, every other client would have to wait until the current client disconnects. That is unacceptable for most real-world services.
The classic solution — and what TLPI Chapter 60 covers — is to fork a child process for each new client. The child handles that client exclusively, while the parent immediately goes back to accept() for the next client.
How TCP Changes the Server Design
- No connections — each datagram is independent
- recvfrom() gets data + sender address
- sendto() replies immediately
- No accept(), no listen()
- One socket serves all clients
- Persistent connections — clients hold a socket open
- listen() + accept() per new client
- Each client gets a dedicated connected socket
- Must handle multiple simultaneous connections
- fork() creates a child per client
How the Concurrent TCP Server Works
Step-by-Step Flow
Complete Concurrent TCP Echo Server
This is a standalone concurrent TCP echo server implementing the full design from Chapter 60. Every part is heavily commented:
/*
* tcp_echo_server.c - Concurrent TCP echo server
* (Mirrors the design from TLPI Chapter 60, Section 60.3)
*
* Compile: gcc -o tcp_echo_server tcp_echo_server.c
* Run: sudo ./tcp_echo_server 7 (port 7 needs root)
* ./tcp_echo_server 50007 (any non-privileged port)
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <errno.h>
#include <signal.h>
#include <sys/wait.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#define BUF_SIZE 4096
#define BACKLOG 50 /* listen() backlog — max pending connections */
/*
* SIGCHLD handler — reaps all terminated child processes.
*
* IMPORTANT: use a loop with WNOHANG because:
* - Multiple children may exit around the same time
* - Linux may coalesce multiple SIGCHLD signals into one
* - A single waitpid() call might miss some children
*/
static void sigchld_handler(int sig)
{
int saved_errno = errno; /* Must save/restore — waitpid changes errno */
while (waitpid(-1, NULL, WNOHANG) > 0)
continue; /* keep reaping until no more zombies */
errno = saved_errno;
}
/*
* serve_client() — this runs in the child process.
*
* Reads data from the connected socket and writes it back.
* Loops until the client closes the connection (read returns 0).
*/
static void serve_client(int cfd)
{
char buf[BUF_SIZE];
ssize_t numRead;
while ((numRead = read(cfd, buf, BUF_SIZE)) > 0) {
/*
* write() in a loop is important — write() may not send all bytes
* in one call if the socket send buffer is full.
*/
ssize_t written = 0;
while (written < numRead) {
ssize_t n = write(cfd, buf + written, numRead - written);
if (n <= 0) {
if (n == -1)
perror("write");
return;
}
written += n;
}
}
if (numRead == -1)
perror("read");
/* numRead == 0 means client closed connection — normal exit */
}
int main(int argc, char *argv[])
{
int lfd, cfd;
struct sockaddr_in svaddr, claddr;
socklen_t cllen;
struct sigaction sa;
int port;
char clAddrStr[INET_ADDRSTRLEN];
pid_t childPid;
if (argc != 2) {
fprintf(stderr, "Usage: %s port\n", argv[0]);
exit(EXIT_FAILURE);
}
port = atoi(argv[1]);
/*
* ---- Step 1: Install SIGCHLD handler ----
*
* Must do this BEFORE calling fork() the first time.
* SA_RESTART: restart accept() if interrupted by SIGCHLD.
* Without SA_RESTART, accept() would return -1/EINTR when a child exits.
*/
sigemptyset(&sa.sa_mask);
sa.sa_flags = SA_RESTART;
sa.sa_handler = sigchld_handler;
if (sigaction(SIGCHLD, &sa, NULL) == -1) {
perror("sigaction");
exit(EXIT_FAILURE);
}
/*
* ---- Step 2: Create TCP listening socket ----
*/
lfd = socket(AF_INET, SOCK_STREAM, 0);
if (lfd == -1) {
perror("socket");
exit(EXIT_FAILURE);
}
/* Allow reuse of the port immediately after server restart */
int optval = 1;
setsockopt(lfd, SOL_SOCKET, SO_REUSEADDR, &optval, sizeof(optval));
memset(&svaddr, 0, sizeof(svaddr));
svaddr.sin_family = AF_INET;
svaddr.sin_addr.s_addr = htonl(INADDR_ANY);
svaddr.sin_port = htons(port);
if (bind(lfd, (struct sockaddr *) &svaddr, sizeof(svaddr)) == -1) {
perror("bind");
exit(EXIT_FAILURE);
}
if (listen(lfd, BACKLOG) == -1) {
perror("listen");
exit(EXIT_FAILURE);
}
printf("TCP echo server listening on port %d\n", port);
/*
* ---- Step 3: The main accept loop ----
*
* Parent loops here forever, accepting new clients and forking children.
*/
for (;;) {
cllen = sizeof(struct sockaddr_in);
/*
* accept() blocks until a client connects.
* Returns a NEW socket (cfd) connected to that specific client.
* lfd continues to accept future connections.
*/
cfd = accept(lfd, (struct sockaddr *) &claddr, &cllen);
if (cfd == -1) {
if (errno == EINTR)
continue; /* Interrupted by signal (SIGCHLD) — retry */
perror("accept");
continue;
}
inet_ntop(AF_INET, &claddr.sin_addr, clAddrStr, INET_ADDRSTRLEN);
printf("Client connected: %s:%d\n",
clAddrStr, ntohs(claddr.sin_port));
/*
* Fork a child to handle this client.
*/
childPid = fork();
switch (childPid) {
case -1:
/*
* fork() failed — can't serve this client.
* Close the connection gracefully and continue accepting.
*/
perror("fork");
close(cfd);
break;
case 0:
/*
* ---- CHILD PROCESS ----
*
* MUST close lfd: child never calls accept(), and keeping
* lfd open wastes a file descriptor. If parent later closes
* lfd, the socket won't fully close until all children also
* close their copies.
*/
close(lfd);
serve_client(cfd);
close(cfd);
printf("Child %d: done serving client\n", (int)getpid());
_exit(EXIT_SUCCESS); /* _exit() avoids flushing stdio buffers */
/* NOTREACHED */
default:
/*
* ---- PARENT PROCESS ----
*
* MUST close cfd: the child has its own copy (via fork).
* If parent doesn't close cfd, the client's connection will
* stay open even after child exits and closes its copy,
* because the socket ref-count won't reach zero.
*/
close(cfd);
printf("Parent: forked child PID %d for client\n", (int)childPid);
/* Go back to accept() */
break;
}
}
/* Never reached */
close(lfd);
return 0;
}
TCP Echo Client for Testing
/*
* tcp_echo_client.c - test client for the TCP echo server
* Compile: gcc -o tcp_echo_client tcp_echo_client.c
* Usage: ./tcp_echo_client 127.0.0.1 50007
* Then type messages — each line is echoed back
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#define BUF_SIZE 4096
int main(int argc, char *argv[])
{
int sfd;
struct sockaddr_in svaddr;
char buf[BUF_SIZE];
ssize_t numRead;
if (argc != 3) {
fprintf(stderr, "Usage: %s server-ip port\n", argv[0]);
exit(EXIT_FAILURE);
}
sfd = socket(AF_INET, SOCK_STREAM, 0);
if (sfd == -1) { perror("socket"); exit(EXIT_FAILURE); }
memset(&svaddr, 0, sizeof(svaddr));
svaddr.sin_family = AF_INET;
svaddr.sin_port = htons(atoi(argv[2]));
if (inet_pton(AF_INET, argv[1], &svaddr.sin_addr) <= 0) {
fprintf(stderr, "Invalid address\n"); exit(EXIT_FAILURE);
}
if (connect(sfd, (struct sockaddr *) &svaddr, sizeof(svaddr)) == -1) {
perror("connect"); exit(EXIT_FAILURE);
}
printf("Connected to %s:%s. Type messages (Ctrl+D to quit):\n",
argv[1], argv[2]);
/* Read from stdin, send to server, read echo, print it */
while (fgets(buf, BUF_SIZE, stdin) != NULL) {
size_t len = strlen(buf);
if (write(sfd, buf, len) != (ssize_t)len) {
perror("write"); break;
}
numRead = read(sfd, buf, BUF_SIZE);
if (numRead <= 0) break;
printf("Echo: %.*s", (int)numRead, buf);
}
close(sfd);
printf("Connection closed.\n");
return 0;
}
The Four Critical Rules — Summary
After fork(), child must close the listening socket. It never calls accept() and keeping lfd wastes a file descriptor.
After fork(), parent must close the connected socket. If it does not, the client connection will never truly close even after the child exits.
Install the SIGCHLD handler before accepting any clients. If a child exits before the handler is installed, it becomes a zombie that may never be reaped.
Child must call _exit(), not exit(). exit() flushes and closes stdio buffers — in a forked process, this could corrupt the parent’s stdio state and cause double-flushing.
Why SO_REUSEADDR Matters
When a TCP server crashes or is stopped, its port may stay in TIME_WAIT state for up to 2 minutes. During this time, trying to restart the server fails with “Address already in use”. Setting SO_REUSEADDR before bind() tells the kernel to allow reuse of the port even in TIME_WAIT:
int optval = 1;
/* Must set BEFORE bind() */
setsockopt(lfd, SOL_SOCKET, SO_REUSEADDR, &optval, sizeof(optval));
bind(lfd, (struct sockaddr *) &svaddr, sizeof(svaddr));
Why write() Needs a Loop
A common mistake is assuming write() always sends all the bytes you ask for. On sockets, write() may send fewer bytes than requested if the kernel’s send buffer is full. You must loop until all bytes are sent:
/* WRONG — may not send all bytes */
write(cfd, buf, numRead);
/* RIGHT — loop until everything is sent */
ssize_t written = 0;
while (written < numRead) {
ssize_t n = write(cfd, buf + written, numRead - written);
if (n <= 0) {
/* n == 0: unusual, n == -1: error */
perror("write");
return;
}
written += n;
}
Interview Questions & Answers
accept() waits for an incoming TCP connection on the listening socket. When a client completes the three-way handshake, accept() returns a new file descriptor (the connected socket) specific to that client. The original listening socket remains open and continues to accept future connections. The connected socket is what you use to read from and write to that specific client.
The backlog parameter to listen() specifies the maximum number of pending connections that the kernel will queue before the server calls accept(). These are connections that have completed the TCP handshake but not yet been accept()ed. When the queue is full, the kernel silently drops new SYN packets (the client’s connection request), which causes the client to retry. The client does not get an explicit error — it just times out or retries until a slot opens up.
After fork(), the child inherits the parent’s stdio buffers. If the child calls exit(), it flushes and closes those buffers. But the parent is still running and may have unflushed data in the same buffers. This can cause the parent’s buffered output to be written twice (once by the child on exit, once by the parent later). Using _exit() bypasses stdio cleanup entirely — it terminates the child immediately without touching stdio buffers, which is the correct behavior for a forked child in a server.
TCP is a stream protocol. write() on a socket writes to the kernel’s send buffer. If the send buffer is full (because the network is slow or the receiver’s buffer is full), write() may return having sent only part of the data without an error. If you do not check the return value and loop, you silently lose the remaining bytes. A robust server always loops write() until all bytes have been accepted by the kernel.
If five child processes terminate at nearly the same time, the kernel might deliver only one or two SIGCHLD signals (signals of the same type can be merged/coalesced). If the handler calls waitpid() exactly once per SIGCHLD, it reaps one zombie per signal. The remaining terminated children stay as zombies permanently, wasting process table entries. A loop with WNOHANG ensures all ready zombies are reaped every time the handler runs.
When a signal arrives and interrupts a blocking system call like accept(), the system call returns -1 with errno == EINTR. Without SA_RESTART, the server code must check for EINTR and manually restart the call. With SA_RESTART, the kernel automatically restarts certain interrupted system calls (including accept(), read(), write()) as if they were never interrupted. This simplifies the server code significantly.
When a TCP server stops, any connections that were in the process of closing enter TIME_WAIT state for up to 4 minutes (2 × MSL). During this time, the kernel will reject attempts to bind the same port, giving “Address already in use”. Setting SO_REUSEADDR before bind() tells the kernel to allow the bind even if the port is in TIME_WAIT. This lets you restart a server immediately without waiting. Every production TCP server should set this option as a matter of practice.
Track the number of active children in a global counter. Increment it when you fork, decrement it in the SIGCHLD handler when a child exits. Before calling accept(), check if the counter has hit the maximum. If so, either pause accepting (let the listen backlog buffer clients) or close the new connection with an error. Example:
static volatile sig_atomic_t numChildren = 0;
#define MAX_CHILDREN 100
/* In SIGCHLD handler */
while (waitpid(-1, NULL, WNOHANG) > 0)
numChildren--;
/* In accept loop */
if (numChildren >= MAX_CHILDREN) {
/* Too many children - close and wait */
close(cfd);
continue;
}
fork();
numChildren++;
Continue to Part 4
Learn about inetd — the super-daemon that launches servers on demand