Creating a new process using fork() is not free. It involves:

• Allocating a new process table entry in the kernel.
• Copying page table entries (even with copy-on-write, there is overhead).
• Duplicating file descriptors.
• Setting up signal handlers, memory mappings, etc.

For a server handling a few clients per second, this is fine. But for a web server serving thousands of requests per minute, the accumulated cost of all those fork() calls becomes a serious burden.

Instead of calling fork() when a client arrives, the server creates a pool of child processes at startup, before any clients connect. These children are called worker processes. Each worker waits for a client, handles it, and then loops back to wait for the next client — it never terminates between clients.

The key insight: the listening socket is created by the parent before forking. Every child inherits a copy of that file descriptor and can call accept() on it. The kernel ensures only one child gets each incoming connection.

When multiple worker processes all block in accept() on the same socket, and a new client connects, the kernel wakes up all the sleeping workers. Only one of them will successfully complete accept(); the rest will sleep again. This is called the thundering herd problem.

On Linux, this is handled correctly by the kernel itself: modern Linux wakes only one process per incoming connection for accept(), so the thundering herd is not a real problem here.

However, on older UNIX implementations, accept() was not atomic. Multiple processes could simultaneously be woken up and race to get the connection. The fix was to wrap accept() in a file lock (mutex):

#include <fcntl.h>

/* Each worker acquires a lock before calling accept() */
static int lockFd;   /* Shared lock file */

static void lockAcquire(void)
{
    struct flock fl;
    fl.l_type   = F_WRLCK;
    fl.l_whence = SEEK_SET;
    fl.l_start  = 0;
    fl.l_len    = 0;
    if (fcntl(lockFd, F_SETLKW, &fl) == -1) {
        perror("fcntl F_SETLKW");
        exit(EXIT_FAILURE);
    }
}

static void lockRelease(void)
{
    struct flock fl;
    fl.l_type   = F_UNLCK;
    fl.l_whence = SEEK_SET;
    fl.l_start  = 0;
    fl.l_len    = 0;
    if (fcntl(lockFd, F_SETLK, &fl) == -1) {
        perror("fcntl F_SETLK");
        exit(EXIT_FAILURE);
    }
}

/* Worker main loop with locking */
for (;;) {
    lockAcquire();          /* Only ONE worker in accept() at a time */
    cfd = accept(lfd, NULL, NULL);
    lockRelease();

    if (cfd == -1) {
        perror("accept");
        continue;
    }

    handleRequest(cfd);
    close(cfd);
}

On modern Linux this lock is not needed, but you may still see it in portable server code written for multiple UNIX systems.

The same idea applied to threads instead of processes: create a pool of worker threads at startup. Threads are much cheaper to create than processes because they share the same address space (no page table copy needed).

#include <pthread.h>

#define NUM_THREADS  10
#define PORT         9000

static int lfd;                    /* Listening socket - shared by all threads */
static pthread_mutex_t acceptMutex = PTHREAD_MUTEX_INITIALIZER;

static void *workerThread(void *arg)
{
    int cfd;
    char buf[1024];
    ssize_t n;

    for (;;) {
        /* Serialize access to accept() using a mutex */
        pthread_mutex_lock(&acceptMutex);
        cfd = accept(lfd, NULL, NULL);
        pthread_mutex_unlock(&acceptMutex);

        if (cfd == -1) {
            perror("accept");
            continue;
        }

        /* Echo data back to client */
        while ((n = read(cfd, buf, sizeof(buf))) > 0)
            write(cfd, buf, n);

        close(cfd);
        /* Loop back to accept() - no thread creation overhead! */
    }
    return NULL;
}

int main(void)
{
    pthread_t tid[NUM_THREADS];
    struct sockaddr_in addr;
    int i;

    /* Create listening socket */
    lfd = socket(AF_INET, SOCK_STREAM, 0);
    int opt = 1;
    setsockopt(lfd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));

    memset(&addr, 0, sizeof(addr));
    addr.sin_family      = AF_INET;
    addr.sin_port        = htons(PORT);
    addr.sin_addr.s_addr = htonl(INADDR_ANY);
    bind(lfd, (struct sockaddr *)&addr, sizeof(addr));
    listen(lfd, 10);

    /* Pre-create the thread pool */
    for (i = 0; i < NUM_THREADS; i++)
        pthread_create(&tid[i], NULL, workerThread, NULL);

    /* Main thread waits for workers (they run forever) */
    for (i = 0; i < NUM_THREADS; i++)
        pthread_join(tid[i], NULL);

    return 0;
}

Compile with gcc -o prethreaded_server server.c -lpthread

A fixed pool size is a simple starting point, but a production server needs dynamic pool management. The parent (or main thread) monitors the pool and adjusts it based on load:

/* Simplified pool management concept */
#define MIN_WORKERS   5
#define MAX_WORKERS   100
#define IDLE_THRESHOLD 2   /* if fewer than this are idle, add more */

static int activeWorkers = 0;
static int idleWorkers   = 0;

void managePool(void)
{
    /* Called periodically by the parent process or a management thread */

    if (idleWorkers < IDLE_THRESHOLD && activeWorkers < MAX_WORKERS) {
        /* Too few idle workers; add more */
        fork_new_worker();
        activeWorkers++;
    }

    if (idleWorkers > MIN_WORKERS * 2 && activeWorkers > MIN_WORKERS) {
        /* Too many idle workers; reduce pool */
        signal_worker_to_exit();
        activeWorkers--;
    }
}

Real implementations (like Apache HTTP server’s prefork MPM) implement sophisticated pool management with configurable MinSpareServers, MaxSpareServers, StartServers, and MaxRequestWorkers parameters.

Instead of all workers calling accept() themselves, the parent can do all the accept() calls and then pass the connected socket to a free worker using a UNIX domain socket (a technique called file descriptor passing).

This design gives the parent full control over load balancing — it can choose the least-loaded worker, implement priority queues, etc. The trade-off is the overhead of the socket message for each new connection. This technique (using SCM_RIGHTS) is covered in detail in TLPI Section 61.13.3.