Problems with select() and poll()

 

Problems with select() and poll()
Chapter 63 ยท Why These APIs Break Down at Scale ยท Linux Programming Interface
โšก Scaling
๐Ÿ” Kernel Copy
๐Ÿง  O(n) Problem
โ“ Interview Q&A

The Core Problem

select() and poll() work fine for small numbers of file descriptors. But when you are building a high-performance server that handles thousands or tens of thousands of simultaneous connections, these APIs start to become a serious bottleneck. This tutorial explains exactly why โ€” and what the kernel-level limitation is.

Key Terms in This Tutorial

O(n) scaling kernel scan user-to-kernel copy FD_SETSIZE persistent fd list epoll signal-driven I/O C10K problem

1. The Three Core Problems

Every time you call select() or poll(), three expensive things happen:

What happens on every select() / poll() call

1
Kernel scans ALL specified file descriptors
Even if 9999 out of 10000 fds have no activity, the kernel must check every single one. This is pure wasted CPU for large idle sets.

โ–ผ

2
Full data structure copied between user space and kernel space on every call
Your program passes the entire fd list to the kernel. The kernel modifies it and returns it. This copying happens EVERY call โ€” even when the monitored set never changes.

โ–ผ

3
Program must scan entire returned structure to find ready fds
After select()/poll() returns, your code must loop through every entry to find which ones are ready. Again O(n) work even if only 1 fd is ready.

The combined result: the CPU time consumed grows linearly with the number of file descriptors being monitored, not with the number of active (ready) fds. This is called O(n) scaling โ€” very bad for high-connection servers.

2. The Data Structure Copy Cost

Let’s look at the data structures involved and understand why copying them is expensive.

select() โ€” fd_set bitmask
  • Size is always fixed: FD_SETSIZE bits (usually 1024 bits = 128 bytes)
  • Full bitmask copied to kernel EVERY call
  • Kernel modifies it, copies full bitmask back
  • You must reinitialize before every call
  • Wastes bandwidth for small active sets
poll() โ€” pollfd array
  • Size grows with number of fds being watched
  • Each pollfd = 8 bytes (fd + events + revents)
  • 10,000 fds = 80 KB copied to kernel per call
  • No forced reinit โ€” but still copied every call
  • More flexible but still O(n) per call
/*
 * Rough illustration of copy cost with poll() at scale
 *
 * Each struct pollfd = 8 bytes
 * 10,000 connections = 10,000 * 8 = 80,000 bytes per call
 * If poll() is called 1000 times/second:
 *   = 80,000 * 1000 = 80 MB/sec just for copies between user and kernel
 *
 * This is pure overhead unrelated to actual I/O work.
 */

#include <poll.h>
#include <stdlib.h>
#include <stdio.h>

#define MAX_CLIENTS 10000

int main(void)
{
    struct pollfd *fds = malloc(MAX_CLIENTS * sizeof(struct pollfd));
    if (!fds) {
        perror("malloc");
        return 1;
    }

    /* Initialize all entries */
    for (int i = 0; i < MAX_CLIENTS; i++) {
        fds[i].fd     = i + 3;    /* pretend fds start at 3 */
        fds[i].events = POLLIN;
        fds[i].revents = 0;
    }

    /* Every call to poll() copies 80,000 bytes to kernel and back */
    while (1) {
        int ret = poll(fds, MAX_CLIENTS, -1);

        /* After returning, must loop through ALL 10,000 entries */
        for (int i = 0; i < MAX_CLIENTS; i++) {
            if (fds[i].revents & POLLIN) {
                /* handle fd fds[i].fd */
                printf("fd %d is ready\n", fds[i].fd);
                fds[i].revents = 0;
            }
        }
    }

    free(fds);
    return 0;
}

3. The Root Cause โ€” Kernel Has No Memory Between Calls

The deepest reason select() and poll() scale poorly is fundamental to their design:

๐Ÿ”‘ Root Cause

The kernel does not remember the list of file descriptors between successive calls to select() or poll(). Each call is completely stateless from the kernel’s perspective. So every time you call these functions, you must hand the entire list to the kernel again, the kernel must build up its internal watch state from scratch, scan all entries, and then destroy that state when it returns.

โŒ What select()/poll() does
  1. App sends full fd list to kernel
  2. Kernel builds watch state
  3. Kernel waits for activity
  4. Kernel reports ready fds
  5. Kernel forgets everything
  6. Repeat from step 1 next call
โœ… What we actually need
  1. App registers fd list once
  2. Kernel keeps watch state permanently
  3. Kernel only notifies when something happens
  4. No re-copying, no re-scanning
  5. Scales with events, not with fd count
  6. โ† This is what epoll provides!

4. The Solutions โ€” Signal-Driven I/O and epoll

Both signal-driven I/O and epoll solve the root cause problem by letting the kernel maintain a persistent interest list. You register your interest once, and the kernel remembers it across multiple wait operations.

Scaling Comparison
Mechanism Scales with Kernel memory? Best for
select() O(highest fd number) No โ€” stateless Small fd counts (<100)
poll() O(number of monitored fds) No โ€” stateless Medium fd counts, sparse sets
Signal-driven I/O O(events occurred) Yes โ€” persistent Single fd, simple cases
epoll O(events occurred) Yes โ€” persistent High-scale servers (10K+ fds)

The key insight: select() and poll() performance degrades even when no I/O is happening and the fd set is unchanged. epoll and signal-driven I/O only do work proportional to how much actual I/O activity is occurring.

5. Code Demo โ€” Measuring the O(n) Problem
/*
 * demo_poll_scale.c
 *
 * This shows the O(n) nature of poll():
 * - Create N pipe pairs
 * - Monitor read ends with poll()
 * - Only write to the last one
 * - Notice poll() still scans ALL N entries
 */

#include <poll.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <time.h>

#define N_PIPES 1000   /* try 10, 100, 1000, 10000 */

int main(void)
{
    int pipes[N_PIPES][2];
    struct pollfd fds[N_PIPES];

    /* Create N pipes */
    for (int i = 0; i < N_PIPES; i++) {
        if (pipe(pipes[i]) == -1) {
            perror("pipe");
            exit(1);
        }
        fds[i].fd     = pipes[i][0];  /* read end */
        fds[i].events = POLLIN;
        fds[i].revents = 0;
    }

    /* Write one byte to the LAST pipe only */
    write(pipes[N_PIPES - 1][1], "x", 1);

    /* Time the poll() call */
    struct timespec start, end;
    clock_gettime(CLOCK_MONOTONIC, &start);

    /* poll() must scan ALL N_PIPES entries, even though
     * only the last one is ready */
    int ret = poll(fds, N_PIPES, 0);

    clock_gettime(CLOCK_MONOTONIC, &end);

    long ns = (end.tv_sec - start.tv_sec) * 1000000000L
              + (end.tv_nsec - start.tv_nsec);

    printf("N_PIPES=%d, ret=%d, time=%ld ns\n", N_PIPES, ret, ns);

    /* Check which fd is ready */
    for (int i = 0; i < N_PIPES; i++) {
        if (fds[i].revents & POLLIN)
            printf("Ready: index %d, fd %d\n", i, fds[i].fd);
    }

    /* Cleanup */
    for (int i = 0; i < N_PIPES; i++) {
        close(pipes[i][0]);
        close(pipes[i][1]);
    }
    return 0;
}

/*
 * Compile: gcc -o demo_poll_scale demo_poll_scale.c
 * Run with different N_PIPES values and compare the time output.
 * You will see time grows linearly with N_PIPES.
 * With epoll, time would remain near-constant regardless of N_PIPES.
 */

Interview Questions & Answers
Q1. What are the three main performance problems with select() and poll() when monitoring large numbers of file descriptors?First, the kernel must check every specified file descriptor on each call regardless of whether any activity occurred. Second, the program must copy the entire fd list to the kernel and receive it back on every call โ€” this copy is O(n) in size. Third, after the call returns, the program must iterate through every entry to find which ones are ready, again O(n) work. All three operations scale with the number of monitored fds rather than with actual I/O activity.

Q2. What is the fundamental reason select() and poll() do not scale well?The kernel does not remember the list of file descriptors between calls. Each call is stateless โ€” the full interest list must be rebuilt from scratch on every invocation. This forces repeated redundant work proportional to the number of descriptors even when the monitored set and the activity level have not changed.

Q3. How do signal-driven I/O and epoll solve the scaling problem that select() and poll() cannot?Both allow the kernel to maintain a persistent interest list. With signal-driven I/O, you register interest in a file descriptor once and the kernel delivers a signal only when I/O becomes possible. With epoll, you add descriptors to a kernel-side interest list using epoll_ctl() once, and then call epoll_wait() which returns only the descriptors that are actually ready. Neither mechanism requires sending the full fd list to the kernel on every wait. They scale with the number of I/O events that occur, not with the total number of monitored descriptors.

Q4. Why is the data structure copy overhead worse for poll() than for select() as the number of connections grows?For select(), the fd_set bitmask size is fixed at FD_SETSIZE bits (typically 1024 bits = 128 bytes), regardless of how many fds are actually monitored. For poll(), the pollfd array grows proportionally โ€” each pollfd entry is 8 bytes, so 10,000 connections means 80 KB is copied between user space and kernel space on every call. The copy cost for poll() scales linearly with the number of monitored descriptors, whereas select() has a fixed ceiling (though FD_SETSIZE limits which fds you can use).

Q5. What is the C10K problem and how does it relate to select() and poll()?The C10K problem refers to the challenge of handling 10,000 simultaneous client connections on a single server. With select() or poll(), each wait call involves O(n) kernel scanning, O(n) data copying, and O(n) result inspection โ€” all proportional to 10,000. Even with no actual client activity, the server wastes significant CPU just managing the monitoring overhead. This made select()/poll() unsuitable for high-concurrency servers, which led to the development of epoll (Linux), kqueue (BSD), and IOCP (Windows).

Q6. For select(), does passing a larger nfds value increase overhead even if fewer descriptors are in the fd_set?Yes. The nfds parameter tells the kernel how many positions to scan in the fd_set bitmask. The kernel scans from position 0 to position (nfds – 1) regardless of how many are actually set. So if you pass nfds = 10000 but only have one fd set to bit position 9999, the kernel still examines 10000 positions. This is why select() performs poorly with sparse high-numbered descriptor sets.

Next Topic

Now that you understand the limitations, learn how Signal-Driven I/O addresses them.

Next: Signal-Driven I/O โ†’ โ† Back: select() vs poll()

Leave a Reply

Your email address will not be published. Required fields are marked *