You have now learned all four alternative I/O models: nonblocking I/O, I/O multiplexing (select/poll), signal-driven I/O, and epoll. This final part ties everything together. We look at a complete side-by-side comparison, a decision guide for choosing the right model, the libevent portability layer, and close with a comprehensive set of interview questions covering the entire chapter.
Here is every model side by side across the most important dimensions:
| Criterion | Blocking I/O | Nonblocking + Poll Loop | select() / poll() | Signal-driven I/O | epoll |
|---|---|---|---|---|---|
| Process blocks? | Yes (in read/write) | No | Yes (in select/poll) | No | Yes (in epoll_wait) |
| Handles multiple FDs? | No | Yes | Yes | Yes | Yes |
| CPU usage (idle) | Low (sleeping) | 100% (spinning) | Low (sleeping) | Low | Low (sleeping) |
| Scales to 1000s of FDs? | No | No (CPU waste) | Poor (O(n) scan) | Yes | Excellent |
| POSIX portable? | Yes | Yes | Yes | Partially | Linux only |
| Complexity | Very simple | Simple | Moderate | High (signal safety) | Moderate |
| FD limit | System limit | System limit | 1024 (select) / None (poll) | System limit | System limit |
| Best use case | Single client / simple tools | Never preferred in practice | Portable apps, low FD count | Rarely used directly | High-perf Linux servers |
Simple, clean, easy to understand. Perfect for single-FD programs.
poll() works everywhere. libevent picks the best backend automatically.
O(n) overhead is negligible at small scale. Simpler code.
Best performance on Linux. Used by Nginx, Node.js, Redis.
| Software | I/O Model Used | Why |
|---|---|---|
| Nginx | epoll (Linux), kqueue (BSD) | Handles millions of concurrent connections with minimal threads |
| Node.js | libuv (uses epoll/kqueue/IOCP) | Single-threaded event loop based on epoll on Linux |
| Redis | epoll (Linux), kqueue (BSD) | Single-threaded high-speed event loop |
| Python asyncio | selectors module (epoll on Linux) | Uses best available I/O mechanism per OS |
| BlueZ (Bluetooth stack) | poll() / epoll | Monitors HCI socket, L2CAP sockets, and control sockets |
| Apache httpd | select() or poll() (event MPM uses epoll) | Older versions used select(). Modern event MPM uses epoll. |
The practical problem: your server might need to run on Linux (where epoll is available), macOS/BSD (where kqueue is available), Solaris (where /dev/poll is available), and even Windows. Each platform has its own best I/O event API. Writing separate code for each is painful.
libevent solves this by providing a single event-monitoring API that internally uses the best mechanism available on the current platform. Written by Niels Provos, libevent is used by Memcached, Tor, and many other well-known projects.
libevent picks the best available backend automatically at runtime
| Platform | Best Native API | libevent uses |
|---|---|---|
| Linux 2.6+ | epoll | epoll |
| macOS / FreeBSD / NetBSD | kqueue | kqueue |
| Solaris | /dev/poll | /dev/poll |
| Windows | IOCP / select() | select() (IOCP in newer libevent) |
| Old UNIX / fallback | poll() or select() | poll() then select() |
- libuv — Used by Node.js. Handles I/O events, timers, threads, DNS. More feature-rich than libevent.
- libev — Lighter-weight alternative to libevent by the same author as libevent’s fork.
- Boost.Asio — C++ async I/O library using epoll/kqueue/IOCP.
This is the skeleton of a real production-style TCP server using epoll. It demonstrates the complete event-loop pattern used by Nginx, Redis and others.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <errno.h>
#include <netinet/in.h>
#include <sys/socket.h>
#include <sys/epoll.h>
#define PORT 8080
#define MAX_EVENTS 64
#define BUF_SIZE 1024
/* Helper: set a socket to nonblocking mode */
static int set_nonblocking(int fd) {
int flags = fcntl(fd, F_GETFL);
if (flags == -1) return -1;
return fcntl(fd, F_SETFL, flags | O_NONBLOCK);
}
int main() {
int listen_fd, conn_fd, epfd, n, i, ret;
struct sockaddr_in addr;
struct epoll_event ev, events[MAX_EVENTS];
char buf[BUF_SIZE];
/* 1. Create listening TCP socket */
listen_fd = socket(AF_INET, SOCK_STREAM, 0);
if (listen_fd == -1) { perror("socket"); exit(1); }
/* Allow port reuse (avoid "Address already in use" on restart) */
int opt = 1;
setsockopt(listen_fd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));
/* 2. Bind and listen */
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_addr.s_addr = INADDR_ANY;
addr.sin_port = htons(PORT);
if (bind(listen_fd, (struct sockaddr *)&addr, sizeof(addr)) == -1) {
perror("bind"); exit(1);
}
if (listen(listen_fd, 128) == -1) {
perror("listen"); exit(1);
}
/* 3. Set listen socket nonblocking (good practice with epoll) */
set_nonblocking(listen_fd);
/* 4. Create epoll instance */
epfd = epoll_create1(0);
if (epfd == -1) { perror("epoll_create1"); exit(1); }
/* 5. Register listen socket with epoll */
ev.events = EPOLLIN;
ev.data.fd = listen_fd;
epoll_ctl(epfd, EPOLL_CTL_ADD, listen_fd, &ev);
printf("Echo server listening on port %d...\n", PORT);
/* 6. Main event loop */
while (1) {
ret = epoll_wait(epfd, events, MAX_EVENTS, -1); /* -1 = wait forever */
if (ret == -1) {
if (errno == EINTR) continue; /* interrupted by signal, retry */
perror("epoll_wait");
break;
}
/* 7. Process only the ready events */
for (i = 0; i < ret; i++) {
if (events[i].data.fd == listen_fd) {
/*
* New client connection on the listening socket.
* Accept it, set nonblocking, add to epoll.
*/
conn_fd = accept(listen_fd, NULL, NULL);
if (conn_fd == -1) {
if (errno != EAGAIN && errno != EWOULDBLOCK)
perror("accept");
continue;
}
set_nonblocking(conn_fd);
ev.events = EPOLLIN | EPOLLET; /* Edge-triggered for clients */
ev.data.fd = conn_fd;
epoll_ctl(epfd, EPOLL_CTL_ADD, conn_fd, &ev);
printf("New client: fd=%d\n", conn_fd);
} else if (events[i].events & EPOLLIN) {
/*
* Data from an existing client.
* In edge-triggered mode: read in a loop until EAGAIN.
*/
int fd = events[i].data.fd;
while (1) {
n = read(fd, buf, BUF_SIZE);
if (n == -1) {
if (errno == EAGAIN || errno == EWOULDBLOCK)
break; /* All data read. Done for now. */
perror("read");
goto close_fd;
}
if (n == 0) goto close_fd; /* Client disconnected */
/* Echo the data back to the client */
write(fd, buf, n);
}
continue;
close_fd:
printf("Client fd=%d disconnected\n", events[i].data.fd);
epoll_ctl(epfd, EPOLL_CTL_DEL, events[i].data.fd, NULL);
close(events[i].data.fd);
} else if (events[i].events & (EPOLLERR | EPOLLHUP)) {
/* Error or hangup: close the connection */
epoll_ctl(epfd, EPOLL_CTL_DEL, events[i].data.fd, NULL);
close(events[i].data.fd);
}
}
}
close(epfd);
close(listen_fd);
return 0;
}
Compile: gcc -o echo_server echo_server.c && ./echo_server
Test: telnet localhost 8080 (type anything and see it echoed back)
The C10K problem (from a 1999 paper by Dan Kegel) is the challenge of handling 10,000 concurrent client connections on a single server. With the traditional one-thread-per-client model, 10,000 threads consume enormous memory and CPU context-switching overhead. epoll-based servers use a single thread (or small thread pool) with an event loop. Only the FDs with actual events are processed, so the server does minimal work per idle connection. This is how Nginx handles millions of connections with little memory.
You will lose data. Suppose 1000 bytes arrive and you read only 500 bytes, then return from your event handler. In edge-triggered mode, epoll_wait() will NOT report that FD as ready again because no new data has arrived. The remaining 500 bytes sit in the kernel buffer unread, and the client may time out waiting for a response. This is why edge-triggered mode mandates reading in a loop until read() returns -1 with errno == EAGAIN.
Yes, you can. A file descriptor can be registered with multiple epoll instances. Each instance will independently notify you when that FD is ready. This is useful in multi-threaded servers where different threads maintain their own epoll instances and share some FDs (like a listening socket).
The Reactor pattern is a design pattern for handling service requests delivered concurrently by one or more inputs. A dispatcher (the reactor) monitors event sources (file descriptors) and dispatches events to associated handlers. epoll is the Linux kernel mechanism used to implement the Reactor pattern. Your epoll event loop IS a reactor: epoll_wait() is the dispatcher, and your per-FD handling code is the handler. Nginx, Node.js, and Redis all implement the Reactor pattern using epoll on Linux.
When a file descriptor is closed, the kernel removes it from all epoll interest lists automatically. However this gets tricky with dup()/dup2() or fork(). If two FDs point to the same underlying file description and you close one, the epoll entry is NOT removed because the file description is still open through the other FD. This is a known gotcha: always call epoll_ctl(EPOLL_CTL_DEL) before close() if duplicates might exist, to be safe.
No, you do not need to explicitly add EPOLLERR or EPOLLHUP to the events field. The kernel always reports these in revents regardless of what you set in events. They are like POLLERR and POLLHUP in poll(). Always check for them in your event loop to handle error and hangup conditions correctly, even though you never set them when calling epoll_ctl.
select() uses a struct timeval which specifies the timeout in seconds and microseconds. poll() uses a plain int in milliseconds. Both return 0 when the timeout expires. Additionally, select() may modify the timeout struct on return (showing remaining time) on some systems, but Linux updates it to show the remaining time. Poll() does not modify its timeout parameter.
If multiple clients connect simultaneously before your epoll_wait() runs, the listening socket will have multiple connections queued. In edge-triggered mode, epoll only fires once (on the transition). If you call accept() only once, you accept only one connection and leave the others queued. The next epoll_wait() will NOT fire for the listening socket again because no NEW connection has arrived since the last notification. So with edge-triggered epoll you must call accept() in a loop until it returns EAGAIN, just like you must drain read data.
libevent provides a single API for registering events and handling them, regardless of the underlying OS. Internally it uses epoll on Linux, kqueue on BSD/macOS, /dev/poll on Solaris, and select()/poll() as a fallback. Your application code calls libevent’s event_add(), event_dispatch() etc. and libevent handles the platform-specific calls transparently. This lets you write high-performance event-driven code once and run it on multiple platforms without code changes.
Undefined behavior. The fd_set is a fixed bitmask of size FD_SETSIZE (1024 on Linux). If you call FD_SET with an FD number equal to or greater than FD_SETSIZE, you write outside the bounds of the fd_set structure, corrupting memory. This is a classic buffer overflow. The program may crash or produce incorrect results silently. This is why you should use poll() or epoll for servers that may open more than 1024 file descriptors.
EPOLLONESHOT is a flag you can add to ev.events. After epoll_wait() returns an event for that FD, the FD is automatically disabled in the interest list. No further events will be reported for it until you re-arm it with epoll_ctl(EPOLL_CTL_MOD). This is useful in multi-threaded servers where you want only one thread to process each FD at a time. Without EPOLLONESHOT, multiple threads could simultaneously receive events for the same FD and process it concurrently, causing race conditions.
On embedded Linux systems the number of concurrent connections is usually small (tens or low hundreds). In this case select()/poll() are perfectly adequate and have the advantage of being simpler to understand and debug. epoll is justified when you have hundreds or thousands of simultaneous connections or very high event rates. For production embedded networking daemons that handle many Bluetooth or socket connections simultaneously, epoll is the better choice. For simple utilities or control programs with a handful of FDs, poll() keeps the code clean and portable.