Level-Triggered vs Edge-Triggered The Most Important Concept in Linux I/O Multiplexing

 

Level-Triggered vs Edge-Triggered
Chapter 63 — Part 5: The Most Important Concept in Linux I/O Multiplexing

Why This Matters

If you are using epoll, this is the concept you absolutely must understand. Getting it wrong causes subtle bugs where data silently sits unread in a buffer forever — and your server appears to hang for certain clients.

The terms come from electronics: level-triggered and edge-triggered describe when a circuit responds to a signal — based on the signal’s current level, or based on the transition (edge) in the signal.

The Core Difference — Simple Analogy

Level-Triggered
“Keep ringing the bell as long as there is food on the plate”
  • Notification repeats while condition is true
  • If data remains unread, you are notified again next time
  • Forgiving — you cannot miss data by reading partially
  • select(), poll(), and epoll default mode
Edge-Triggered
“Ring the bell once when food arrives — do not ring again until new food arrives”
  • Notification only on state change (new data arriving)
  • If you only partially read data, no more notifications until new data
  • Requires reading ALL data until EAGAIN after each notification
  • epoll with EPOLLET flag; signal-driven I/O approximates this

Visualizing the Difference — Timeline

Scenario: 100 bytes arrive. You read 50 bytes. 50 bytes remain unread.

Event Level-Triggered (LT) Edge-Triggered (ET)
100 bytes arrive on socket ✓ Notified ✓ Notified
You read 50 bytes (50 left)
Next epoll_wait() call ✓ Notified again (50 bytes still there) ✗ NOT notified (no new data arrived)
New 10 bytes arrive ✓ Notified (60 bytes now) ✓ Notified (triggered by new arrival)
The ET bug: With edge-triggered, if you read only 50 of 100 bytes and no new data arrives, those 50 bytes sit in the buffer silently forever — the client appears to hang. This is why ET requires reading until EAGAIN every time you get an event.

Level-Triggered epoll — The Safe Default

/* Level-triggered is the DEFAULT in epoll — just don't add EPOLLET */

struct epoll_event ev;
ev.events  = EPOLLIN;         /* No EPOLLET — this is level-triggered */
ev.data.fd = fd;
epoll_ctl(epfd, EPOLL_CTL_ADD, fd, &ev);

/* In event loop — with LT you can read partial data safely */
if (events[i].events & EPOLLIN) {
    char buf[512];
    ssize_t n = read(fd, buf, sizeof(buf));  /* Read what we can */
    if (n > 0) {
        process(buf, n);
        /* If 2000 bytes were available but we only read 512,
           next epoll_wait() will notify us again — safe */
    }
}

Edge-Triggered epoll — The Fast But Tricky Mode

/* Edge-triggered: add EPOLLET flag */

struct epoll_event ev;
ev.events  = EPOLLIN | EPOLLET;   /* Edge-triggered */
ev.data.fd = fd;
epoll_ctl(epfd, EPOLL_CTL_ADD, fd, &ev);

/* RULE: With ET, you MUST read until EAGAIN every single time */
if (events[i].events & EPOLLIN) {
    char buf[512];
    ssize_t n;

    /* Keep reading until no more data (EAGAIN/EWOULDBLOCK) */
    while (1) {
        n = read(fd, buf, sizeof(buf));

        if (n == -1) {
            if (errno == EAGAIN || errno == EWOULDBLOCK) {
                /* All data consumed — stop reading */
                break;
            }
            /* Real error */
            perror("read");
            close(fd);
            epoll_ctl(epfd, EPOLL_CTL_DEL, fd, NULL);
            break;
        }

        if (n == 0) {
            /* EOF — client disconnected */
            close(fd);
            epoll_ctl(epfd, EPOLL_CTL_DEL, fd, NULL);
            break;
        }

        /* Process data */
        process(buf, n);
        /* Loop continues to drain ALL available data */
    }
}
Critical Rule for Edge-Triggered: The fd must be set to nonblocking (O_NONBLOCK) before adding it to epoll with EPOLLET. If the fd is blocking and you read until EAGAIN, you will block forever on the last read() when the buffer becomes empty — because a blocking fd waits for more data instead of returning EAGAIN.

Setting Nonblocking — Required for Edge-Triggered

#include <fcntl.h>

void set_nonblocking(int fd)
{
    int flags = fcntl(fd, F_GETFL, 0);
    if (flags == -1) {
        perror("fcntl F_GETFL");
        return;
    }
    if (fcntl(fd, F_SETFL, flags | O_NONBLOCK) == -1) {
        perror("fcntl F_SETFL");
    }
}

/* Usage in accept() loop: */
int conn_fd = accept(listen_fd, NULL, NULL);
set_nonblocking(conn_fd);                       /* Must do this BEFORE */
ev.events  = EPOLLIN | EPOLLET;
ev.data.fd = conn_fd;
epoll_ctl(epfd, EPOLL_CTL_ADD, conn_fd, &ev);  /* Adding to epoll */

The Starvation Problem with Edge-Triggered

Imagine fd A has 1MB of data and fd B has 100 bytes. With edge-triggered and a loop “read until EAGAIN”, you will spend a long time draining fd A’s 1MB before you even look at fd B. fd B’s client is waiting.

This is the starvation problem. One solution is to maintain a list of fds that have been notified but not fully drained, and round-robin through them.

ET Event Loop with Starvation Risk
fd A: 1MB data
read loop runs for long time…
finally EAGAIN
fd B: 100 bytes
waiting this whole time…
finally processed

When to Use Level-Triggered vs Edge-Triggered

Situation Use Reason
Simple server, easier to reason about Level-Triggered Forgiving — partial reads are safe
Maximum performance, experienced developer Edge-Triggered Fewer kernel-user transitions
Multi-threaded server ET + EPOLLONESHOT Prevents concurrent event delivery per fd
Porting from select()/poll() Level-Triggered Behavior matches select()/poll() semantics

Which I/O Model Uses Which Trigger Model?

select()
Level-Triggered only
poll()
Level-Triggered only
Signal-Driven I/O
Approximately Edge-Triggered
epoll
Both — LT default, ET with EPOLLET

Interview Questions

Q1: Explain the difference between level-triggered and edge-triggered in epoll.
Level-triggered (the default) means epoll notifies you whenever a condition is true — if an fd has unread data, every call to epoll_wait() reports it as readable until all data is consumed. Edge-triggered (EPOLLET) only notifies you when a state change occurs — when new data arrives. If you partially read data, you will not get another notification until the next new data arrives, even if there is still unread data in the buffer.
Q2: Why must you use nonblocking I/O with edge-triggered epoll?
With edge-triggered, you must read all available data on each event notification. That means reading in a loop until read() returns EAGAIN (no more data available). If the fd is in blocking mode, when the buffer becomes empty, read() will block indefinitely waiting for new data rather than returning EAGAIN. This deadlocks your event loop. Nonblocking mode ensures read() returns immediately with EAGAIN when no more data is available, so your loop can exit and process other events.
Q3: What is the starvation problem with edge-triggered epoll?
When using edge-triggered mode, you must drain each fd completely. If one fd has a very large amount of data (say 1MB), your loop keeps reading that one fd until EAGAIN, while other fds with pending events wait. This is called starvation — some clients get all the attention while others are ignored. A solution is to read a fixed maximum per event (but then switch back to level-triggered), or maintain a pending-drain list and round-robin through it.
Q4: Does select() support edge-triggered notifications?
No. select() and poll() are always level-triggered. They report an fd as ready as long as the condition is true. Only epoll supports both models — level-triggered by default and edge-triggered when you add the EPOLLET flag. Signal-driven I/O approximates edge-triggered behavior since you get one signal per state change, not continuous signals.
Q5: In terms of performance, which is faster — level-triggered or edge-triggered epoll?
Edge-triggered can be faster in theory because when you drain all data in one shot (until EAGAIN), the kernel’s ready list is cleared and subsequent epoll_wait() calls have less work to do. With level-triggered, the kernel must keep putting the fd back onto the ready list every time you call epoll_wait() while data remains unread. However, the performance difference is usually small compared to actual I/O costs, and the complexity of edge-triggered makes level-triggered the better choice for most applications.

Leave a Reply

Your email address will not be published. Required fields are marked *