Alternative I/O Models Nonblocking I/O: O_NONBLOCK and EAGAIN

 

Chapter 63: Alternative I/O Models
Part 2 of 5 โ€” Nonblocking I/O: O_NONBLOCK and EAGAIN
๐Ÿ“– TLPI Chapter 63
โฑ ~20 min read
๐Ÿ’ป Linux Systems

Nonblocking I/O is the first technique you learn to break free from blocking. Instead of sleeping when data is not available, your process gets an error immediately and can go do something else. This sounds great, but as you will see, naive use of nonblocking I/O has serious problems. Understanding these problems is what motivates select(), poll(), and epoll.

Keywords:

O_NONBLOCK EAGAIN EWOULDBLOCK fcntl() F_SETFL F_GETFL Polling Loop CPU Spin Busy-wait

How Nonblocking I/O Works

By default, every file descriptor you open is in blocking mode. When you enable O_NONBLOCK on a file descriptor, the behavior changes:

Blocking vs Nonblocking Behavior
Blocking Mode (default)
1. Call read(fd, buf, n)
2. No data? โ†’ Process SLEEPS
3. Data arrives โ†’ Kernel wakes process
4. read() returns with data
Nonblocking Mode (O_NONBLOCK)
1. Call read(fd, buf, n)
2. No data? โ†’ Returns -1, errno=EAGAIN
3. Process is NOT blocked
4. Process can do other work

The file descriptor types that support O_NONBLOCK include: pipes, FIFOs, sockets, terminals, pseudoterminals, and some other device types. Regular disk files mostly ignore O_NONBLOCK because the kernel buffer cache makes them appear always ready.

How to Set O_NONBLOCK

You can set nonblocking mode in two ways:

Method 1: At open() time

#include <fcntl.h>

/* Set O_NONBLOCK when opening the file */
int fd = open("/dev/ttyS0", O_RDWR | O_NONBLOCK);
if (fd == -1) {
    perror("open");
    return 1;
}
    
Method 2: On an existing fd using fcntl()

#include <fcntl.h>
#include <unistd.h>

/*
 * To change mode on an existing fd (like STDIN, a socket, etc.)
 * Step 1: Get current flags
 * Step 2: Add O_NONBLOCK to flags
 * Step 3: Set updated flags back
 */
int set_nonblocking(int fd) {
    int flags;

    /* Step 1: Read current file status flags */
    flags = fcntl(fd, F_GETFL);
    if (flags == -1) {
        perror("fcntl F_GETFL");
        return -1;
    }

    /* Step 2: Add nonblocking flag */
    flags |= O_NONBLOCK;

    /* Step 3: Write flags back */
    if (fcntl(fd, F_SETFL, flags) == -1) {
        perror("fcntl F_SETFL");
        return -1;
    }

    return 0;  /* success */
}
    
โš  Important: Always use F_GETFL first, then OR in O_NONBLOCK. Do not just set O_NONBLOCK directly with F_SETFL or you will clear all the other flags (like O_APPEND, O_RDWR etc.) that may already be set on the fd.

Handling EAGAIN / EWOULDBLOCK

When you call read() on a nonblocking fd with no data available, it returns -1 and sets errno to EAGAIN. On some systems you may also see EWOULDBLOCK. On Linux these are the same value (both equal 11), but you should check for both for portability.


#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <errno.h>
#include <string.h>

int main() {
    char buf[256];
    ssize_t n;

    /* Put stdin in nonblocking mode */
    int flags = fcntl(STDIN_FILENO, F_GETFL);
    fcntl(STDIN_FILENO, F_SETFL, flags | O_NONBLOCK);

    printf("Trying nonblocking read...\n");

    n = read(STDIN_FILENO, buf, sizeof(buf) - 1);

    if (n == -1) {
        if (errno == EAGAIN || errno == EWOULDBLOCK) {
            /*
             * This is NOT a real error. It just means
             * no data is available right now.
             * We can come back and try again later.
             */
            printf("No data available right now (EAGAIN)\n");
            printf("We are free to do other work!\n");
        } else {
            /* This IS a real error */
            perror("read");
        }
    } else if (n == 0) {
        printf("EOF reached\n");
    } else {
        buf[n] = '\0';
        printf("Read %zd bytes: %s\n", n, buf);
    }

    return 0;
}
    

The Polling Problem: Why Tight Loops are Bad

Now you understand nonblocking I/O. A naive developer might think: “Great! I will just loop and keep calling read() until it returns data. Since it returns immediately with EAGAIN when no data is there, I can keep checking all my file descriptors in a loop!”

This technique is called busy-waiting or polling in a tight loop. Here is what it looks like:


/*
 * WRONG APPROACH: Tight polling loop
 * This works but wastes 100% CPU doing nothing useful
 */
int fd1, fd2;  /* two sockets, both nonblocking */
char buf[256];
ssize_t n;

while (1) {
    /* Try socket 1 */
    n = read(fd1, buf, sizeof(buf));
    if (n > 0) {
        /* Process data from fd1 */
        handle_data(fd1, buf, n);
    } else if (n == -1 && errno != EAGAIN) {
        perror("read fd1");
        break;
    }

    /* Try socket 2 */
    n = read(fd2, buf, sizeof(buf));
    if (n > 0) {
        /* Process data from fd2 */
        handle_data(fd2, buf, n);
    } else if (n == -1 && errno != EAGAIN) {
        perror("read fd2");
        break;
    }

    /*
     * PROBLEM: Most of the time both reads return EAGAIN.
     * We spin in this loop calling read() millions of times
     * per second, consuming 100% of one CPU core.
     * The OS scheduler cannot give that CPU time to other
     * useful processes.
     */
}
    

CPU Time Distribution in Tight Polling Loop
Activity % of CPU Time Is it Useful?
read() returning EAGAIN (no data) ~99.9% โŒ Wasted
Actually processing real data ~0.1% โœ” Useful

The correct solution is to use a mechanism where your process sleeps until at least one file descriptor has data ready, then wakes up and only reads from the ones that are ready. This is exactly what select(), poll(), and epoll() do.

๐Ÿ’ก Infrequent polling is also bad: If you add a sleep() in your loop to reduce CPU usage, you add latency. If data arrives right after you went to sleep, your response will be delayed by the sleep duration. There is no good tuning point.

Which File Descriptors Support O_NONBLOCK?
File Descriptor Type O_NONBLOCK Supported? Behavior
Sockets (TCP/UDP/Unix) โœ” Yes Returns EAGAIN if no data / send buffer full
Pipes โœ” Yes Returns EAGAIN if pipe empty or full
FIFOs (named pipes) โœ” Yes Returns EAGAIN if no data
Terminals / TTYs โœ” Yes Returns EAGAIN if no input
Pseudoterminals โœ” Yes Returns EAGAIN if no data
Regular Disk Files โœ– Mostly No Kernel buffer cache makes them appear always ready. O_NONBLOCK is effectively ignored for reads.

Full Example: Nonblocking Read on a Pipe

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <errno.h>

/*
 * Demonstrates nonblocking read on a pipe.
 * Child writes after a delay.
 * Parent reads in nonblocking mode and can detect "not ready".
 */
int main() {
    int pipefd[2];
    pid_t pid;
    char buf[64];
    ssize_t n;
    int flags;

    /* Create a pipe */
    if (pipe(pipefd) == -1) {
        perror("pipe");
        exit(EXIT_FAILURE);
    }

    pid = fork();
    if (pid == -1) {
        perror("fork");
        exit(EXIT_FAILURE);
    }

    if (pid == 0) {
        /* Child: close read end, write after 2 seconds */
        close(pipefd[0]);
        sleep(2);
        const char *msg = "Hello from child!";
        write(pipefd[1], msg, strlen(msg));
        close(pipefd[1]);
        exit(EXIT_SUCCESS);
    }

    /* Parent: close write end */
    close(pipefd[1]);

    /* Set read end to nonblocking */
    flags = fcntl(pipefd[0], F_GETFL);
    if (flags == -1) {
        perror("fcntl F_GETFL");
        exit(EXIT_FAILURE);
    }
    if (fcntl(pipefd[0], F_SETFL, flags | O_NONBLOCK) == -1) {
        perror("fcntl F_SETFL");
        exit(EXIT_FAILURE);
    }

    /*
     * Try reading every 500ms.
     * In a real program you would use select/poll/epoll instead
     * of sleeping. This is just for demonstration.
     */
    for (int i = 0; i < 10; i++) {
        n = read(pipefd[0], buf, sizeof(buf) - 1);

        if (n == -1) {
            if (errno == EAGAIN) {
                printf("Attempt %d: No data yet (EAGAIN) - doing other work...\n", i+1);
            } else {
                perror("read");
                break;
            }
        } else if (n == 0) {
            printf("Pipe closed by writer (EOF)\n");
            break;
        } else {
            buf[n] = '\0';
            printf("Attempt %d: Got data: '%s'\n", i+1, buf);
            break;
        }

        usleep(500000);  /* sleep 500ms between attempts */
    }

    close(pipefd[0]);
    return 0;
}
    
Expected output:
Attempt 1: No data yet (EAGAIN) - doing other work...
Attempt 2: No data yet (EAGAIN) - doing other work...
Attempt 3: No data yet (EAGAIN) - doing other work...
Attempt 4: No data yet (EAGAIN) - doing other work...
Attempt 5: Got data: 'Hello from child!'

Interview Questions
Q1. What errno value is returned when a nonblocking read has no data available?

errno is set to EAGAIN. On some systems it may be EWOULDBLOCK. On Linux both are the same value. Your code should check for both: if (errno == EAGAIN || errno == EWOULDBLOCK). This is NOT a real error. It simply means try again later.

Q2. What is the correct way to add O_NONBLOCK to an already-open file descriptor?

Use fcntl() with F_GETFL to get existing flags first, then OR in O_NONBLOCK, then call F_SETFL. Never call F_SETFL with just O_NONBLOCK alone because that clears all other flags like O_RDWR or O_APPEND that may already be set.

Q3. Does O_NONBLOCK work on regular disk files?

Not really. The Linux kernel treats regular files as always ready for I/O because of the buffer cache. Setting O_NONBLOCK on a regular file has no practical effect on read() and write(). Nonblocking I/O is mainly meaningful for sockets, pipes, FIFOs, and terminals.

Q4. What is a tight polling loop and why is it a problem?

A tight polling loop is when a process continuously calls read() in a loop on nonblocking file descriptors. When no data is available, the calls return EAGAIN immediately and the loop repeats. This consumes nearly 100% of one CPU core even when there is no real work to do. It also reduces the CPU time available to other processes on the system.

Q5. What is the difference between EAGAIN and EINTR?

EAGAIN means the I/O would block because no data is available. EINTR means the system call was interrupted by a signal. Both cause a system call to return -1, but they mean different things. For EINTR you typically restart the system call. For EAGAIN you wait and try later (or use select/poll/epoll to wait efficiently).

Q6. In which scenarios does write() return EAGAIN on a nonblocking socket?

When the socket send buffer is full. TCP sockets have a kernel send buffer. If the remote end is not consuming data fast enough, the buffer fills up. A blocking write() would sleep until space is available. A nonblocking write() returns -1 with EAGAIN immediately, telling you to try again later when buffer space frees up.

Next: I/O Multiplexing with select() and poll()
Part 3 explains how select() and poll() let you efficiently wait for any of many file descriptors to become ready, solving the CPU waste problem of nonblocking polling loops.

โ† Part 1: Overview Part 3: select() and poll() โ†’

Leave a Reply

Your email address will not be published. Required fields are marked *