stdio Library Buffering in Linux: Complete Developer Guide

📦 stdio Library Buffering

printf() and fwrite() add their own buffer on top of the kernel. Here’s how to control it.

In Part 1, we saw the kernel buffer cache. But there’s another buffer that comes before that — inside the C standard library itself. When you call printf() or fwrite(), the C library doesn’t call write() every time. It stores data in its own buffer and calls write() only when the buffer fills up. This is stdio buffering — and understanding it prevents hard-to-find bugs.

1. The Three Buffering Modes

The C standard library supports three modes of buffering for streams (FILE * objects):

_IONBF — No Buffering

Every single character you write triggers a write() system call immediately. No data is held back.

Default for: stderr

Use when: You need every byte delivered immediately (error logs, debug output)

_IOLBF — Line Buffering

Data accumulates until a newline ('\n') is written (or buffer fills), then write() is called.

Default for: stdout when connected to a terminal

Use when: Interactive terminal output, line-oriented protocols

_IOFBF — Full Buffering

Data accumulates in a fixed buffer. write() is called only when the buffer is full (or stream is flushed/closed).

Default for: stdout/stdin when redirected to a file

Use when: Disk files — maximum efficiency

Visual: Where stdio Buffer Sits

Your Code
printf(“hello”); fwrite(buf, 1, n, fp); fputs(str, fp);

stdio Buffer (Layer 1 — User Space)
Controlled by: setvbuf() · setbuf() · fflush()
Default size: 8192 bytes (BUFSIZ)

write() system call (when buffer flushes)

Kernel Buffer Cache (Layer 2 — Kernel Space)
Controlled by: fsync() · fdatasync() · O_SYNC

Physical Disk

2. setvbuf() — Set Buffering Mode

#include <stdio.h>

int setvbuf(FILE *stream, char *buf, int mode, size_t size);
/* Returns: 0 on success, nonzero on error */

Parameters:

stream — the FILE* to configure (e.g., stdout, or a file you opened with fopen())
buf — pointer to your own buffer (or NULL to let the library allocate one)
mode — one of _IONBF, _IOLBF, _IOFBF
size — size of the buffer (ignored if buf is NULL in glibc)

⚠️ Rule: You must call setvbuf() before any I/O operations on the stream. Call it right after opening the file.

Code Example: Using setvbuf()

#include <stdio.h>
#include <stdlib.h>

int main(void) {
    /* --- Example 1: Disable buffering on stdout --- */
    /* Useful when you want every printf() to appear immediately */
    if (setvbuf(stdout, NULL, _IONBF, 0) != 0) {
        perror("setvbuf stdout unbuffered");
        return 1;
    }
    printf("This appears immediately (no buffering)\n");

    /* --- Example 2: Use your own buffer for a file --- */
    FILE *fp = fopen("data.txt", "w");
    if (!fp) { perror("fopen"); return 1; }

    #define MY_BUF_SIZE 8192
    /* IMPORTANT: buf must be static or heap — NOT a local stack variable
       because it must survive until the stream is closed */
    static char my_buf[MY_BUF_SIZE];

    /* Set full buffering with our custom buffer */
    if (setvbuf(fp, my_buf, _IOFBF, MY_BUF_SIZE) != 0) {
        perror("setvbuf file");
        fclose(fp);
        return 1;
    }

    /* These writes go to my_buf first — no system call yet */
    fputs("Line 1\n", fp);
    fputs("Line 2\n", fp);
    /* ... more writes ... */

    fclose(fp);  /* Flushes my_buf to kernel buffer and disk */

    /* --- Example 3: Line buffering for a log file --- */
    FILE *log = fopen("app.log", "a");
    if (!log) { perror("fopen log"); return 1; }

    /* Each fprintf ending with '\n' triggers a write() */
    setvbuf(log, NULL, _IOLBF, 0);
    fprintf(log, "App started: timestamp=%ld\n", (long)0);
    fprintf(log, "Processing...\n");  /* Each line written immediately */

    fclose(log);
    return 0;
}

🚨 Stack Buffer Bug: Never pass a local (stack) variable as the buffer to setvbuf(). When your function returns, that stack memory is gone — but the FILE* still thinks it owns that buffer. Use static or malloc() instead.

3. setbuf() and setbuffer() — Simpler Wrappers

#include <stdio.h>

void setbuf(FILE *stream, char *buf);
/* buf = NULL → _IONBF (no buffering) */ /* buf = pointer → _IOFBF with BUFSIZ (8192) bytes */

/* BSD extension — not in SUSv3, but available on most UNIX */ #define _BSD_SOURCE
void setbuffer(FILE *stream, char *buf, size_t size);

Both are simplified wrappers around setvbuf(). The call setbuf(fp, buf) is equivalent to:

setvbuf(fp, buf, (buf != NULL) ? _IOFBF : _IONBF, BUFSIZ);
/* BUFSIZ is defined in stdio.h — typically 8192 in glibc */

Code Example: setbuf() and setbuffer()

#include <stdio.h>
#define _BSD_SOURCE

int main(void) {
    FILE *fp = fopen("output.txt", "w");
    if (!fp) { perror("fopen"); return 1; }

    /* Method 1: setbuf(fp, NULL) — disable buffering */
    setbuf(fp, NULL);
    fputs("This goes to kernel immediately\n", fp);

    fclose(fp);

    /* Method 2: setbuf(fp, buf) — full buffering with BUFSIZ */
    static char buf[BUFSIZ];
    fp = fopen("output2.txt", "w");
    if (!fp) { perror("fopen"); return 1; }
    setbuf(fp, buf);   /* Equivalent: setvbuf(fp, buf, _IOFBF, BUFSIZ) */
    fputs("Buffered output\n", fp);
    fclose(fp);

    /* Method 3: setbuffer — full buffering with custom size */
    static char small_buf[256];
    fp = fopen("output3.txt", "w");
    if (!fp) { perror("fopen"); return 1; }
    setbuffer(fp, small_buf, sizeof(small_buf));
    fputs("Custom buffer size\n", fp);
    fclose(fp);

    return 0;
}

4. fflush() — Force Flush at Any Time

#include <stdio.h>

int fflush(FILE *stream);
/* Returns: 0 on success, EOF on error */ /* stream = NULL → flushes ALL open stdio streams */

fflush(fp) forces all data in the stdio buffer to be passed to the kernel via write(). It does not guarantee data is on disk — it only moves data from stdio buffer to kernel buffer cache. To guarantee disk write, follow with fsync(fileno(fp)).

Automatic flush happens when:

The buffer fills up
A newline is written (line-buffered mode)
The stream is closed with fclose()
The program exits normally (via exit())
In glibc: when reading from stdin (implicitly flushes stdout)

Code Example: fflush() Use Cases

#include <stdio.h>
#include <unistd.h>  /* for sleep() */
#include <string.h>

/* Use case 1: Progress bar / interactive prompts */
void show_progress(void) {
    for (int i = 0; i <= 100; i += 10) {
        printf("\rProgress: [");
        for (int j = 0; j < i/10; j++) printf("=");
        for (int j = i/10; j < 10; j++) printf(" ");
        printf("] %d%%", i);

        /* Without fflush, nothing appears on terminal until '\n' or buffer fills
           because stdout is line-buffered when connected to terminal.
           When redirected to a file/pipe, stdout becomes FULLY buffered! */
        fflush(stdout);

        sleep(1);  /* Simulate work */
    }
    printf("\nDone!\n");
}

/* Use case 2: Flushing before a prompt */
void ask_user(void) {
    printf("Enter your name: ");  /* No newline at the end! */
    fflush(stdout);               /* Must flush or prompt may not appear */

    char name[64];
    fgets(name, sizeof(name), stdin);
    printf("Hello, %s", name);
}

/* Use case 3: Flush all streams before fork() */
#include <unistd.h>
void safe_fork(void) {
    /* If you fork() without flushing, child inherits the stdio buffer
       with unflushed data. Both parent and child will eventually write
       that data — causing duplicate output! */
    fflush(NULL);   /* Flush ALL open streams */
    pid_t pid = fork();
    if (pid == 0) {
        /* child: do child work */
        _exit(0);  /* Use _exit() in child — not exit() — to avoid flushing again */
    }
    /* parent continues */
}

/* Use case 4: Guaranteed disk write */
#include <fcntl.h>  /* for fileno() */
void guaranteed_disk_write(FILE *fp, const char *data) {
    fputs(data, fp);
    fflush(fp);               /* Moves stdio buf → kernel buf */
    fsync(fileno(fp));        /* Moves kernel buf → disk */
    /* Now data is guaranteed on disk */
}

int main(void) {
    show_progress();
    ask_user();
    return 0;
}

🚨 The fork() + stdio Bug: If you have unflushed stdio data and call fork(), the buffer is copied to the child. Both parent and child will flush it on exit, causing duplicate output. Always call fflush(NULL) before fork().

5. Practical Scenarios and Gotchas

Scenario: stdout Behavior Changes When Redirected

/* save as demo.c */
#include <stdio.h>
#include <unistd.h>

int main(void) {
    printf("Line 1\n");
    write(STDOUT_FILENO, "write() output\n", 15);
    printf("Line 2\n");
    return 0;
}

/* Compile: gcc -o demo demo.c

   Running interactively (stdout = terminal, LINE-buffered):
   $ ./demo
   Line 1           ← printf flushes on '\n'
   write() output   ← goes directly to fd 1
   Line 2

   Running with redirection (stdout = file, FULLY-buffered):
   $ ./demo > output.txt && cat output.txt
   write() output   ← write() bypasses stdio buffer, goes to kernel directly
   Line 1           ← printf output flushed at exit, AFTER write()
   Line 2

   The order changes! This surprises many developers.
   Fix: add fflush(stdout) before the write() call.
*/

Code Example: Correctly Mixing printf and write

#include <stdio.h>
#include <unistd.h>
#include <string.h>

int main(void) {
    /* Always flush before mixing write() and printf() on same fd */
    printf("This comes from printf — part 1. ");
    fflush(stdout);   /* ← Critical! */

    write(STDOUT_FILENO, "This from write(). ", 19);

    printf("Back to printf.\n");
    fflush(stdout);   /* Flush again before next write() */

    write(STDOUT_FILENO, "Final write.\n", 13);
    return 0;
}
/* Output will now always be in the correct order */

Code Example: Embedded System — Unbuffered Serial Log

#include <stdio.h>
#include <stdarg.h>
#include <time.h>

/* In embedded systems, you often want logs to appear immediately.
   This logger disables stdio buffering so every log line is written
   to the kernel buffer right away. */

static FILE *log_fp = NULL;

void logger_init(const char *filename) {
    if (filename) {
        log_fp = fopen(filename, "a");
    } else {
        log_fp = stderr;  /* stderr is already unbuffered by default */
    }
    if (!log_fp) { perror("log init"); return; }

    /* Disable buffering — each log_write() will immediately call write() */
    setvbuf(log_fp, NULL, _IONBF, 0);
}

void log_write(const char *level, const char *fmt, ...) {
    if (!log_fp) return;

    time_t now = time(NULL);
    struct tm *t = localtime(&now);
    fprintf(log_fp, "[%02d:%02d:%02d] [%s] ",
            t->tm_hour, t->tm_min, t->tm_sec, level);

    va_list args;
    va_start(args, fmt);
    vfprintf(log_fp, fmt, args);
    va_end(args);

    fputc('\n', log_fp);
    /* No fflush needed — _IONBF means each write is immediate */
}

int main(void) {
    logger_init("/tmp/app.log");
    log_write("INFO",  "System initialized");
    log_write("DEBUG", "Sensor value = %d", 42);
    log_write("ERROR", "BLE connection dropped, retrying...");
    /* Even if program crashes, all logs are safely in kernel buffer */
    return 0;
}

🎯 Interview Questions – stdio Buffering

Q1. What are the three buffering modes in stdio? What is the default for stdout?

The three modes are: _IONBF (no buffering — writes trigger write() immediately), _IOLBF (line buffering — flush on newline), and _IOFBF (full buffering — flush only when buffer fills). stdout defaults to _IOLBF when connected to a terminal, and _IOFBF when redirected to a file. stderr defaults to _IONBF.

Q2. You print “Enter name: ” with printf() but the prompt never appears. Why? How to fix?

Because printf() goes through the stdio buffer and there’s no newline to trigger a flush. The data sits in the stdio buffer. Fix: call fflush(stdout) immediately after the printf(). This forces the library to call write() and send the data to the kernel.

Q3. What happens if you pass a stack-allocated buffer to setvbuf()?

When the function that declared the stack variable returns, that memory is freed (the stack frame is deallocated). The stdio library still holds a pointer to that now-invalid memory. Any future I/O on that stream will read/write to garbage memory, causing undefined behavior — typically corruption or a crash. Always use static or heap-allocated buffers.

Q4. Does fflush() guarantee data is on disk?

No. fflush() moves data from the stdio user-space buffer to the kernel buffer cache by calling write(). Data is still in RAM, in the kernel. To guarantee it’s on disk, you must also call fsync(fileno(fp)) after fflush(fp).

Q5. Why must you call fflush(NULL) before fork()?

fork() duplicates the parent process, including its stdio buffers. Any data in the stdio buffer at the time of fork() will be in both parent and child. When each calls exit(), they both flush the buffer, producing duplicate output. Calling fflush(NULL) before fork() empties all buffers, preventing this duplication.

Q6. What does setvbuf() return and when does it fail?

setvbuf() returns 0 on success and a nonzero value on error (not necessarily -1, unlike most syscalls). It fails if called after I/O has already been done on the stream, or if an invalid mode is given. Always check its return value.

Q7. What is the value of BUFSIZ in glibc and what does it represent?

BUFSIZ is defined in stdio.h and is typically 8192 bytes in glibc. It represents the default buffer size used by setbuf() and the stdio library when it automatically allocates a buffer for a stream. It was historically chosen to be efficient for disk I/O on typical systems.

Q8. You mix printf() and write() on the same fd. Output is in wrong order. Why?

printf() stores data in the stdio buffer, while write() bypasses stdio and goes directly to the kernel. When stdout is fully buffered (e.g., redirected to a file), printf() data stays in the stdio buffer until flush, but write() output appears in the kernel buffer immediately. The write() output arrives at the kernel before the printf() buffer is flushed at program exit. Fix: call fflush(stdout) before every write() call on the same descriptor.

✅ Summary of Part 2

stdio has its own buffer before calling the kernel’s write(). This is Layer 1.
Three modes: _IONBF (no buffer), _IOLBF (line), _IOFBF (full).
stderr = unbuffered. stdout to terminal = line-buffered. stdout to file = fully buffered.
Use setvbuf() to change mode. Must call before any I/O on that stream.
fflush(fp) forces stdio → kernel. Use fflush(NULL) to flush everything.
Always fflush(NULL) before fork(). Use _exit() in the child.
fflush() alone does NOT put data on disk — use fsync() for that.

embeddedpathashala.com

stdio Library Buffering in Linux: Complete Developer Guide

📦 stdio Library Buffering

1. The Three Buffering Modes

_IONBF — No Buffering

_IOLBF — Line Buffering

_IOFBF — Full Buffering

Visual: Where stdio Buffer Sits

2. setvbuf() — Set Buffering Mode

Code Example: Using setvbuf()

3. setbuf() and setbuffer() — Simpler Wrappers

Code Example: setbuf() and setbuffer()

4. fflush() — Force Flush at Any Time

Code Example: fflush() Use Cases

5. Practical Scenarios and Gotchas

Scenario: stdout Behavior Changes When Redirected

Code Example: Correctly Mixing printf and write

Code Example: Embedded System — Unbuffered Serial Log

🎯 Interview Questions – stdio Buffering

✅ Summary of Part 2

Leave a Reply Cancel reply