sendfile() — Efficient File Transfer

 

Part 3: sendfile() — Efficient File Transfer
Chapter 61 · Sockets: Advanced Topics · TLPI

The Problem — Sending a File the Naive Way

Imagine you are writing a web server or a file server. A client requests a file. You open the file and write its contents to the socket so the client can download it. The natural way to do this is a read-then-write loop:

/* The naive approach — works, but inefficient for large files */
char buf[4096];
ssize_t n;

while ((n = read(diskfilefd, buf, sizeof(buf))) > 0) {
    write(sockfd, buf, n);
}

This works perfectly well for many applications. But if you are transferring large files frequently — think of a busy web server serving thousands of files per second — this approach has a serious performance cost.

Why Is the Read-Write Loop Slow?

Step What Happens Memory Location
1 Disk → Kernel buffer cache (page cache) Kernel space
2 Kernel buffer cache → User space buffer (read() copies data to buf[]) User space ← unnecessary copy
3 User space buffer → Socket send buffer (write() copies buf[] to kernel) Kernel space ← another unnecessary copy
4 Socket send buffer → Network card → Client Network

Steps 2 and 3 are the problem. Every time we loop, data travels from kernel to user space and back to kernel space again. This means:

2 extra memory copies
Data is copied from kernel to user space (read), then back from user space to kernel (write). For large files, this is a lot of wasted memory bandwidth.
2 system call transitions per chunk
Each read() and write() switches from user mode to kernel mode and back. Thousands of these per second adds up.
User space buffer is useless
The application never looks at the data in buf[]. We are just moving it from one place to another. There is no reason for it to ever leave the kernel.

The Solution — sendfile() Zero-Copy Transfer

sendfile() solves this by transferring data directly inside the kernel — from the file’s page cache to the socket’s send buffer — without ever copying it into user space. This is called a zero-copy transfer from the application’s perspective.

Step What Happens with sendfile() Memory Location
1 Disk → Kernel buffer cache (page cache) Kernel space
2 Kernel buffer cache → Socket send buffer (inside kernel — no user space involved) Kernel space only ✓
3 Socket send buffer → Network card → Client Network

Result: Data never touches user space. The two wasteful memory copies are eliminated. Only one system call is needed instead of the read-write loop with hundreds of system calls for a large file. For high-throughput servers, this makes a significant performance difference.

The sendfile() API

#include <sys/sendfile.h>

ssize_t sendfile(int out_fd, int in_fd, off_t *offset, size_t count);
/* Returns: number of bytes transferred, or -1 on error */

Argument Type Description
out_fd int The destination — must be a socket (on Linux; some systems allow files)
in_fd int The source — must be a regular file or block device (not a socket or pipe)
offset off_t * Pointer to the starting file offset. If NULL, uses and advances the file’s current position. If non-NULL, sendfile() reads from that offset and updates it — the file’s own offset is NOT changed.
count size_t Number of bytes to transfer

Basic Usage — Send an Entire File

#include <sys/sendfile.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>

/*
 * Send the entire contents of 'filename' through 'sockfd'.
 * Returns 0 on success, -1 on error.
 */
int send_file(int sockfd, const char *filename)
{
    int filefd;
    struct stat st;
    ssize_t sent;

    /* Open the file for reading */
    filefd = open(filename, O_RDONLY);
    if (filefd == -1) {
        perror("open");
        return -1;
    }

    /* Find out how large the file is */
    if (fstat(filefd, &st) == -1) {
        perror("fstat");
        close(filefd);
        return -1;
    }

    /*
     * sendfile():
     *   out_fd  = sockfd  (destination: the network socket)
     *   in_fd   = filefd  (source: the file on disk)
     *   offset  = NULL    (start from current file position, i.e., the beginning)
     *   count   = st.st_size (transfer the whole file)
     *
     * Data goes: disk → kernel page cache → socket send buffer
     * It NEVER passes through user space (zero-copy from app's perspective).
     */
    sent = sendfile(sockfd, filefd, NULL, st.st_size);
    if (sent == -1) {
        perror("sendfile");
        close(filefd);
        return -1;
    }

    printf("Sent %zd bytes from '%s'\n", sent, filename);
    close(filefd);
    return 0;
}

Using the offset Parameter — Send Part of a File

The offset parameter is powerful. It lets you send a portion of a file without seeking, and it does not disturb the file descriptor’s own position — which is important if another part of your program is also reading from that file.

#include <sys/sendfile.h>
#include <stdio.h>

/*
 * Send 'count' bytes from 'filefd' starting at byte 'start_offset'.
 * The file descriptor's own position is NOT changed.
 */
int send_file_range(int sockfd, int filefd, off_t start_offset, size_t count)
{
    off_t offset = start_offset;  /* sendfile will update this */
    ssize_t sent;

    /*
     * By passing &offset (non-NULL):
     * - sendfile() reads from 'offset' bytes into the file
     * - After the call, 'offset' is updated to the next byte position
     * - The file descriptor's own seek position is left unchanged
     */
    sent = sendfile(sockfd, filefd, &offset, count);
    if (sent == -1) {
        perror("sendfile");
        return -1;
    }

    printf("Sent %zd bytes starting from offset %ld\n",
           sent, (long)start_offset);
    printf("Next offset would be: %ld\n", (long)offset);
    return 0;
}

/* Example: Serve byte ranges for HTTP Range requests */
void serve_range_request(int sockfd, int filefd,
                         off_t range_start, off_t range_end)
{
    size_t bytes_to_send = (size_t)(range_end - range_start + 1);
    send_file_range(sockfd, filefd, range_start, bytes_to_send);
}

Handling Large Files — Loop Until Done

sendfile() may transfer fewer bytes than requested — for example, if the socket send buffer fills up temporarily. For large files, always loop until all bytes are sent:

#include <sys/sendfile.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>

int send_file_complete(int sockfd, const char *filename)
{
    int filefd;
    struct stat st;
    off_t offset = 0;
    ssize_t sent;
    size_t remaining;

    filefd = open(filename, O_RDONLY);
    if (filefd == -1) { perror("open"); return -1; }

    if (fstat(filefd, &st) == -1) {
        perror("fstat");
        close(filefd);
        return -1;
    }

    remaining = st.st_size;

    /*
     * Loop until all bytes are sent.
     * sendfile() updates 'offset' on each call, so we do not
     * need to track it separately.
     */
    while (remaining > 0) {
        sent = sendfile(sockfd, filefd, &offset, remaining);
        if (sent == -1) {
            perror("sendfile");
            close(filefd);
            return -1;
        }
        if (sent == 0)
            break;   /* unexpected EOF on source file */

        remaining -= sent;
        printf("Progress: %zu bytes remaining\n", remaining);
    }

    printf("File transfer complete: %lld bytes total\n",
           (long long)st.st_size);
    close(filefd);
    return 0;
}

Real-World Example — Minimal HTTP File Server

This is a simplified example of how a real web server might serve a static file. It combines HTTP header sending (with send()) and file body transfer (with sendfile()).

#include <sys/sendfile.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>
#include <string.h>

void serve_file(int client_sockfd, const char *filepath)
{
    int filefd;
    struct stat st;
    char http_header[512];
    int header_len;

    /* Open requested file */
    filefd = open(filepath, O_RDONLY);
    if (filefd == -1) {
        /* File not found — send 404 */
        const char *not_found =
            "HTTP/1.1 404 Not Found\r\n"
            "Content-Length: 0\r\n\r\n";
        send(client_sockfd, not_found, strlen(not_found), 0);
        return;
    }

    fstat(filefd, &st);

    /* Build HTTP response header */
    header_len = snprintf(http_header, sizeof(http_header),
        "HTTP/1.1 200 OK\r\n"
        "Content-Length: %lld\r\n"
        "Connection: close\r\n"
        "\r\n",
        (long long)st.st_size);

    /* Send header using regular send() */
    send(client_sockfd, http_header, header_len, 0);

    /*
     * Send file body using sendfile() — zero-copy.
     * No need to allocate a buffer or loop for read/write.
     * Data goes straight from kernel page cache to socket buffer.
     */
    off_t offset = 0;
    size_t remaining = st.st_size;

    while (remaining > 0) {
        ssize_t sent = sendfile(client_sockfd, filefd, &offset, remaining);
        if (sent <= 0) break;
        remaining -= sent;
    }

    close(filefd);
}

Limitations and Important Notes

in_fd must be a regular file
On Linux, in_fd (the source) must be a file descriptor that supports mmap()-like operations — typically a regular file or block device. You cannot use a socket or a pipe as the input to sendfile().
out_fd should be a socket
On Linux, out_fd (the destination) is typically a socket. Some other UNIX systems restrict this further. On Linux 2.6.33+, out_fd can also be a regular file, but this is less commonly needed.
May transfer less than count
Like write(), sendfile() may transfer fewer bytes than requested — for example if the socket buffer is full. Always check the return value and loop if needed.

read-write Loop vs sendfile() — Performance Comparison

Aspect read() + write() loop sendfile()
Memory copies 2 per chunk (kernel→user, user→kernel) 0 (stays in kernel)
System calls 2 per chunk (read + write), hundreds for large files 1 (or few iterations for very large files)
User space buffer needed? Yes — must allocate buf[] No
Can modify data in transit? Yes — you see data in buf[] No — data bypasses user space
Best for When you need to inspect or transform the data Serving static files unchanged (web server, FTP)
Portability POSIX — works everywhere Linux-specific (also on FreeBSD, macOS with different API)

Important — sendfile() and SSL/TLS:

sendfile() bypasses user space, so you cannot use it if the data needs to be encrypted before sending (e.g., HTTPS). SSL/TLS libraries (OpenSSL, GnuTLS) must encrypt the data in user space before it goes to the socket. For HTTPS servers, you must fall back to the read-then-encrypt-then-write approach. Some kernels support TLS offload that can work with sendfile-like semantics, but that is a specialized topic.

Interview Questions & Answers

Q1. What is the problem with using a read()-write() loop to send a file over a socket?
A: The main problem is unnecessary data copying. The data first moves from disk to the kernel’s page cache, then gets copied into a user-space buffer by read(), and then gets copied back into kernel space (the socket send buffer) by write(). For large or frequently-transferred files, these extra copies consume significant CPU time and memory bandwidth. Additionally, two system calls are needed per chunk — each involving a user-kernel mode switch.
Q2. What is zero-copy and how does sendfile() achieve it?
A: Zero-copy (from the application’s perspective) means data is transferred without being copied into user space. sendfile() tells the kernel to move data from the source file’s page cache directly to the socket’s send buffer, entirely within kernel space. The application never touches the data in a user-space buffer. This eliminates the two extra memory copies of the read-write loop and reduces the number of system calls to one (or a few for very large files).
Q3. What are the restrictions on in_fd and out_fd for sendfile() on Linux?
A: On Linux, in_fd (the source) must be a regular file or block device — it must support mmap-like access. You cannot use a socket or pipe as the source. out_fd (the destination) is typically a socket. Since Linux 2.6.33, out_fd can also be a regular file, but the most common use case is a socket for network file transfer.
Q4. What does the offset parameter in sendfile() do?
A: If offset is NULL, sendfile() reads from the file’s current position and advances it. If offset points to an off_t value, sendfile() reads from that position and updates the pointed-to value with the next offset after the transfer. Crucially, the file descriptor’s own seek position is NOT changed when a non-NULL offset is provided. This is useful for serving HTTP Range requests or sending different portions of a file concurrently from multiple threads.
Q5. Can sendfile() be used for HTTPS (SSL/TLS encrypted) connections?
A: No, not directly. sendfile() bypasses user space, meaning the application never sees the raw data. SSL/TLS encryption must happen in user space before the data reaches the socket. For HTTPS servers, you must decrypt the data, process it, and re-encrypt it in user space before writing to the socket — which rules out sendfile() in its standard form. Some advanced kernel-level TLS (kTLS) setups can work around this, but that is not standard usage.
Q6. Does sendfile() always transfer exactly count bytes?
A: No. Like write(), sendfile() may transfer fewer bytes than requested — for example, if the socket send buffer is temporarily full or the call is interrupted. The return value tells you how many bytes were actually sent. For large files you should loop: update the offset and remaining byte count on each iteration until all data is transferred.
Q7. When would you still use a read-write loop instead of sendfile()?
A: Use a read-write loop when you need to inspect or transform the data before sending — for example, compressing it on the fly, applying encryption, performing protocol translation, or injecting headers into the middle of a stream. Since sendfile() bypasses user space, you cannot see or modify the data during transfer. If the data must be processed, you need it in user space, which requires the traditional approach.

Free Embedded & Linux Tutorials

EmbeddedPathashala — Learn Linux System Programming, BLE, and Embedded Systems for free.

Visit EmbeddedPathashala

Leave a Reply

Your email address will not be published. Required fields are marked *