sendfile() — Efficient File-to-Socket Transfer

 

sendfile() — Efficient File-to-Socket Transfer
Chapter 61 · Part 4 of 7 — Zero-copy file serving without touching user space

The Problem: Costly User-Space Copies

Imagine a web server that serves static files. The naive approach is: open the file, read() a chunk into a user-space buffer, then write() that buffer to the socket — then repeat until the file is done. This works, but it involves four memory copies for every chunk of data.

sendfile() is a Linux system call that transfers data directly from a file descriptor to a socket entirely inside the kernel, with no user-space copy at all. For servers that serve large files, this can dramatically reduce CPU usage and latency.

Why sendfile() Is Faster

Data Path Comparison
Naive read()+write() — 4 Copies, 2 System Calls Per Chunk
Disk
→ copy 1 →
Kernel
Page Cache
→ copy 2 →
User
Buffer
→ copy 3 →
Kernel
Socket Buffer
→ copy 4 →
NIC
sendfile() — 2 Copies, 1 System Call, No User-Space Involvement
Disk
→ copy 1 →
Kernel
Page Cache
→ copy 2 →
NIC
(via DMA)
User space
NOT involved

On modern hardware with DMA (Direct Memory Access) support, copy 2 can even happen directly from the page cache to the NIC without involving the CPU at all, making it truly “zero-copy” from the CPU’s perspective.

The sendfile() API

Function Signature
#include <sys/sendfile.h>

ssize_t sendfile(int out_fd,    /* Destination: must be a socket */
                 int in_fd,     /* Source: must be a file (mmap-able) */
                 off_t *offset, /* File offset to start from (or NULL) */
                 size_t count); /* Number of bytes to transfer */

/* Returns: number of bytes transferred on success, -1 on error */

Key constraints on Linux:

in_fd must be a regular file or block device that supports mmap() — you cannot use another socket as the source. out_fd must be a socket. If offset is not NULL, the file is read from that position and the pointed-to value is updated to reflect the new position. The file descriptor’s own position is not updated. If offset is NULL, reading starts from the file descriptor’s current position.

Basic Example

Sending a File to a Socket
#include <stdio.h>
#include <stdlib.h>
#include <sys/sendfile.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>

/*
 * send_file - Transfer a file to a socket using sendfile().
 *
 * sockfd  - connected socket to send to
 * path    - path to the file to send
 *
 * Returns 0 on success, -1 on error.
 */
int send_file(int sockfd, const char *path)
{
    int         file_fd;
    struct stat st;
    off_t       offset = 0;
    ssize_t     sent;
    size_t      remaining;

    file_fd = open(path, O_RDONLY);
    if (file_fd == -1) {
        perror("open");
        return -1;
    }

    /* Get file size */
    if (fstat(file_fd, &st) == -1) {
        perror("fstat");
        close(file_fd);
        return -1;
    }

    remaining = st.st_size;
    printf("Sending %zu bytes from '%s'\n", remaining, path);

    /*
     * sendfile() may not transfer all bytes in one call —
     * loop until the entire file is sent, just like writen().
     */
    while (remaining > 0) {
        sent = sendfile(sockfd, file_fd, &offset, remaining);
        if (sent == -1) {
            perror("sendfile");
            close(file_fd);
            return -1;
        }
        remaining -= sent;
        printf("Sent %zd bytes, %zu remaining\n", sent, remaining);
    }

    close(file_fd);
    return 0;
}

Notice that we loop on sendfile() just like we loop on write() — a single call may not transfer the entire file if the socket’s send buffer fills up. The offset variable is automatically updated by the kernel after each call, so we do not need to update it manually.

Mini HTTP File Server Using sendfile()

Here is a simplified HTTP/1.0 file server that uses sendfile() to serve static files efficiently:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/socket.h>
#include <sys/stat.h>
#include <sys/sendfile.h>
#include <netinet/in.h>

#define PORT    8080
#define BACKLOG 10

void handle_client(int cfd)
{
    char         req_buf[1024];
    char         path[256];
    char         resp_hdr[512];
    int          file_fd;
    struct stat  st;
    off_t        offset = 0;
    ssize_t      nr;

    /* Read the HTTP request (simplified — just get first line) */
    nr = read(cfd, req_buf, sizeof(req_buf) - 1);
    if (nr <= 0) { close(cfd); return; }
    req_buf[nr] = '\0';

    /* Parse "GET /filename HTTP/1.0" — extract filename */
    char method[8], url[256];
    sscanf(req_buf, "%7s %255s", method, url);

    /* Build file path — map "/" to "./index.html" */
    if (strcmp(url, "/") == 0)
        snprintf(path, sizeof(path), "./index.html");
    else
        snprintf(path, sizeof(path), ".%s", url);  /* e.g. "./style.css" */

    file_fd = open(path, O_RDONLY);
    if (file_fd == -1) {
        /* File not found — send 404 */
        const char *not_found = "HTTP/1.0 404 Not Found\r\n\r\n404 Not Found\n";
        write(cfd, not_found, strlen(not_found));
        close(cfd);
        return;
    }

    fstat(file_fd, &st);

    /* Send HTTP response headers */
    snprintf(resp_hdr, sizeof(resp_hdr),
             "HTTP/1.0 200 OK\r\n"
             "Content-Length: %lld\r\n"
             "\r\n",
             (long long)st.st_size);
    write(cfd, resp_hdr, strlen(resp_hdr));

    /* Send file body using sendfile() — no user-space copy */
    while (offset < st.st_size) {
        ssize_t sent = sendfile(cfd, file_fd, &offset,
                                st.st_size - offset);
        if (sent <= 0) break;
    }

    close(file_fd);
    close(cfd);
}

int main(void)
{
    int                lfd, cfd;
    struct sockaddr_in addr;
    int                opt = 1;

    lfd = socket(AF_INET, SOCK_STREAM, 0);
    setsockopt(lfd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));

    memset(&addr, 0, sizeof(addr));
    addr.sin_family      = AF_INET;
    addr.sin_port        = htons(PORT);
    addr.sin_addr.s_addr = INADDR_ANY;

    bind(lfd, (struct sockaddr *)&addr, sizeof(addr));
    listen(lfd, BACKLOG);
    printf("Serving files on port %d ...\n", PORT);

    for (;;) {
        cfd = accept(lfd, NULL, NULL);
        if (cfd >= 0) {
            handle_client(cfd);
        }
    }

    return 0;
}

Exercise 61-3: Implement sendfile() Using read()/write()/lseek()

If sendfile() were not available, you would implement it manually. Here is how:

#include <unistd.h>
#include <stdlib.h>

#define XFER_BUFSZ 65536  /* 64 KB transfer buffer */

/*
 * my_sendfile - Transfer 'count' bytes from 'in_fd' (starting at *offset)
 *               to 'out_fd'. Updates *offset like the real sendfile().
 *
 * Returns bytes sent on success, -1 on error.
 */
ssize_t my_sendfile(int out_fd, int in_fd, off_t *offset, size_t count)
{
    char    *buf;
    size_t   total_sent = 0;
    ssize_t  nr, nw;

    buf = malloc(XFER_BUFSZ);
    if (!buf) return -1;

    /* If offset given, seek to that position */
    if (offset != NULL) {
        if (lseek(in_fd, *offset, SEEK_SET) == -1) {
            free(buf);
            return -1;
        }
    }

    while (total_sent < count) {
        size_t to_read = count - total_sent;
        if (to_read > XFER_BUFSZ)
            to_read = XFER_BUFSZ;

        nr = read(in_fd, buf, to_read);
        if (nr == 0) break;           /* EOF */
        if (nr == -1) { free(buf); return -1; }

        /* Write all read bytes to output */
        size_t written = 0;
        while ((size_t)written < (size_t)nr) {
            nw = write(out_fd, buf + written, nr - written);
            if (nw <= 0) { free(buf); return -1; }
            written += nw;
        }

        total_sent += nr;
    }

    if (offset != NULL)
        *offset += total_sent;

    free(buf);
    return (ssize_t)total_sent;
}

This implementation requires a user-space buffer and copies data twice (file → buffer → socket), which is exactly what the real sendfile() avoids.

Interview Questions

Q1. What is zero-copy I/O and why is sendfile() considered zero-copy?

Answer: Zero-copy refers to transferring data without copying it into user-space memory. With read()/write(), data moves: disk → kernel page cache → user buffer → kernel socket buffer → NIC (4 copies, 2 involving user space). With sendfile(), data moves: disk → kernel page cache → NIC (kernel-to-kernel only, bypassing user space entirely). This is zero-copy from the application’s perspective.

Q2. What are the restrictions on sendfile() on Linux?

Answer: On Linux, in_fd must be a regular file or block device (something that can be mmap’d). out_fd must be a socket. You cannot use sendfile() to transfer from one socket to another (use splice() for that). Also, since Linux 2.6, the file does not need to be in a special location — any regular file works.

Q3. Should you call sendfile() in a loop? Why?

Answer: Yes. Like write(), a single sendfile() call may transfer fewer bytes than requested if the socket’s send buffer is full. You must loop until all bytes are transferred, checking the return value and using the offset parameter to track progress. The offset is automatically updated by the kernel after each successful call.

Q4. Why is sendfile() particularly beneficial for a web server?

Answer: A web server serving static files would otherwise spend most of its CPU time copying data between the kernel page cache and user-space buffers. With sendfile(), this copying is eliminated. The kernel can transfer file data directly to the NIC’s DMA engine. The CPU is freed for other work. This dramatically increases throughput and reduces latency for high-traffic file-serving scenarios.

Q5. What is the difference between sendfile() and mmap()+write()?

Answer: Both avoid a copy from the page cache to a user-space buffer. With mmap()+write(), you map the file into your address space and then write from it — the kernel still copies from your mapped pages to the socket buffer (2 copies total). With sendfile(), on hardware that supports scatter-gather DMA, there may be only 1 copy (page cache to NIC). Also, mmap() requires managing virtual memory mappings explicitly, while sendfile() is a single clean system call.

Leave a Reply

Your email address will not be published. Required fields are marked *