sendfile() — Zero-Copy File Transfer

 

Chapter 61 · Part 4 of 7
sendfile() — Zero-Copy File Transfer
Sending a file to a socket without copying data through user space

The Problem: Expensive File-to-Socket Transfer

Imagine writing a simple HTTP file server. The naive approach is: open the file, read() chunks into a user-space buffer, write() from that buffer to the socket. This works, but it involves multiple data copies and multiple context switches between kernel and user space.

sendfile() eliminates the user-space detour entirely. The kernel copies data directly from the file’s page cache to the socket’s send buffer — no trip through user space. This is called zero-copy I/O.

Traditional Approach vs sendfile()

Traditional: 4 copies, 4 switches
1. Disk → Kernel buffer
(DMA copy — kernel mode)
↓ context switch
2. Kernel buffer → User buffer
(CPU copy — user mode)
↓ context switch
3. User buffer → Socket buffer
(CPU copy — kernel mode)
↓ context switch
4. Socket buffer → NIC
(DMA copy — kernel mode)

sendfile(): 2 copies, 2 switches
1. Disk → Kernel page cache
(DMA copy — kernel mode)
↓ stays in kernel
2. Page cache → Socket buffer
(kernel-to-kernel copy)
↓ no user-space trip!
3. Socket buffer → NIC
(DMA copy — kernel mode)
No user-space buffer needed

The sendfile() API

#define _GNU_SOURCE
#include <sys/sendfile.h>

ssize_t sendfile(int out_fd, int in_fd, off_t *offset, size_t count);

/*
 * out_fd  — destination: must be a socket (on Linux)
 * in_fd   — source: must be a regular file or block device
 * offset  — pointer to file offset to start reading from.
 *            If NULL, uses in_fd's current file offset and advances it.
 *            If non-NULL, uses *offset and updates it; in_fd's position unchanged.
 * count   — number of bytes to transfer
 *
 * Returns: bytes transferred on success, -1 on error.
 * A return value less than count is possible (partial transfer).
 */
Linux restriction: On Linux, out_fd must be a socket. in_fd must be a regular file or block device that supports mmap()-like operations. You cannot use sendfile() to copy between two regular files on Linux (use copy_file_range() for that).

Basic Example: Sending a File over TCP

#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/sendfile.h>
#include <sys/socket.h>
#include <netinet/in.h>

#define PORT 8080

/*
 * Send the file at 'filepath' to the connected socket 'connfd'.
 * Uses sendfile() for zero-copy efficiency.
 */
int send_file_to_socket(int connfd, const char *filepath)
{
    int         filefd;
    struct stat st;
    off_t       offset;
    ssize_t     sent;
    size_t      remaining;

    /* Open file for reading */
    filefd = open(filepath, O_RDONLY);
    if (filefd == -1) {
        perror("open"); return -1;
    }

    /* Get file size */
    if (fstat(filefd, &st) == -1) {
        perror("fstat"); close(filefd); return -1;
    }

    printf("Sending file: %s (%ld bytes)\n", filepath, (long)st.st_size);

    offset    = 0;
    remaining = (size_t)st.st_size;

    /*
     * sendfile() may not transfer all bytes in one call
     * (similar to write() partial writes).
     * Loop until all bytes are sent.
     */
    while (remaining > 0) {
        sent = sendfile(connfd, filefd, &offset, remaining);

        if (sent == -1) {
            perror("sendfile");
            close(filefd);
            return -1;
        }

        remaining -= (size_t)sent;
        printf("Sent %zd bytes, %zu remaining\n", sent, remaining);
    }

    printf("File sent completely\n");
    close(filefd);
    return 0;
}

int main(void)
{
    int                listenfd, connfd;
    struct sockaddr_in servaddr;

    listenfd = socket(AF_INET, SOCK_STREAM, 0);
    if (listenfd == -1) { perror("socket"); exit(1); }

    int opt = 1;
    setsockopt(listenfd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));

    memset(&servaddr, 0, sizeof(servaddr));
    servaddr.sin_family      = AF_INET;
    servaddr.sin_addr.s_addr = INADDR_ANY;
    servaddr.sin_port        = htons(PORT);

    if (bind(listenfd, (struct sockaddr *)&servaddr, sizeof(servaddr)) == -1) {
        perror("bind"); exit(1);
    }
    listen(listenfd, 5);
    printf("File server listening on port %d\n", PORT);

    connfd = accept(listenfd, NULL, NULL);
    if (connfd == -1) { perror("accept"); exit(1); }
    printf("Client connected\n");

    /* Send a specific file — change to a real file on your system */
    send_file_to_socket(connfd, "/etc/passwd");

    close(connfd);
    close(listenfd);
    return 0;
}

Test with netcat:

gcc -o fileserver fileserver.c
./fileserver &
# Receive the file:
nc 127.0.0.1 8080 > received_file.txt
diff /etc/passwd received_file.txt   # should be identical

Realistic Example: Minimal HTTP File Server

This shows how sendfile() fits into a real HTTP server responding to GET requests.

#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/sendfile.h>
#include <sys/socket.h>
#include <netinet/in.h>

#define PORT      8080
#define DOCROOT   "/var/www/html"

void handle_client(int connfd)
{
    char    req[2048];
    char    method[16], path[256], version[16];
    char    filepath[512];
    char    header[512];
    int     filefd;
    struct  stat st;
    off_t   offset;
    size_t  remaining;
    ssize_t n;

    /* Read HTTP request (simplified — just first line) */
    n = recv(connfd, req, sizeof(req) - 1, 0);
    if (n <= 0) return;
    req[n] = '\0';

    sscanf(req, "%15s %255s %15s", method, path, version);
    printf("Request: %s %s\n", method, path);

    /* Build file path */
    snprintf(filepath, sizeof(filepath), "%s%s",
             DOCROOT, strcmp(path, "/") == 0 ? "/index.html" : path);

    filefd = open(filepath, O_RDONLY);
    if (filefd == -1) {
        /* 404 response */
        const char *r404 =
            "HTTP/1.1 404 Not Found\r\n"
            "Content-Length: 9\r\n\r\nNot Found";
        send(connfd, r404, strlen(r404), MSG_NOSIGNAL);
        return;
    }

    fstat(filefd, &st);

    /* Send HTTP 200 header */
    snprintf(header, sizeof(header),
             "HTTP/1.1 200 OK\r\n"
             "Content-Length: %ld\r\n"
             "Connection: close\r\n\r\n",
             (long)st.st_size);
    send(connfd, header, strlen(header), MSG_NOSIGNAL);

    /* Send file body using zero-copy sendfile() */
    offset    = 0;
    remaining = (size_t)st.st_size;

    while (remaining > 0) {
        ssize_t sent = sendfile(connfd, filefd, &offset, remaining);
        if (sent <= 0) break;
        remaining -= (size_t)sent;
    }

    close(filefd);
}

int main(void)
{
    int                listenfd, connfd;
    struct sockaddr_in servaddr;

    listenfd = socket(AF_INET, SOCK_STREAM, 0);
    int opt = 1;
    setsockopt(listenfd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));

    memset(&servaddr, 0, sizeof(servaddr));
    servaddr.sin_family      = AF_INET;
    servaddr.sin_addr.s_addr = INADDR_ANY;
    servaddr.sin_port        = htons(PORT);

    bind(listenfd, (struct sockaddr *)&servaddr, sizeof(servaddr));
    listen(listenfd, 10);
    printf("HTTP server at http://127.0.0.1:%d/\n", PORT);

    for (;;) {
        connfd = accept(listenfd, NULL, NULL);
        if (connfd == -1) { perror("accept"); continue; }
        handle_client(connfd);
        close(connfd);
    }
}

Using the offset Parameter

The offset parameter is powerful. When non-NULL, sendfile() reads from that position in the file, and updates the value pointed to by offset after the transfer. The file’s own seek position (lseek position) is NOT changed. This is ideal for serving byte-range requests (HTTP Range: headers).

/*
 * Serve a byte range from a file (for HTTP Range: requests).
 * Example: Range: bytes=1000-2999  (2000 bytes starting at offset 1000)
 */
int serve_range(int connfd, int filefd, off_t start, size_t length)
{
    off_t  offset    = start;   /* sendfile updates this after each call */
    size_t remaining = length;

    while (remaining > 0) {
        ssize_t sent = sendfile(connfd, filefd, &offset, remaining);
        if (sent == -1) { perror("sendfile"); return -1; }
        if (sent == 0)  break;   /* shouldn't happen but guard anyway */
        remaining -= (size_t)sent;
    }

    printf("Served %zu bytes starting at offset %lld\n",
           length, (long long)start);

    /*
     * filefd's lseek position is UNCHANGED by sendfile()
     * when offset is non-NULL. We can seek it independently.
     */
    return 0;
}

Interview Questions & Answers

Q1. What problem does sendfile() solve?
Serving a file over a socket normally requires: read() into a user-space buffer, then write() from that buffer to the socket. This involves two extra copies through user space and multiple kernel/user context switches. sendfile() lets the kernel transfer data directly from the file’s page cache to the socket buffer, eliminating the user-space buffer entirely.
Q2. What are the restrictions on sendfile() on Linux?
On Linux, the output fd (out_fd) must be a socket. The input fd (in_fd) must be a regular file or block device (it needs to support mmap-like kernel operations). You cannot use sendfile() to copy between two regular files on Linux — use copy_file_range() for that.
Q3. Can sendfile() return before sending all requested bytes?
Yes — just like write(), sendfile() can perform a partial transfer. The offset parameter is updated to reflect how far it got. Always check the return value and loop until remaining bytes reach zero.
Q4. What happens to the file descriptor’s seek position when you use sendfile() with a non-NULL offset?
When offset is non-NULL, sendfile() reads from the position given by *offset and updates *offset afterwards. The file descriptor’s own lseek position is NOT changed. This allows multiple concurrent goroutines/threads to serve different byte ranges from the same file descriptor safely.
Q5. How much performance improvement can sendfile() give?
It depends on file size and system load. In benchmarks serving static files over loopback, sendfile() typically reduces CPU usage by 30–50% compared to read+write loops, because it eliminates two buffer copies and two user/kernel context switches per chunk. For large files, the gain is significant. For tiny files or already-cached data, the difference is smaller.
Q6. How would you implement HTTP byte-range serving with sendfile()?
Parse the Range: header to get start and end byte positions. Open the file, call sendfile() with offset=start and count=(end-start+1). sendfile() will transfer exactly that range without seeking the fd. Send HTTP 206 Partial Content with Content-Range header before the sendfile() call.

Leave a Reply

Your email address will not be published. Required fields are marked *