sendfile() – Zero-Copy File Transfer

 

sendfile() – Zero-Copy File Transfer
Chapter 61 · Part 1 of 4 · Sockets: Advanced Topics
0
User-Space Copies
1
Syscall Instead of 2
Linux
Specific API

The Problem – Why read() + write() is Wasteful

Imagine you are writing a simple file server. A client requests a file. You open the file, read its contents into a buffer, and then write that buffer to the socket. That is the classic approach and it looks like this in code:

/* Traditional approach – two system calls, two data copies */
int fd = open("page.html", O_RDONLY);
char buf[4096];
ssize_t n;

while ((n = read(fd, buf, sizeof(buf))) > 0) {
    write(sockfd, buf, n);   /* send to client */
}

This works, but it is not efficient. Here is what actually happens inside the kernel when you call read() followed by write():

Step Action Data Location Copy?
1 Kernel reads file from disk Disk → Kernel buffer cache DMA copy
2 read() copies to user buffer Kernel cache → User-space buf CPU copy ⚠️
3 write() copies to socket buffer User-space buf → Socket send buffer CPU copy ⚠️
4 Kernel transmits over network Socket buffer → NIC DMA copy

Steps 2 and 3 involve CPU-driven copies through user space. Your application never actually looks at the data – it just moves it. These copies burn CPU cycles and pollute CPU caches, especially for large files. This is the inefficiency that sendfile() was designed to fix.

The Solution – Zero-Copy with sendfile()

With sendfile(), the kernel transfers data from the file’s buffer cache directly to the socket’s send buffer without ever passing through user space. The application does not need a buffer at all.

Step Action Data Location Copy?
1 Kernel reads file from disk Disk → Kernel buffer cache DMA copy
2 sendfile() moves data kernel-side Kernel cache → Socket send buffer Kernel-only (no user copy) ✅
3 Kernel transmits over network Socket buffer → NIC DMA copy

The two wasteful CPU copies in the middle are gone. This is called a zero-copy transfer because no data passes through user space.

Key insight: On modern hardware with scatter-gather DMA, the kernel can even avoid copying data into the socket send buffer and instead pass a reference (file descriptor + offset + length) to the NIC directly. This makes sendfile() essentially free in terms of CPU for large file transfers.

The sendfile() API

#include <sys/sendfile.h>

ssize_t sendfile(int out_fd, int in_fd, off_t *offset, size_t count);

/* Returns: number of bytes transferred on success, -1 on error */

Let us go through each argument one by one:

Argument Type Meaning Restriction
out_fd int Destination – where data goes Must be a socket
in_fd int Source – where data comes from Must support mmap() (regular file)
offset off_t * File offset to start reading from NULL = use current file offset
count size_t Number of bytes to transfer May transfer fewer if EOF reached

Understanding the offset argument

The offset argument gives you two modes of operation:

offset = NULL

sendfile() reads from the current file offset of in_fd and updates the file offset after the transfer. Behaves like a regular read().

offset = &my_offset

sendfile() starts at the position you specify. After the call, my_offset is updated to the next byte position. The file’s own offset is not changed. Useful for concurrent access.

Code Examples

Example 1 – Basic sendfile() usage

#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/stat.h>
#include <sys/sendfile.h>
#include <sys/socket.h>

/*
 * send_file_to_socket - sends a file to a connected socket using zero-copy
 * file_path : path of file to send
 * sockfd    : connected TCP socket
 * Returns   : 0 on success, -1 on error
 */
int send_file_to_socket(const char *file_path, int sockfd)
{
    int       file_fd;
    struct stat file_stat;
    off_t     offset;
    ssize_t   sent;
    size_t    remaining;

    /* Open the file for reading */
    file_fd = open(file_path, O_RDONLY);
    if (file_fd == -1) {
        perror("open");
        return -1;
    }

    /* Get the file size */
    if (fstat(file_fd, &file_stat) == -1) {
        perror("fstat");
        close(file_fd);
        return -1;
    }

    offset    = 0;                   /* start from beginning of file */
    remaining = file_stat.st_size;   /* total bytes to send */

    /*
     * Loop because sendfile() may transfer fewer bytes than requested.
     * This can happen when the socket send buffer is temporarily full.
     */
    while (remaining > 0) {
        sent = sendfile(sockfd, file_fd, &offset, remaining);
        if (sent == -1) {
            perror("sendfile");
            close(file_fd);
            return -1;
        }
        remaining -= sent;
        /* offset is automatically updated by sendfile() */
    }

    close(file_fd);
    return 0;  /* all bytes sent */
}

Example 2 – Sending a file from a specific offset

#include <sys/sendfile.h>
#include <fcntl.h>
#include <stdio.h>

/*
 * Demonstrate offset-based sendfile().
 * Useful for sending a specific portion of a large file,
 * e.g. resumable downloads (HTTP Range requests).
 */
void send_range(int sockfd, int file_fd, off_t start, size_t length)
{
    off_t   offset = start;   /* sendfile() will update this */
    ssize_t sent;
    size_t  remaining = length;

    while (remaining > 0) {
        sent = sendfile(sockfd, file_fd, &offset, remaining);
        if (sent <= 0) {
            perror("sendfile range");
            return;
        }
        remaining -= sent;
        /*
         * After the call:
         *   offset == start + bytes_transferred_so_far
         * The file's own seek position is NOT changed.
         */
    }

    printf("Sent %zu bytes starting from offset %ld\n", length, (long)start);
}

Example 3 – A minimal file server using sendfile()

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/socket.h>
#include <sys/sendfile.h>
#include <netinet/in.h>

#define PORT 8080

int main(void)
{
    int                server_fd, client_fd, file_fd;
    struct sockaddr_in addr;
    struct stat        st;
    off_t              offset;
    char               filepath[] = "data.bin";

    /* Create TCP socket */
    server_fd = socket(AF_INET, SOCK_STREAM, 0);

    int opt = 1;
    setsockopt(server_fd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));

    memset(&addr, 0, sizeof(addr));
    addr.sin_family      = AF_INET;
    addr.sin_addr.s_addr = INADDR_ANY;
    addr.sin_port        = htons(PORT);

    bind(server_fd, (struct sockaddr *)&addr, sizeof(addr));
    listen(server_fd, 5);

    printf("File server listening on port %d...\n", PORT);

    while (1) {
        client_fd = accept(server_fd, NULL, NULL);
        if (client_fd == -1) { perror("accept"); continue; }

        file_fd = open(filepath, O_RDONLY);
        if (file_fd == -1) { perror("open"); close(client_fd); continue; }

        fstat(file_fd, &st);
        offset = 0;

        /* Send entire file in one call (or loop if partial) */
        ssize_t n = sendfile(client_fd, file_fd, &offset, st.st_size);
        if (n == -1) perror("sendfile");
        else         printf("Sent %zd bytes to client\n", n);

        close(file_fd);
        close(client_fd);
    }

    close(server_fd);
    return 0;
}

Important Restrictions

Restriction Detail
out_fd must be a socket You cannot use sendfile() to copy between two regular files (since kernel 2.6). On kernel 2.4 and earlier, out_fd could be a regular file – this was removed in 2.6.
in_fd must support mmap() in_fd must be a regular file or another file type that can be memory-mapped. You cannot use a socket as in_fd. So you cannot relay data from one socket to another with sendfile() alone.
Not in POSIX / SUSv3 sendfile() is Linux-specific. Other UNIX systems (Solaris, FreeBSD) have sendfile() but with a different argument list. Portable code should abstract this behind a wrapper.
Partial transfer possible sendfile() can return fewer bytes than requested (e.g., socket send buffer full). Always loop until all bytes are sent.

splice(), vmsplice(), and tee()

Starting with Linux kernel 2.6.17, three new system calls were added that are a superset of sendfile():

Syscall What it does Advantage over sendfile()
splice() Moves data between two file descriptors via a kernel pipe buffer Can transfer socket-to-socket, not just file-to-socket
vmsplice() Maps user-space memory into a pipe Zero-copy from user-space to pipe
tee() Duplicates data in a pipe without consuming it Fan-out to multiple destinations without extra copies
Note: For most embedded and server applications, sendfile() is still the simplest and most widely supported zero-copy API. Use splice() when you need socket-to-socket relay without a file involved.

Interview Questions & Answers

Q1. What is a zero-copy transfer and why is it faster than read() + write()?

In the traditional read+write path, file data is copied from the kernel buffer cache into a user-space buffer (by read()) and then from the user-space buffer into the socket send buffer (by write()). These are CPU-driven copies that consume memory bandwidth and CPU cache. A zero-copy transfer, as done by sendfile(), moves data entirely within the kernel from the file buffer cache directly to the socket send buffer, eliminating the two CPU copies and reducing system call overhead from two calls to one.

Q2. What are the restrictions on in_fd and out_fd in sendfile()?

out_fd must be a socket. in_fd must be a file descriptor that supports mmap(), which in practice means a regular file. You cannot pass a socket as in_fd, and since Linux 2.6 you cannot pass a regular file as out_fd. This means sendfile() can only be used for file-to-socket transfers, not socket-to-socket or file-to-file.

Q3. What does the offset argument do, and when would you set it to NULL?

When offset is a non-NULL pointer, sendfile() reads from in_fd starting at the position the pointer points to, updates that value after the call, and does not change the file descriptor’s own seek position. When offset is NULL, sendfile() uses and updates the file descriptor’s current seek position, just like read() does. You set offset to NULL when you want sequential, simple reading and do not care about the file offset separately. You use a non-NULL offset when implementing range requests or when multiple threads share the same file descriptor and need independent position tracking.

Q4. Can sendfile() transfer data from one socket to another?

No. in_fd must support mmap() and sockets do not. To relay data from one socket to another in a zero-copy manner, you should use splice() instead, which was introduced in Linux 2.6.17 and can work with any pair of file descriptors as long as at least one end is a pipe.

Q5. Why should you loop around sendfile() instead of calling it once?

sendfile() can return fewer bytes than the count you requested. This happens when the socket send buffer is temporarily full or when a signal interrupts the call. If you do not loop, you silently truncate the transfer and the receiver gets an incomplete file. The correct pattern is to decrement remaining by the return value and call sendfile() again with the updated offset until remaining reaches zero.

Q6. Is sendfile() portable across UNIX systems?

No. sendfile() is not defined by POSIX or SUSv3. Solaris and FreeBSD also have a sendfile() function but with a different argument list. If you are writing portable code, you should wrap the OS-specific call behind an abstraction function and use a fallback read+write loop on platforms that do not support it.

Q7. What was changed in kernel 2.6 regarding sendfile()?

In Linux 2.4 and earlier, out_fd could be either a socket or a regular file, allowing file-to-file zero-copy. The underlying implementation was reworked in the 2.6 kernel and this ability was removed. Since then, out_fd must be a socket. The feature may be reinstated in a future kernel version, but as of the time of writing it is not available.

Next: TCP_CORK – Batching TCP Segments

Learn how to combine sendfile() with TCP_CORK to send HTTP headers and file data in a single TCP segment.

Continue to Part 2 → Back to Index

Leave a Reply

Your email address will not be published. Required fields are marked *