What is MSG_MORE?

MSG_MORE is a flag you pass to the send() system call. It tells the kernel: “I have more data coming after this call, so please do not transmit a segment yet – wait for the next send() call.”

It achieves the same goal as TCP_CORK – batching multiple writes into fewer TCP segments – but at the level of individual system calls rather than at the socket level.

#include <sys/socket.h>

/*
 * send() with MSG_MORE:
 * flags = MSG_MORE means "do not send yet, more data is coming"
 * flags = 0        means "this is the last piece, go ahead and send"
 */
ssize_t send(int sockfd, const void *buf, size_t len, int flags);

Simple Example

#include <stdio.h>
#include <string.h>
#include <sys/socket.h>

/*
 * Send HTTP headers and body using MSG_MORE.
 * The kernel batches them together into one TCP segment.
 */
void send_response_msg_more(int sockfd,
                            const char *headers, size_t hlen,
                            const char *body,    size_t blen)
{
    /* Tell the kernel: more data is coming after these headers */
    ssize_t n = send(sockfd, headers, hlen, MSG_MORE);
    if (n == -1) {
        perror("send headers MSG_MORE");
        return;
    }

    /* Last piece – no MSG_MORE flag, kernel flushes now */
    n = send(sockfd, body, blen, 0);
    if (n == -1) {
        perror("send body");
        return;
    }
}

MSG_MORE vs TCP_CORK – Detailed Comparison

The exec() Inheritance Advantage of TCP_CORK

This is a subtle but important difference. Consider the following server architecture:

/* Parent server – sets up the socket, then exec()s a child handler */
int optval = 1;
setsockopt(sockfd, IPPROTO_TCP, TCP_CORK, &optval, sizeof(optval));

/*
 * The child program inherits sockfd AND the TCP_CORK option.
 * The child does not know about TCP_CORK but benefits from it.
 * All its write() calls are buffered until the parent uncorks,
 * or until the child's socket is closed.
 */
execl("/usr/lib/cgi-bin/handler", "handler", NULL);

With TCP_CORK: The parent sets the socket option. The child exec’d program inherits the socket file descriptor and the socket option is still active. The child’s output is buffered without any changes to its source code.

With MSG_MORE: The child would need to be modified to use send(..., MSG_MORE) instead of write() on every call where batching is desired. You cannot inject MSG_MORE into an existing program without changing its code.

Alternative: writev() – Gather Write

Before MSG_MORE and TCP_CORK existed, a common technique was to assemble all data into a buffer and call write() once, or to use writev() which takes a vector of buffers and writes them all in one system call.

#include <sys/uio.h>

/*
 * writev() – scatter/gather write: combine multiple buffers in one syscall.
 * This avoids separate TCP segments without needing TCP_CORK or MSG_MORE.
 * Limitation: does not give zero-copy for file data like sendfile() does.
 */
ssize_t writev(int fd, const struct iovec *iov, int iovcnt);

/* Example: send headers + body together using writev() */
void send_with_writev(int sockfd,
                      const char *headers, size_t hlen,
                      const char *body,    size_t blen)
{
    struct iovec iov[2];

    iov[0].iov_base = (void *)headers;
    iov[0].iov_len  = hlen;

    iov[1].iov_base = (void *)body;
    iov[1].iov_len  = blen;

    ssize_t n = writev(sockfd, iov, 2);
    if (n == -1)
        perror("writev");
    else
        printf("Sent %zd bytes in one writev() call\n", n);
}

The combination of TCP_CORK + sendfile() is the most powerful approach: you get both zero-copy file transfer and TCP segment batching. Use writev() when you have all data in memory and do not need sendfile().

MSG_MORE – Multi-Part Send Example

#include <stdio.h>
#include <string.h>
#include <sys/socket.h>

/*
 * Demonstrates MSG_MORE for a multi-part message.
 * Parts 1..N-1 use MSG_MORE, last part uses flags=0.
 *
 * Think of it like building a train:
 *   MSG_MORE = "attach another car, train is not leaving yet"
 *   flags=0  = "train departs now"
 */
int send_multipart(int sockfd, const char **parts, size_t *lens, int count)
{
    int i;
    ssize_t sent;

    for (i = 0; i < count; i++) {
        int flags = (i < count - 1) ? MSG_MORE : 0;

        sent = send(sockfd, parts[i], lens[i], flags);
        if (sent == -1) {
            perror("send multipart");
            return -1;
        }
        if ((size_t)sent < lens[i]) {
            /* Partial send – need to loop (simplified here) */
            fprintf(stderr, "Partial send on part %d\n", i);
            return -1;
        }
    }
    return 0;
}

int main(void)
{
    /* Usage example (assumes sockfd is a connected TCP socket) */
    int sockfd = /* ... connect to server ... */ 0;

    const char *parts[] = {
        "GET / HTTP/1.1\r\n",
        "Host: example.com\r\n",
        "Connection: close\r\n",
        "\r\n"                   /* blank line ends HTTP request */
    };
    size_t lens[] = {
        strlen(parts[0]),
        strlen(parts[1]),
        strlen(parts[2]),
        strlen(parts[3])
    };

    /*
     * First 3 parts sent with MSG_MORE (not transmitted yet).
     * 4th part sent with flags=0 (flush – all 4 parts go in one segment).
     */
    send_multipart(sockfd, parts, lens, 4);
    return 0;
}

Interview Questions & Answers