Normally, if no data is available in the socket receive buffer, recv() blocks and waits. MSG_DONTWAIT changes that behavior for just this one call: if no data is ready, it returns immediately with an error instead of blocking.

When there is no data and MSG_DONTWAIT is set, recv() returns -1 and sets errno to EAGAIN (or EWOULDBLOCK, which is the same value on Linux).

How is this different from O_NONBLOCK? Setting O_NONBLOCK via fcntl() makes the socket permanently non-blocking — every subsequent call behaves this way. MSG_DONTWAIT is per-call — it only affects that single recv(). This is handy when you want to do an occasional non-blocking check without changing the socket’s global mode.

#include <sys/socket.h>
#include <errno.h>
#include <stdio.h>

void try_nonblocking_recv(int sockfd)
{
    char buf[256];
    ssize_t n;

    n = recv(sockfd, buf, sizeof(buf), MSG_DONTWAIT);

    if (n == -1) {
        if (errno == EAGAIN || errno == EWOULDBLOCK) {
            /* No data available right now — that is OK */
            printf("No data ready yet, come back later\n");
        } else {
            /* A real error occurred */
            perror("recv");
        }
    } else if (n == 0) {
        printf("Connection closed by peer\n");
    } else {
        printf("Got %zd bytes\n", n);
        /* process buf[0..n-1] */
    }
}

Normally, when you call recv(), the bytes are removed from the socket receive buffer. With MSG_PEEK, you get a copy of the bytes but they stay in the buffer. The next call to recv() (with or without MSG_PEEK) will see the same data again.

When is this useful? Imagine you need to look at the first few bytes of a message to decide what protocol it is (e.g., HTTP vs. some custom binary protocol), but you do not want to consume those bytes before passing them to the right handler. MSG_PEEK lets you inspect without destroying.

#include <sys/socket.h>
#include <stdio.h>
#include <string.h>

void peek_and_decide(int sockfd)
{
    char header[4];
    ssize_t n;

    /* Peek at the first 4 bytes to determine message type */
    n = recv(sockfd, header, sizeof(header), MSG_PEEK);
    if (n <= 0) {
        perror("recv peek");
        return;
    }

    if (memcmp(header, "HTTP", 4) == 0) {
        printf("HTTP request detected\n");
        handle_http(sockfd);   /* will re-read including these 4 bytes */
    } else {
        printf("Custom protocol detected\n");
        handle_custom(sockfd); /* will re-read including these 4 bytes */
    }
}

/* Both handlers can now do a normal recv() and see the full message
   including the 4 bytes that were peeked */

Normally, recv() returns as soon as any data is available — even if it is just 1 byte when you asked for 100. This is correct behavior for a stream socket because data can arrive in fragments. But often you know exactly how many bytes you expect (for example, a fixed-size header or struct), and you want to block until all of them arrive.

MSG_WAITALL tells the kernel: “do not return until you have filled my entire buffer.” The call will keep blocking and accumulating data until length bytes are received.

Exceptions — MSG_WAITALL may still return fewer bytes if:

• A signal interrupts the call
• The peer closes the connection (you get whatever arrived before the close)
• An out-of-band data byte is encountered
• The datagram received is smaller than length
• A socket error occurs

Always check the return value even with MSG_WAITALL.

#include <sys/socket.h>
#include <stdio.h>
#include <stdint.h>

/* A fixed-size message header for a custom protocol */
struct msg_header {
    uint32_t msg_type;
    uint32_t payload_len;
};

int recv_full_header(int sockfd, struct msg_header *hdr)
{
    ssize_t n;

    /*
     * MSG_WAITALL: block until ALL sizeof(*hdr) bytes have arrived.
     * Without this flag, we might get a partial header, which is hard
     * to handle correctly.
     */
    n = recv(sockfd, hdr, sizeof(*hdr), MSG_WAITALL);

    if (n == 0) {
        printf("Connection closed by peer\n");
        return -1;
    }
    if (n == -1) {
        perror("recv");
        return -1;
    }
    if (n < (ssize_t)sizeof(*hdr)) {
        /* Fewer bytes than expected — connection was interrupted */
        fprintf(stderr, "Partial header: got %zd of %zu bytes\n",
                n, sizeof(*hdr));
        return -1;
    }

    return 0;  /* success */
}

TCP supports a concept called out-of-band (OOB) data — also called urgent data. This is a way to send a special high-priority byte that jumps ahead of the normal data stream. A classic use was the telnet protocol sending interrupt commands without waiting for normal data to be consumed.

To receive OOB data, you pass MSG_OOB to recv(). This is an advanced and rarely-used feature in modern applications. The full details are in Section 61.13.1 of TLPI.

char oob_byte;
ssize_t n = recv(sockfd, &oob_byte, 1, MSG_OOB);
if (n == 1)
    printf("Received OOB byte: 0x%02x\n", (unsigned char)oob_byte);

Just like for recv(), MSG_DONTWAIT on send() makes that single send call non-blocking. If the socket’s send buffer is full (the kernel cannot accept more data right now because the network is backed up), instead of blocking until space is available, send() returns -1 with errno set to EAGAIN.

#include <sys/socket.h>
#include <errno.h>
#include <string.h>

int try_send(int sockfd, const char *data, size_t len)
{
    ssize_t n = send(sockfd, data, len, MSG_DONTWAIT);

    if (n == -1) {
        if (errno == EAGAIN || errno == EWOULDBLOCK) {
            /* Send buffer full — try again later */
            printf("Send buffer full, will retry\n");
            return 0;  /* caller should retry */
        }
        perror("send");
        return -1;
    }

    printf("Sent %zd bytes\n", n);
    return n;
}

When you send small chunks of data using multiple send() calls, TCP may send each chunk as a separate small packet. This is wasteful — it is more efficient to collect several small pieces and send them together in one larger packet.

TCP_CORK is a socket option that does this globally — it accumulates data until you un-cork the socket. MSG_MORE achieves the same effect but per-call, without permanently changing the socket option.

When you set MSG_MORE on a send() call, the kernel holds the data in the send buffer waiting for more. When you make a send() call without MSG_MORE, all the buffered data is flushed as one packet.

Since Linux 2.6, MSG_MORE on UDP datagram sockets also works — successive sends with MSG_MORE accumulate into a single UDP datagram, transmitted on the next call without the flag.

MSG_MORE has no effect on UNIX domain sockets.

#include <sys/socket.h>

void send_http_response(int sockfd)
{
    const char *header   = "HTTP/1.1 200 OK\r\nContent-Length: 13\r\n\r\n";
    const char *body     = "Hello, World!";

    /*
     * Send the header with MSG_MORE — tell kernel: "more data is coming,
     * hold this in the buffer for now"
     */
    send(sockfd, header, strlen(header), MSG_MORE);

    /*
     * Send the body WITHOUT MSG_MORE — this flushes the buffer.
     * Both header and body will be sent together in (ideally) one packet.
     */
    send(sockfd, body, strlen(body), 0);
}

When you try to send() data on a connected stream socket but the peer has already closed their end, two things normally happen:

• The kernel delivers a SIGPIPE signal to your process
• The send() call fails with errno = EPIPE

The default action for SIGPIPE is to terminate the process. This catches many programmers off guard — their server crashes silently because a client disconnected.

You have two options to handle this:

Option 1 — Global: Ignore SIGPIPE for the whole process with signal(SIGPIPE, SIG_IGN). Then every failed send() just returns -1 / EPIPE which you can handle.

Option 2 — Per call: Use MSG_NOSIGNAL on each send(). No SIGPIPE is generated for that call — you just get -1 / EPIPE as an error return.

#include <sys/socket.h>
#include <errno.h>
#include <stdio.h>

int safe_send(int sockfd, const void *buf, size_t len)
{
    ssize_t n;

    /*
     * MSG_NOSIGNAL: if the peer has closed, do NOT send SIGPIPE.
     * Just return -1 with errno = EPIPE so we can handle it gracefully.
     */
    n = send(sockfd, buf, len, MSG_NOSIGNAL);

    if (n == -1) {
        if (errno == EPIPE) {
            /* Peer closed their end — clean up gracefully */
            printf("Client disconnected\n");
            return -1;
        }
        perror("send");
        return -1;
    }

    return (int)n;
}

MSG_OOB in send() is the sending counterpart to MSG_OOB in recv(). It sends data as TCP urgent/OOB data. The remote side can receive it with recv(sockfd, buf, 1, MSG_OOB). This is a niche feature, mainly used by legacy protocols like telnet.

char urgent = 0xFF;
send(sockfd, &urgent, 1, MSG_OOB);