TCP Congestion Control Sockets: Fundamentals of TCP/IP Networks

 

TCP Congestion Control
Chapter 58 | Sockets: Fundamentals of TCP/IP Networks โ€” Part 5 of 5
๐Ÿ“š TLPI Ch.58
๐Ÿ“ˆ Slow Start
๐Ÿšซ Congestion Avoidance

What is Congestion Control?

Flow control protects the receiver. But what about the network itself? A fast TCP sender can flood an intermediate router, causing it to drop packets. If every TCP sender then retransmits at the same high rate, the network gets worse, not better โ€” a scenario called congestion collapse.

TCP’s congestion control algorithms prevent this. They work at the sender, reducing the transmission rate when congestion is detected, and increasing it gradually when the network is healthy. TCP uses two algorithms together: Slow Start and Congestion Avoidance.

Key Terms in This Section

Congestion Window (cwnd) Slow Start Congestion Avoidance ssthresh Packet Loss = Congestion Equilibrium Send Buffer

โš  Why the Network Needs Protection Too

Consider what happens without congestion control: a new TCP connection has a full receiver window (say 64 KB). Without any limit, the sender immediately injects 64 KB of data into the network. If there are 100 simultaneous connections doing the same thing, the router in the middle is suddenly flooded with megabytes of data it cannot forward fast enough.

The router drops packets. The senders all retransmit. The router drops those too. The network enters congestion collapse โ€” it is busy but delivering almost nothing.

Without Congestion Control
Sender 1 โžก
Sender 2 โžก
Sender 3 โžก
โ†’โ†’โ†’
Router OVERLOADED
โ†’
โŒ Dropped packets

TCP detects congestion through packet loss. The assumption is: on modern networks, transmission errors (bit flips) are rare. So if a segment is lost, the most likely cause is a router dropping it due to congestion.

๐Ÿ’ฐ The Congestion Window (cwnd)

The sender maintains a congestion window (cwnd) โ€” a limit on how many bytes can be in-flight at once, independent of the receiver’s advertised window.

The actual amount of data the sender can have unacknowledged is:

effective_window = min(cwnd, receiver_advertised_window)

Both limits apply: the sender respects the receiver’s buffer (flow control) and also self-limits through cwnd (congestion control). The congestion control algorithms manage how cwnd grows and shrinks.

๐Ÿ“ˆ Phase 1: Slow Start

Despite its name, slow start is actually exponential growth. The sender starts with a small cwnd (typically 1โ€“10 segments) and doubles it with every RTT as long as no congestion is detected. This quickly probes the network’s capacity without immediately flooding it.

Slow Start: Exponential cwnd Growth
RTT 1
cwnd=1
RTT 2
cwnd=2
RTT 3
cwnd=4
RTT 4
cwnd=8
ssthresh
RTT 5
linear now
โ–  Slow Start (exponential) โ–  Hits ssthresh โ–  Congestion Avoidance (linear)

Slow start runs until cwnd reaches the slow-start threshold (ssthresh). After that, TCP switches to congestion avoidance mode (linear growth).

Initial ssthresh is typically set to a very large value (so slow start runs for a while on a fresh connection). After the first congestion event, ssthresh is set based on the detected network capacity.

๐Ÿšซ Phase 2: Congestion Avoidance

Once cwnd reaches ssthresh, the sender shifts to congestion avoidance: instead of doubling cwnd every RTT, it increases cwnd by 1 MSS per RTT (linear growth). This cautious growth continues until congestion is detected.

When congestion is detected (a segment is lost):

  • ssthresh is set to half of the current cwnd (estimated network capacity)
  • cwnd is reset (back to 1, or to ssthresh depending on TCP variant)
  • The cycle repeats from slow start or from ssthresh

TCP Congestion Window Lifecycle (Sawtooth Pattern)

ssthresh

Loss!

Loss!

SS
CA (linear)
SS
CA
SS = Slow Start ย |ย  CA = Congestion Avoidance ย |ย  Red = packet loss event

โš– Reaching Equilibrium

Together, slow start and congestion avoidance allow TCP to:

  • Quickly ramp up to available bandwidth (slow start’s exponential growth)
  • Back off when the network signals congestion (packet loss)
  • Reach equilibrium โ€” a steady state where the sender transmits at roughly the same rate as the network can carry, without significant queuing

The actual amount of data the sender can inject at any time is limited by:

bytes_in_flight โ‰ค min(cwnd, receiver_window, send_buffer)

All three limits apply simultaneously. The most restrictive one wins at any given moment.

Congestion Window
cwnd
Sender-maintained
Protects network
min()
Receiver Window
rwnd
Receiver-advertised
Protects receiver buffer
min()
Send Buffer
SO_SNDBUF
Local socket limit
App write buffer
The smallest of these three is the actual limit on bytes in-flight at any moment

๐Ÿ’ป Code Example: Observing Congestion Window via TCP_INFO
/* tcp_cwnd_monitor.c - monitor congestion window in real time */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netinet/tcp.h>
#include <arpa/inet.h>

#define SERVER_IP   "127.0.0.1"
#define SERVER_PORT 8080
#define CHUNK_SIZE  (4 * 1024)        /* 4 KB per write */
#define TOTAL_BYTES (4 * 1024 * 1024) /* send 4 MB total */

void print_cwnd(int sockfd, int iteration)
{
    struct tcp_info ti;
    socklen_t len = sizeof(ti);

    if (getsockopt(sockfd, IPPROTO_TCP, TCP_INFO, &ti, &len) == 0) {
        printf("[%3d] cwnd=%4u segs | ssthresh=%4u | rtt=%5u us | "
               "retrans=%u\n",
               iteration,
               ti.tcpi_snd_cwnd,      /* congestion window in segments */
               ti.tcpi_snd_ssthresh,  /* slow-start threshold */
               ti.tcpi_rtt,           /* smoothed RTT in microseconds */
               ti.tcpi_retransmits);  /* current retransmission count */
    }
}

int main(void)
{
    int sockfd;
    struct sockaddr_in srv;
    char buf[CHUNK_SIZE];
    int sent = 0, iter = 0;

    sockfd = socket(AF_INET, SOCK_STREAM, 0);
    if (sockfd == -1) { perror("socket"); exit(EXIT_FAILURE); }

    memset(&srv, 0, sizeof(srv));
    srv.sin_family      = AF_INET;
    srv.sin_port        = htons(SERVER_PORT);
    srv.sin_addr.s_addr = inet_addr(SERVER_IP);

    if (connect(sockfd, (struct sockaddr *)&srv, sizeof(srv)) == -1) {
        perror("connect"); exit(EXIT_FAILURE);
    }

    memset(buf, 'A', sizeof(buf));

    printf("%-5s %-10s %-12s %-12s %s\n",
           "Iter", "cwnd(seg)", "ssthresh", "RTT(us)", "retrans");
    printf("-------------------------------------------\n");

    while (sent < TOTAL_BYTES) {
        ssize_t n = write(sockfd, buf, CHUNK_SIZE);
        if (n <= 0) { perror("write"); break; }
        sent += n;

        /* Print cwnd every 10 writes */
        if (++iter % 10 == 0) {
            print_cwnd(sockfd, iter);
        }
    }

    printf("\nDone. Total sent: %d bytes\n", sent);
    close(sockfd);
    return 0;
}
/* Checking TCP congestion algorithm used by the kernel */
#include <stdio.h>
#include <sys/socket.h>
#include <netinet/tcp.h>
#include <unistd.h>
#include <string.h>

int main(void)
{
    int sockfd = socket(AF_INET, SOCK_STREAM, 0);
    char ca_name[16] = {0};
    socklen_t len = sizeof(ca_name);

    /*
     * TCP_CONGESTION: get or set the congestion control algorithm
     * Common Linux algorithms: cubic (default), reno, bbr, htcp
     */
    if (getsockopt(sockfd, IPPROTO_TCP, TCP_CONGESTION,
                   ca_name, &len) == 0) {
        printf("Current congestion algorithm: %s\n", ca_name);
    }

    /* Switch to BBR (Bottleneck Bandwidth and RTT) if available */
    const char *bbr = "bbr";
    if (setsockopt(sockfd, IPPROTO_TCP, TCP_CONGESTION,
                   bbr, strlen(bbr)) == 0) {
        printf("Switched to BBR congestion control\n");
    } else {
        perror("setsockopt TCP_CONGESTION bbr");
        printf("BBR may not be available; using: %s\n", ca_name);
    }

    close(sockfd);
    return 0;
}
# Check system-wide TCP congestion control setting
cat /proc/sys/net/ipv4/tcp_congestion_control

# List all available algorithms
cat /proc/sys/net/ipv4/tcp_available_congestion_control

# Change system default (requires root)
echo bbr > /proc/sys/net/ipv4/tcp_congestion_control

# Enable BBR (needs kernel 4.9+)
modprobe tcp_bbr
echo tcp_bbr >> /etc/modules-load.d/modules.conf

๐Ÿ“š Important RFCs for TCP

Each Internet protocol is defined in an RFC (Request for Comments) document. RFCs are published by the RFC Editor and developed under the IETF (Internet Engineering Task Force). Here are the most important ones for TCP/IP:

RFC 791
Internet Protocol (IP) โ€” Postel, 1981
RFC 793
Transmission Control Protocol (TCP) โ€” Postel, 1981. The foundational TCP spec.
RFC 768
User Datagram Protocol (UDP) โ€” Postel, 1980
RFC 2581
TCP Congestion Control โ€” Allman, Paxson, Stevens, 1999
RFC 1323
TCP Extensions for High Performance โ€” window scaling, timestamps
RFC 2018
TCP Selective Acknowledgment Options (SACK)
RFC 3168
Explicit Congestion Notification (ECN) โ€” router signals congestion without dropping
RFC 6298
Computing TCP’s Retransmission Timer (RTO algorithm)

๐Ÿ…พ Interview Questions โ€” Congestion Control
Q1. What is the difference between flow control and congestion control?

Flow control prevents the sender from overwhelming the receiver’s buffer โ€” it is endpoint-to-endpoint. Congestion control prevents the sender from overwhelming the network’s routers โ€” it is end-to-network. Flow control uses the receiver’s advertised window (rwnd); congestion control uses the sender’s congestion window (cwnd).

Q2. How does TCP detect congestion?

TCP assumes that on modern networks, transmission bit errors are rare. Therefore, any segment loss is assumed to be caused by congestion (a router dropping packets because its queue is full). TCP detects loss via: (1) retransmission timer expiry, or (2) receiving 3 duplicate ACKs (fast retransmit).

Q3. Explain slow start. Is it really slow?

Slow start begins with a small cwnd (1โ€“10 segments) and doubles it every RTT as ACKs arrive. This is exponential growth โ€” not slow at all in terms of rate. It is “slow” only compared to immediately sending the full window. The name reflects the contrast with the old approach (injecting the full window immediately). Slow start continues until cwnd reaches ssthresh.

Q4. What is ssthresh and how is it set?

ssthresh (slow-start threshold) marks the boundary between slow-start and congestion avoidance. Initially set large (so slow start runs freely). When congestion is detected, ssthresh is set to half of the current cwnd (the in-flight bytes at time of loss), capturing an estimate of the network’s safe capacity.

Q5. What is the sawtooth pattern in TCP congestion control?

A plot of cwnd vs. time shows a repeating sawtooth: cwnd rises (slow start then linear) until a loss event, then drops suddenly (reset to 1 or ssthresh), then rises again. This is the steady-state behavior of TCP Reno/CUBIC. BBR (Bottleneck Bandwidth and RTT), a newer algorithm, avoids this pattern by probing bandwidth without creating packet loss.

Q6. What is CUBIC and how is it different from TCP Reno?

CUBIC is the default TCP congestion control in Linux. Unlike Reno (which grows cwnd linearly based on RTT), CUBIC uses a cubic function of time since the last congestion event to set cwnd, making it independent of RTT. This gives CUBIC an advantage on high-bandwidth, high-latency links (long-distance) where Reno is too conservative.

Q7. What is ECN (Explicit Congestion Notification)?

ECN (RFC 3168) allows routers to signal congestion by marking packets (instead of dropping them). The receiver echoes the ECN marking back to the sender in the ACK. The sender reduces its cwnd as if a loss occurred, but without the actual packet loss. This improves performance because the sender reacts earlier and no retransmission is needed. Both endpoints and the path must support ECN.

Chapter 58 Complete!
You have covered all 5 topics: Connection, ACK/Retransmissions, Sequencing, Flow Control, and Congestion Control

๐Ÿ“… Chapter Index โ† Part 4

Leave a Reply

Your email address will not be published. Required fields are marked *