What is Congestion Control?
Flow control protects the receiver. But what about the network itself? A fast TCP sender can flood an intermediate router, causing it to drop packets. If every TCP sender then retransmits at the same high rate, the network gets worse, not better โ a scenario called congestion collapse.
TCP’s congestion control algorithms prevent this. They work at the sender, reducing the transmission rate when congestion is detected, and increasing it gradually when the network is healthy. TCP uses two algorithms together: Slow Start and Congestion Avoidance.
Consider what happens without congestion control: a new TCP connection has a full receiver window (say 64 KB). Without any limit, the sender immediately injects 64 KB of data into the network. If there are 100 simultaneous connections doing the same thing, the router in the middle is suddenly flooded with megabytes of data it cannot forward fast enough.
The router drops packets. The senders all retransmit. The router drops those too. The network enters congestion collapse โ it is busy but delivering almost nothing.
TCP detects congestion through packet loss. The assumption is: on modern networks, transmission errors (bit flips) are rare. So if a segment is lost, the most likely cause is a router dropping it due to congestion.
The sender maintains a congestion window (cwnd) โ a limit on how many bytes can be in-flight at once, independent of the receiver’s advertised window.
The actual amount of data the sender can have unacknowledged is:
effective_window = min(cwnd, receiver_advertised_window)
Both limits apply: the sender respects the receiver’s buffer (flow control) and also self-limits through cwnd (congestion control). The congestion control algorithms manage how cwnd grows and shrinks.
Despite its name, slow start is actually exponential growth. The sender starts with a small cwnd (typically 1โ10 segments) and doubles it with every RTT as long as no congestion is detected. This quickly probes the network’s capacity without immediately flooding it.
cwnd=1
cwnd=2
cwnd=4
cwnd=8
linear now
Slow start runs until cwnd reaches the slow-start threshold (ssthresh). After that, TCP switches to congestion avoidance mode (linear growth).
Initial ssthresh is typically set to a very large value (so slow start runs for a while on a fresh connection). After the first congestion event, ssthresh is set based on the detected network capacity.
Once cwnd reaches ssthresh, the sender shifts to congestion avoidance: instead of doubling cwnd every RTT, it increases cwnd by 1 MSS per RTT (linear growth). This cautious growth continues until congestion is detected.
When congestion is detected (a segment is lost):
- ssthresh is set to half of the current cwnd (estimated network capacity)
- cwnd is reset (back to 1, or to ssthresh depending on TCP variant)
- The cycle repeats from slow start or from ssthresh
Together, slow start and congestion avoidance allow TCP to:
- Quickly ramp up to available bandwidth (slow start’s exponential growth)
- Back off when the network signals congestion (packet loss)
- Reach equilibrium โ a steady state where the sender transmits at roughly the same rate as the network can carry, without significant queuing
The actual amount of data the sender can inject at any time is limited by:
bytes_in_flight โค min(cwnd, receiver_window, send_buffer)
All three limits apply simultaneously. The most restrictive one wins at any given moment.
/* tcp_cwnd_monitor.c - monitor congestion window in real time */
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netinet/tcp.h>
#include <arpa/inet.h>
#define SERVER_IP "127.0.0.1"
#define SERVER_PORT 8080
#define CHUNK_SIZE (4 * 1024) /* 4 KB per write */
#define TOTAL_BYTES (4 * 1024 * 1024) /* send 4 MB total */
void print_cwnd(int sockfd, int iteration)
{
struct tcp_info ti;
socklen_t len = sizeof(ti);
if (getsockopt(sockfd, IPPROTO_TCP, TCP_INFO, &ti, &len) == 0) {
printf("[%3d] cwnd=%4u segs | ssthresh=%4u | rtt=%5u us | "
"retrans=%u\n",
iteration,
ti.tcpi_snd_cwnd, /* congestion window in segments */
ti.tcpi_snd_ssthresh, /* slow-start threshold */
ti.tcpi_rtt, /* smoothed RTT in microseconds */
ti.tcpi_retransmits); /* current retransmission count */
}
}
int main(void)
{
int sockfd;
struct sockaddr_in srv;
char buf[CHUNK_SIZE];
int sent = 0, iter = 0;
sockfd = socket(AF_INET, SOCK_STREAM, 0);
if (sockfd == -1) { perror("socket"); exit(EXIT_FAILURE); }
memset(&srv, 0, sizeof(srv));
srv.sin_family = AF_INET;
srv.sin_port = htons(SERVER_PORT);
srv.sin_addr.s_addr = inet_addr(SERVER_IP);
if (connect(sockfd, (struct sockaddr *)&srv, sizeof(srv)) == -1) {
perror("connect"); exit(EXIT_FAILURE);
}
memset(buf, 'A', sizeof(buf));
printf("%-5s %-10s %-12s %-12s %s\n",
"Iter", "cwnd(seg)", "ssthresh", "RTT(us)", "retrans");
printf("-------------------------------------------\n");
while (sent < TOTAL_BYTES) {
ssize_t n = write(sockfd, buf, CHUNK_SIZE);
if (n <= 0) { perror("write"); break; }
sent += n;
/* Print cwnd every 10 writes */
if (++iter % 10 == 0) {
print_cwnd(sockfd, iter);
}
}
printf("\nDone. Total sent: %d bytes\n", sent);
close(sockfd);
return 0;
}
/* Checking TCP congestion algorithm used by the kernel */
#include <stdio.h>
#include <sys/socket.h>
#include <netinet/tcp.h>
#include <unistd.h>
#include <string.h>
int main(void)
{
int sockfd = socket(AF_INET, SOCK_STREAM, 0);
char ca_name[16] = {0};
socklen_t len = sizeof(ca_name);
/*
* TCP_CONGESTION: get or set the congestion control algorithm
* Common Linux algorithms: cubic (default), reno, bbr, htcp
*/
if (getsockopt(sockfd, IPPROTO_TCP, TCP_CONGESTION,
ca_name, &len) == 0) {
printf("Current congestion algorithm: %s\n", ca_name);
}
/* Switch to BBR (Bottleneck Bandwidth and RTT) if available */
const char *bbr = "bbr";
if (setsockopt(sockfd, IPPROTO_TCP, TCP_CONGESTION,
bbr, strlen(bbr)) == 0) {
printf("Switched to BBR congestion control\n");
} else {
perror("setsockopt TCP_CONGESTION bbr");
printf("BBR may not be available; using: %s\n", ca_name);
}
close(sockfd);
return 0;
}
# Check system-wide TCP congestion control setting
cat /proc/sys/net/ipv4/tcp_congestion_control
# List all available algorithms
cat /proc/sys/net/ipv4/tcp_available_congestion_control
# Change system default (requires root)
echo bbr > /proc/sys/net/ipv4/tcp_congestion_control
# Enable BBR (needs kernel 4.9+)
modprobe tcp_bbr
echo tcp_bbr >> /etc/modules-load.d/modules.conf
Each Internet protocol is defined in an RFC (Request for Comments) document. RFCs are published by the RFC Editor and developed under the IETF (Internet Engineering Task Force). Here are the most important ones for TCP/IP:
Flow control prevents the sender from overwhelming the receiver’s buffer โ it is endpoint-to-endpoint. Congestion control prevents the sender from overwhelming the network’s routers โ it is end-to-network. Flow control uses the receiver’s advertised window (rwnd); congestion control uses the sender’s congestion window (cwnd).
TCP assumes that on modern networks, transmission bit errors are rare. Therefore, any segment loss is assumed to be caused by congestion (a router dropping packets because its queue is full). TCP detects loss via: (1) retransmission timer expiry, or (2) receiving 3 duplicate ACKs (fast retransmit).
Slow start begins with a small cwnd (1โ10 segments) and doubles it every RTT as ACKs arrive. This is exponential growth โ not slow at all in terms of rate. It is “slow” only compared to immediately sending the full window. The name reflects the contrast with the old approach (injecting the full window immediately). Slow start continues until cwnd reaches ssthresh.
ssthresh (slow-start threshold) marks the boundary between slow-start and congestion avoidance. Initially set large (so slow start runs freely). When congestion is detected, ssthresh is set to half of the current cwnd (the in-flight bytes at time of loss), capturing an estimate of the network’s safe capacity.
A plot of cwnd vs. time shows a repeating sawtooth: cwnd rises (slow start then linear) until a loss event, then drops suddenly (reset to 1 or ssthresh), then rises again. This is the steady-state behavior of TCP Reno/CUBIC. BBR (Bottleneck Bandwidth and RTT), a newer algorithm, avoids this pattern by probing bandwidth without creating packet loss.
CUBIC is the default TCP congestion control in Linux. Unlike Reno (which grows cwnd linearly based on RTT), CUBIC uses a cubic function of time since the last congestion event to set cwnd, making it independent of RTT. This gives CUBIC an advantage on high-bandwidth, high-latency links (long-distance) where Reno is too conservative.
ECN (RFC 3168) allows routers to signal congestion by marking packets (instead of dropping them). The receiver echoes the ECN marking back to the sender in the ACK. The sender reduces its cwnd as if a loss occurred, but without the actual packet loss. This improves performance because the sender reacts earlier and no retransmission is needed. Both endpoints and the path must support ECN.
