TCP Is a State Machine
TCP does not just “connect” and “disconnect” — it goes through a precisely defined sequence of states. Understanding these states helps you debug real-world network issues, understand why a server can’t restart immediately after being killed, and write code that cooperates correctly with the TCP stack.
TCP State Diagram
TIME_WAIT — The Infamous State
After the active closer sends its final ACK, it does not go immediately to CLOSED. Instead, it enters the TIME_WAIT state and waits for 2 × MSL (Maximum Segment Lifetime) before the socket is truly closed. On Linux, MSL is typically 60 seconds, so TIME_WAIT lasts about 60 seconds (some sources say up to 120 seconds).
During TIME_WAIT, the kernel still owns that 4-tuple (local IP, local port, remote IP, remote port) and will not let another socket use it.
Reason 1: Reliable delivery of the final ACK.
When the active closer sends the final ACK for the peer’s FIN, that ACK might be lost. The peer will then retransmit its FIN. If the active closer went straight to CLOSED, it would have no socket to respond to the retransmitted FIN and would send an RST, which could confuse the peer. By staying in TIME_WAIT, the active closer can resend the final ACK if the peer’s FIN arrives again.
(TIME_WAIT)
(LAST_ACK)
Reason 2: Preventing “ghost” packets from old connections.
IP packets can be delayed in the network and arrive late. If a new connection reuses the same 4-tuple as a just-closed connection, a delayed packet from the old connection might arrive and corrupt the new connection’s data. TIME_WAIT ensures that all old packets have expired (their TTL has counted down) before the 4-tuple can be reused.
When you kill and immediately restart a server, you get this error:
$ ./my_server
bind: Address already in use
This happens because the server was the active closer (it called close() on its end), so the just-closed connections entered TIME_WAIT under the server’s port (e.g. port 8080). The OS refuses to let the new server bind to port 8080 while those TIME_WAIT sockets still hold that port.
You can verify this with netstat:
$ netstat -tan | grep 8080
tcp 0 0 0.0.0.0:8080 0.0.0.0:* LISTEN # new server fails here
tcp 0 0 127.0.0.1:8080 127.0.0.1:54321 TIME_WAIT # old connection in TIME_WAIT
tcp 0 0 127.0.0.1:8080 127.0.0.1:54322 TIME_WAIT # another old connection
SO_REUSEADDR — The Solution
The SO_REUSEADDR socket option tells the kernel to allow binding to a port that has TIME_WAIT sockets on it. The TIME_WAIT sockets still exist and still serve their purpose (protecting old packets), but they no longer block new binds to the same port.
#include <sys/socket.h>
#include <netinet/in.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
int create_server_socket(int port)
{
int sfd, opt = 1;
struct sockaddr_in addr;
sfd = socket(AF_INET, SOCK_STREAM, 0);
if (sfd == -1) { perror("socket"); return -1; }
/*
* SO_REUSEADDR MUST be set BEFORE bind().
* This is the standard boilerplate for every TCP server.
* Without it, restarting the server after kill fails with
* "Address already in use" for ~60 seconds.
*/
if (setsockopt(sfd, SOL_SOCKET, SO_REUSEADDR,
&opt, sizeof(opt)) == -1) {
perror("setsockopt SO_REUSEADDR");
close(sfd);
return -1;
}
memset(&addr, 0, sizeof(addr));
addr.sin_family = AF_INET;
addr.sin_port = htons(port);
addr.sin_addr.s_addr = INADDR_ANY;
if (bind(sfd, (struct sockaddr *)&addr, sizeof(addr)) == -1) {
perror("bind");
close(sfd);
return -1;
}
if (listen(sfd, 50) == -1) {
perror("listen");
close(sfd);
return -1;
}
printf("Server listening on port %d\n", port);
return sfd;
}
int main(void)
{
int lfd = create_server_socket(8080);
if (lfd == -1) return 1;
/* ... accept loop ... */
close(lfd);
return 0;
}
Important: SO_REUSEADDR does NOT disable TIME_WAIT — the old sockets remain in TIME_WAIT and continue to protect against stale packets. It just allows the new bind to succeed in spite of them.
Diagnostic Tools: netstat and tcpdump
netstat shows you the current state of all sockets on the system. It is indispensable for debugging.
# Show all TCP sockets (including listening) with addresses (no DNS lookup)
$ netstat -tan
# Show all TCP sockets with process IDs
$ netstat -tanp
# Show only listening sockets
$ netstat -tln
# Show sockets on a specific port
$ netstat -tan | grep :8080
# On newer Linux systems, use 'ss' (socket statistics) — faster
$ ss -tan
$ ss -tanp
$ ss -tan state time-wait # Show only TIME_WAIT sockets
Proto Local Address Foreign Address State tcp 0.0.0.0:8080 0.0.0.0:* LISTEN tcp 127.0.0.1:8080 127.0.0.1:54321 ESTABLISHED tcp 127.0.0.1:8080 127.0.0.1:54300 TIME_WAIT tcp 127.0.0.1:8080 127.0.0.1:54299 CLOSE_WAIT
The key states to look for in netstat output are: LISTEN (server is accepting), ESTABLISHED (active connection), TIME_WAIT (waiting for old packets to expire), and CLOSE_WAIT (peer has closed but app hasn’t called close() yet — often a bug).
tcpdump captures packets on a network interface and prints them. It is essential for verifying that the 3-way handshake, data transfer, and 4-way termination are happening correctly.
# Capture all TCP traffic on loopback, show sequence numbers
$ sudo tcpdump -i lo -n tcp -S
# Capture traffic on port 8080 only
$ sudo tcpdump -i any -n 'tcp port 8080'
# Capture and show packet contents in hex + ASCII
$ sudo tcpdump -i lo -n -X 'tcp port 8080'
# Save capture to file for analysis in Wireshark
$ sudo tcpdump -i eth0 -w capture.pcap 'tcp port 8080'
# tcpdump output for a TCP connection + close 12:00:01 IP client.54321 > server.8080: Flags [S], seq 1000 # SYN 12:00:01 IP server.8080 > client.54321: Flags [S.], seq 2000 ack 1001 # SYN-ACK 12:00:01 IP client.54321 > server.8080: Flags [.], ack 2001 # ACK 12:00:02 IP client.54321 > server.8080: Flags [P.], seq 1001:1005 # DATA 12:00:02 IP server.8080 > client.54321: Flags [.], ack 1005 # ACK data 12:00:03 IP server.8080 > client.54321: Flags [F.], seq 2001 # FIN (server closes) 12:00:03 IP client.54321 > server.8080: Flags [.], ack 2002 # ACK FIN 12:00:03 IP client.54321 > server.8080: Flags [F.], seq 1005 # FIN (client closes) 12:00:03 IP server.8080 > client.54321: Flags [.], ack 1006 # ACK FIN
TCP flags in tcpdump: [S] = SYN, [.] = ACK, [S.] = SYN+ACK, [F.] = FIN+ACK, [P.] = PUSH+ACK (data), [R] = RST.
Interview Questions
Answer: TIME_WAIT is the state entered by the active closer (the side that initiates the connection termination) after sending the final ACK. It lasts for 2×MSL (about 60 seconds on Linux). It exists for two reasons: (1) to allow the final ACK to be resent if it is lost and the peer retransmits its FIN, and (2) to prevent delayed packets from an old connection from being misinterpreted by a new connection that reuses the same 4-tuple.
Answer: When the server closes connections, it becomes the active closer and its port enters TIME_WAIT. The kernel prevents any new socket from binding to that port while TIME_WAIT sockets still hold it, because those sockets still serve the purpose of protecting against stale packets. The solution is to set SO_REUSEADDR before calling bind().
Answer: No, SO_REUSEADDR does NOT disable TIME_WAIT. The old sockets remain in TIME_WAIT and continue to protect against stale packets. SO_REUSEADDR only tells the kernel to allow a new bind() to succeed despite the existing TIME_WAIT sockets. It is safe and is standard practice — virtually every production TCP server sets it.
Answer: These are on opposite sides of the connection. FIN_WAIT_2 is on the active closer’s side — it has sent its FIN and received the ACK, but is waiting for the peer to send its FIN. CLOSE_WAIT is on the passive closer’s side — it has received the peer’s FIN and sent an ACK, but the application has not yet called close(). If CLOSE_WAIT sockets accumulate in your server (visible in netstat), it almost always means the application has a bug where it is not closing connections promptly after the peer closes.
Answer: Establishment uses a 3-way handshake: SYN → SYN+ACK → ACK (3 packets). Termination typically uses a 4-way handshake: FIN → ACK → FIN → ACK (4 packets), though the middle two can be combined into one FIN+ACK if the passive closer also has no more data to send, making it 3 packets.
Answer: Use netstat -tan | grep TIME_WAIT | grep :<port>. The -t shows TCP, -a shows all sockets including closed/time-wait, -n shows numeric addresses without DNS lookup (faster). On modern Linux, ss -tan state time-wait is faster and more powerful.
