The Problem: Too Many Clients
A single server process can only handle so many clients before it becomes the bottleneck. When client load is very high, we need ways to spread the work across multiple machines. A group of server machines working together to serve clients is called a server farm.
There are two common approaches to building a server farm:
- DNS Round-Robin Load Sharing – cheap and simple, uses DNS itself
- Dedicated Load-Balancing Server – more complex but more powerful
How It Works
In normal DNS, one domain name maps to one IP address. In DNS round-robin, the authoritative DNS server for a zone maps the same domain name to several IP addresses. Each IP belongs to a different physical server.
When a client asks “what is the IP of api.example.com?”, the DNS server answers with all the IPs but rotates their order each time. Client 1 gets Server A first. Client 2 gets Server B first. Client 3 gets Server C first. And so on — round-robin style.
api.example.com
Advantages
- Inexpensive – No extra hardware needed, just a DNS configuration change
- Easy to set up – Supported by all major DNS server software
Problems with DNS Round-Robin
- Remote DNS caching: When a remote DNS server resolves your domain, it caches the result for the TTL (time-to-live) duration. All clients using that DNS server will always hit the same backend server — completely bypassing the round-robin. The rotation only happens at your authoritative DNS, not at cached copies.
- No true load balancing: Different clients place different loads on a server. One client may download 100MB while another just fetches 1KB. Round-robin distributes connections equally, not load equally.
- No high availability: If one of the backend servers crashes, DNS round-robin has no mechanism to detect this and stop sending clients to the dead server.
- Server affinity challenges: A sequence of requests from the same client may go to different servers (because DNS cache expires and a different IP is returned). If Server A stored session data for that client, Server B won’t have it.
How It Works
A single load-balancing server sits in front of all the backend servers. To the outside world, the server farm looks like one machine — it has one IP address (the load balancer’s IP). All client connections go to the load balancer first, and it decides which backend server should handle each request.
Single Public IP
(+ Backup LB)
What a Load Balancer Provides
- Eliminates DNS caching problem: Because the server farm presents only one IP to the world, clients always go through the load balancer — no DNS caching issues.
- Intelligent load distribution: The load balancer uses algorithms that consider actual server load (CPU, memory, active connections) — not just connection count. Backend servers can report their metrics to the load balancer to help it decide.
- Automatic failure detection: The load balancer health-checks backend servers regularly. If a backend server crashes, the load balancer stops sending traffic to it automatically.
- Server affinity support: The load balancer can use techniques like sticky sessions or session cookies to ensure a client always goes back to the same backend server for the duration of a session.
- High availability: To avoid the load balancer itself becoming a single point of failure, a backup load balancer is typically deployed. If the primary crashes, the backup takes over.
Server affinity (also called session persistence or sticky sessions) means ensuring that all requests from the same client always go to the same backend server.
Why does this matter? Many server applications maintain state about a client — for example, a login session, a shopping cart, or a partially processed file upload. If Client X talks to Server A and logs in, Server A stores that session. If the next request from Client X goes to Server B, Server B has no idea who this client is — it sees an unauthenticated request and may reject it.
| Feature | DNS Round-Robin | Load Balancer |
|---|---|---|
| Cost & Complexity | Low – just DNS config | Higher – needs dedicated server/software |
| True Load Balancing | No – just connection rotation | Yes – metrics-based distribution |
| Failure Detection | None | Yes – health checks |
| Server Affinity | Difficult | Yes – built-in support |
| DNS Caching Issue | Yes – bypasses round-robin | None – single IP presented |