The problem: an even split is not a fair split
Six servers, and a balancer handing each one every sixth request. After an hour, each has had exactly 10,000 requests. Perfectly even.
Server 4 is on fire and the other five are bored.
The reason is that your requests are not the same size. Your document service renders most pages in 200 milliseconds, and a few large spreadsheets take 40 seconds. Sooner or later three of those 40-second jobs land on server 4 by chance, its queue backs up, and every quick request routed there afterwards waits behind them.
A balancer that counts requests is not measuring the thing that fills a server up. Work fills a server up, and requests are only a rough stand-in for work.
That gap between requests and work is what every algorithm below is trying to close, and each one closes it differently, at a different price.
Round robin: count the requests
Round robin walks the server list in order. Server 1, then 2, then 3, back to 1. That is the whole rule.
It needs no state, no measurement, and no coordination, so it costs a pointer increment and is the default in every balancer you will meet. On a fleet of identical machines serving requests that cost about the same, it is genuinely correct. Reaching past it is one of the more common ways to overcomplicate a design.
Weighted round robin handles machines that are not identical. Give each server a weight and it receives that share of the rotation, so the box with twice the cores takes twice the traffic. You are still counting requests; you have only changed how many each server gets.
Both break the same way, and it is the document service above. Round robin has no idea that server 4 is holding three spreadsheets, because from where it stands every server has had the same number of turns.
Least connections: count the work in flight
Least connections sends each new request to whichever server currently has the fewest requests still open.
The insight is small and does the whole job. An open connection is a request that has not finished, so the number of them is a live reading of how busy that machine is right now. A server chewing on three spreadsheets is holding three connections and stops being chosen until it clears them.
Round robin asks whose turn is it. Least connections asks who is free. That single change fixes the document service, and it costs one counter per server.
Weighted least connections exists for the same reason as its round robin cousin. Divide each server's open connections by its capacity, and a machine twice as strong is allowed twice as many before it looks equally loaded.
The cost is that the balancer now holds state per server and must update it on every open and close. That is trivial on one balancer and becomes a real question across a tier of them. Each balancer sees only its own connections, and none of them knows the true total.
Least response time: measure the outcome, not the count
Least response time goes one step further and routes toward whichever server is currently answering fastest, usually by blending its open connections with a rolling average of its recent latency.
This catches something the other two cannot. A machine with a failing disk, a noisy neighbour, or a garbage collection pause still accepts connections normally, so least connections keeps feeding it. Its response times climb first, and this algorithm quietly steers traffic away before any health check has decided anything is wrong.
You pay for it in bookkeeping and in tuning. The balancer measures every response and keeps a decaying average per server. Decay that average too slowly and it reacts late; too quickly and it turns twitchy, chasing noise.
Power of two choices: nearly the same result, far less bookkeeping
Power of two random choices is the one people miss, and it is worth knowing because large systems actually run it. Pick two servers at random, compare only those two, and send the request to whichever has fewer connections.
Checking two instead of all of them sounds like it should be much worse. It is not. Picking one at random leaves the worst server badly overloaded, and this variant cuts that imbalance dramatically, landing close to what checking every server gets you.
It matters because comparing all N servers means the balancer holds a global view, and a fleet of balancers cannot cheaply agree on one. Two random probes need almost no shared state, which is why this shows up in Nginx, in service meshes, and in schedulers that place work across thousands of machines.
Hashing: a different question entirely
The four above all ask some version of who is least busy. Hashing asks something else: how do I keep sending the same user to the same server.
IP hash takes the client's address, hashes it, and takes the remainder over the server count. The same client lands on the same server every time, with no cookie and nothing stored anywhere. That gives you sticky sessions for free, and it inherits the flaw from the consistent hashing chapter: change the server count and almost every client is reassigned at once.
Consistent hashing fixes exactly that. Servers and clients are placed on a ring, and adding or removing one moves only the clients in its arc, roughly a fraction of them rather than nearly all. That is why cache tiers use it, and why it reappears in sharding.
Be careful what you are buying. Both pin traffic by client rather than by load. A single busy corporate network behind one address becomes one very busy server, and neither algorithm will do a thing about it.
Choosing, in one question
Ask whether your requests cost roughly the same.
If they do, round robin and move on. If their durations vary wildly, least connections. If you need the same user on the same machine for a warm cache or a session, consistent hashing. If you run many balancers and cannot keep a global view, power of two choices.
In an interview nobody cares which name you pick. They care that you can say why. The sentence that shows it: round robin balances request counts, not load, and the gap between those two is where your latency comes from.
Worked example
Jonas owns a document export service: most requests render in 200 ms, but big spreadsheets take up to 40 seconds. Under round robin on 6 backends, luck occasionally stacks three whale exports on one node; its queue backs up, and every small request routed there afterward waits behind them. p99 latency sits at 38 seconds while average CPU across the fleet reads a healthy 45 percent, which is the tell that the balancing, not the capacity, is the problem. He changes one line in the HAProxy config, balance roundrobin to balance leastconn. Whales now spread out, because a node chewing on two exports holds more connections and stops receiving new work. p99 drops to 6 seconds by the next afternoon, with zero new hardware.