Isn't the load balancer just a new single point of failure?

It would be if you ran exactly one. Production setups run balancers in redundant pairs with a floating IP, or active-active behind DNS or anycast. Managed balancers like AWS ALB spread themselves across availability zones behind one name. The balancer's failure domain ends up smaller than any single app server's.

When is round robin not good enough?

When request durations vary a lot. Round robin balances the number of requests per server, not the amount of work, so a few slow requests can pile onto one node while others idle. Least connections handles that case by routing to whichever backend has the fewest requests in flight.

Should my health check endpoint test the database?

Not the one the load balancer uses. If every node's check depends on the database, a short database blip fails all checks at once and the balancer ejects the whole fleet, turning a 20-second hiccup into minutes of total downtime. Keep the balancer check shallow and alert on dependencies separately.

Scalingchapter 2 of 4 · 4 lessons

Load Balancing

Distribute traffic across servers so no single machine is overwhelmed.

← All chapters View in cheatsheet

You bought the second server. Now a request arrives and something has to decide which of the two gets it.

That something is a load balancer, and it turns out to do four jobs rather than one. It spreads the traffic. It stops sending requests to a machine that has died. It lets you deploy by draining one machine at a time while the other carries the load. And it is usually where HTTPS ends.

It is the first distributed systems component most teams ever run, and it stays load-bearing forever after. It is also, from the day you install it, the thing that everything goes through.

Lessons

4 in this chapter

What a Load Balancer DoesOne address in front, many interchangeable servers behind, and failure handled per request.2 min
Picking an AlgorithmRound robin balances request counts. The other algorithms exist because count is not load.6 min
L4 vs L7L4 forwards connections it cannot read, fast. L7 reads the request and routes on it.3 min
Health Checks and Failure HandlingDetection lag is interval times threshold, and a deep health check can eject your whole fleet.2 min