One rule, and four things that break it

One rule makes horizontal scaling work: any machine must be able to serve any request. Every place your application quietly assumes there is exactly one server is a bug you have not met yet. Going from one machine to two flushes them all out at once.

Expect sessions to break first. Keep them in one server's memory and a user logs in on machine A, the balancer sends their next request to machine B, and B has never heard of them.

Pinning users to a machine patches it, and then a machine dying logs out everybody pinned to it, and your deploys get awkward. Move sessions into a shared store instead, one fast lookup per request.

Watch local disk go next. Uploads written to the filesystem exist on one machine only, and every other machine returns a 404 for them. Move files to object storage, and treat anything left on disk as disposable.

Treat caches inside each process as a consistency problem now. Every machine holds its own copy, clearing one does nothing for the others, and people see different data depending on where the balancer sent them. Use a shared cache, or expiries short enough that you can live with the disagreement.

Count your scheduled jobs, because they multiply. A job baked into the server image now runs once per machine, so ten welcome emails, ten billing runs. You need a lock they compete for, an elected leader, or better, one separate worker that owns the job.

None of these are hard on their own. The trap is that they stay hidden until the second machine arrives, which is usually during the traffic spike that forced you to add it. Teams that go horizontal smoothly are the ones running two machines in staging long before production needed them.

the shape of it

step 1 of 2

State moves out of the nodes and into shared stores, so any node can serve any request.

the four things that only work when there is one machine

Java

// 1. Session in local memory. Machine A knows this user, B does not.
Map<String, Session> sessions = new HashMap<>();   // move to Redis

// 2. Upload written to local disk. Every other machine 404s it.
file.transferTo(new File("/var/uploads/" + name)); // move to object storage

// 3. Cache in this process. Clearing it here clears nothing elsewhere.
cache.evict(productId);                            // move to a shared cache

// 4. Scheduled job baked into the server. Ten machines, ten runs.
@Scheduled(cron = "0 0 9 * * *")                   // move to one worker,
void sendDailyDigest() { ... }                     // or take a lock first

Worked example

Kofi scales his company's Express app from one container to four on a Friday, ahead of a marketing push. By Saturday morning support has 60 tickets. Users are being logged out mid-checkout, because sessions live in process memory and a user authenticated on container 2 looks anonymous to container 3. Profile photos uploaded that morning 404 for three out of four requests, since they sit on one container's disk. And four copies of the nightly invoice cron ran, so 212 customers got four identical emails. The fixes take two days: connect-redis for sessions, S3 for uploads, and the scheduled job moved to a single task holding a lock in Redis, a shared in-memory store, as a belt and suspenders.

What Breaks When You Go Horizontal

One rule, and four things that break it

Worked example