A slow dependency is worse than a dead one
Walk through this one slowly, because the conclusion is counterintuitive. A slow dependency is worse than a dead one.
Say your API service holds 200 worker threads and calls a recommendations service on every page render. Recommendations develops a problem, its own database is overloaded, and instead of failing it simply gets slow. Responses now take 30 seconds.
Follow what that does to you. Each incoming request parks one of your threads for 30 seconds. At 50 requests a second you would need 1,500 threads to keep up, and you have 200.
Within four seconds every thread is parked and your API stops answering anything at all, including the 95 percent of pages that never needed recommendations.
Watch it climb. Your callers now see you as slow, their threads park waiting on you, and the failure walks up the call graph one layer at a time. One overloaded database three hops away has taken down the whole stack.
Compare the dead dependency. Connections refused, errors back in a millisecond, threads freed instantly, and your API keeps serving everything else with a gap where recommendations would be. Fast failure is cheap. Slow failure is what kills.
Retries pour fuel on this. If your callers retry a timeout twice, the struggling service sees triple the traffic exactly when it can least take it, which is how something teetering at capacity gets shoved over and held down.
Timeouts are not enough
Set timeouts, and understand they are not sufficient alone. A two second timeout still parks each thread for two seconds, which at a high request rate is still exhaustion, only slower, and every timed-out request still lands on the sick dependency.
Want a mechanism that, after enough evidence of sickness, stops sending requests entirely and fails in microseconds. That mechanism is the circuit breaker.
Worked example
Netflix documented this class of outage while building Hystrix, and the shape repeats everywhere. Here is a typical replay: at a streaming company, the ratings service backing one row of the home screen starts timing out at 09:14 because of a bad database migration. The page-assembly service calls it with a 10-second timeout and no breaker. By 09:16, all 300 of page-assembly's worker threads are parked on ratings calls, and the entire home screen, of which ratings was one optional row, returns 504s worldwide. The edge tier's retries triple the pressure. Total user-facing outage: 40 minutes for a component that could have been dropped from the page invisibly. The postmortem's math is stark: a breaker tripping at 09:15 would have cost users one missing row.