Closed, open, half-open

A circuit breaker wraps the calls to one dependency and tracks how they go, borrowing its name from the panel in your house. When the circuit is unhealthy, the breaker opens and current stops flowing.

Closed is normal life. Calls pass through and the breaker counts results over a recent window.

Trip it when failures cross a threshold, say half of the last hundred calls failing, or twenty timeouts in ten seconds. Count slow calls as failures too, since the last lesson established that slowness is the deadlier symptom.

Open is the protective state. Every call is refused immediately, in microseconds, without touching the network.

Notice who that helps. Your threads stay free and your latency stays flat, and just as importantly the sick dependency gets a break from your traffic, which is often exactly what lets it recover. A recovery timer starts, usually tens of seconds.

Half-open is the careful question: are you better yet? When the timer expires, the breaker lets a small number of trial requests through while continuing to refuse the rest.

Close it if the probes succeed and normal traffic resumes. Snap back to open if they fail, and restart the timer.

Appreciate why that probing is the clever part. The alternative, reopening the floodgates on a timer alone, would slam your full traffic into a half-recovered service and knock it straight back down.

Two notes that bite people

Keep two implementation notes in mind, because they bite people. Breakers are per dependency and never global, so one sick neighbour does not block calls to healthy ones.

And a breaker complements your timeouts rather than replacing them. The timeout bounds one call, the breaker acts on the pattern across calls, and a capped connection pool per dependency is the third layer you want underneath both.

the shape of it

step 1 of 3

Failures trip the breaker open, a timer admits trial probes, and only successful probes close it again.

closed, open, half-open

Java

String call() {
  if (state == OPEN) {
    if (now().isBefore(openedAt.plusSeconds(30))) {
      throw new CircuitOpen();        // fails in microseconds, frees the thread
    }
    state = HALF_OPEN;                // timer expired, try one request
  }

  try {
    String r = dependency.call(Duration.ofMillis(600));   // timeout per call
    if (state == HALF_OPEN) { state = CLOSED; failures.clear(); }
    return r;
  } catch (TimeoutException | ServiceException e) {
    // Count slow calls as failures too. Part one: slowness is the
    // deadlier symptom, because it holds threads hostage.
    failures.record();
    if (state == HALF_OPEN || failures.rateOver(20) > 0.5) {
      state = OPEN;
      openedAt = now();
    }
    throw e;
  }
}

Worked example

Freya's checkout service calls a fraud-scoring vendor with a Resilience4j breaker: trip when 50 percent of a 20-call window fails or runs past 2 seconds, stay open 30 seconds, then allow 3 half-open probes. On a Thursday the vendor's EU region degrades, with calls timing out at their 2-second ceiling. Eleven timeouts inside a minute trip the breaker at 14:32:05. For the next 30 seconds, checkout calls fail in about 40 microseconds and the code path falls back to a rules-based score. At 14:32:35, three probes go out; two time out, and the breaker reopens for another 30 seconds. The cycle repeats for 9 minutes until probes succeed and the breaker closes at 14:41. Checkout latency at p99, the number its slowest one request in a hundred comes in under, never moves, and the vendor incident becomes a footnote instead of Freya's outage.

Closed, Open, Half-Open