Observabilitychapter 3 of 4 · 4 lessons
Distributed Tracing
Follow a single request across every service to find where time was spent or failures occurred.
Once a request touches ten services, the question of why it was slow stops having an easy answer. Each service's own dashboards look fine while the request as a whole takes four seconds.
Tracing is the signal built for exactly that gap, recording one request's path across every service with timing for each hop. Nobody adds it before they need it, which is unfortunate, because the day you need it is the day you cannot instrument ten services and wait for traffic.
Lessons
4 in this chapter- The Lost Request ProblemIn a microservice system, every team's graphs look fine while the user waits four seconds.2 min
- Traces, Spans, and Context PropagationA trace is a tree of timed spans, glued together by IDs riding on every request.3 min
- Sampling StrategiesYou cannot afford every trace, so choose between deciding early and cheap or late and smart.3 min
- Tracing in PracticeOpenTelemetry to instrument once, a backend to search, and a short list of shapes to hunt for.3 min