Observability Fundamentals

Why Microservices Need Observability

A customer says "my order failed at 9:42 PM". In a monolith you open one log file. In ShopEase microservices the request went through the gateway, order-service, inventory-service, payment-service and Kafka, with 3 instances each. Without help, you are searching 15 log files on 15 containers.

Observability means you can answer new questions about the running system from the data it emits, without adding new code first.

The Three Pillars

PillarAnswersSpring toolBackend
LogsWhat exactly happened in this service?SLF4J + Logback, structured JSON logsELK (Elasticsearch, Logstash, Kibana), Grafana Loki
MetricsHow many, how fast, how full? (numbers over time)Micrometer + ActuatorPrometheus + Grafana
TracesWhich services did ONE request touch, and where was time spent?Micrometer Tracing (Brave or OpenTelemetry bridge)Zipkin, Jaeger, Grafana Tempo
order-service, payment-service, ... (each instance)
            |  logs (JSON)        |  /actuator/prometheus     |  spans
            v                     v                           v
     Loki / Elasticsearch     Prometheus (scrapes)         Zipkin / Tempo
            \                     |                          /
             +-------------------> Grafana <----------------+
                       dashboards, alerts, "jump from log to trace"

Note: The glue between the pillars is the trace ID: it appears in every log line and every span of a request, so you can jump from an error log to the full trace.

Golden Signals to Watch

SignalExample metric for order-serviceAlert when
Latencyp95 of http_server_requests_secondsp95 > 800 ms for 5 min
Trafficrequests per seconddrops to 0 (something upstream broke)
Errorsrate of 5xx responses> 1% for 5 min
SaturationHikariCP active connections, JVM heap, CPUpool 100% used, heap > 90%

Tip: Spring Boot Actuator basics (health, metrics endpoints, custom health indicators) are covered in the Spring Boot material. Here we focus on what changes when there are many services.