Observability and Deployment Questions
What are the three pillars of observability?
| Pillar | Answers | Tools |
|---|---|---|
| Logs | what happened in this request? | JSON logs to stdout → ELK / Loki; include traceId |
| Metrics | how is the system doing overall? | Micrometer → Prometheus → Grafana (latency p99, error rate, RPS) |
| Traces | where did the time go across services? | Micrometer Tracing → Zipkin / Jaeger / Tempo (OpenTelemetry) |
How does distributed tracing work in Spring Boot 3?
Spring Cloud Sleuth is replaced by Micrometer Tracing. Each request gets a trace id; each hop (HTTP call, Kafka message, DB query) is a span with its own span id and parent id. Context propagates in the W3C traceparent header, automatically for RestClient, WebClient, Feign, KafkaTemplate built by Spring.
<!-- pom.xml -->
<dependency><groupId>io.micrometer</groupId><artifactId>micrometer-tracing-bridge-otel</artifactId></dependency>
<dependency><groupId>io.opentelemetry</groupId><artifactId>opentelemetry-exporter-zipkin</artifactId></dependency>management:
tracing:
sampling:
probability: 0.1 # sample 10% in prod, 1.0 in dev
zipkin:
tracing:
endpoint: http://zipkin:9411/api/v2/spans
logging:
pattern:
level: "%5p [%X{traceId:-},%X{spanId:-}]"What is a correlation id and how does it reach the logs of every service?
A single id for one user request, carried in a header to every service and put in the logging MDC, so grep traceId=4bf92f… in Kibana shows the whole request across order, payment and inventory. With Micrometer Tracing the trace id is the correlation id and is added to the MDC as traceId automatically. For Kafka, it travels in message headers. For @Async or custom executors, wrap them with context propagation (ContextExecutorService) or the id is lost.
Which health checks do you expose for Kubernetes? Liveness vs readiness?
| Probe | Question | If it fails | Spring Boot endpoint |
|---|---|---|---|
| Liveness | is the process stuck/broken? | K8s restarts the pod | /actuator/health/liveness |
| Readiness | can it serve traffic now? | pod removed from Service endpoints, not restarted | /actuator/health/readiness |
| Startup | has it finished starting? | other probes wait | same as liveness with a longer budget |
Tip: Do not put database or downstream checks in liveness: if MySQL is down, restarting every pod helps nobody. Put them in readiness (management.endpoint.health.group.readiness.include=readinessState,db).
How do you deploy microservices without downtime? Blue-green vs canary vs rolling?
| Strategy | How | Trade-off |
|---|---|---|
| Rolling | replace pods one by one (K8s default, maxUnavailable/maxSurge) | cheap; old and new versions run together briefly |
| Blue-green | deploy full new stack (green), switch traffic at once, keep blue for rollback | instant rollback; double resources |
| Canary | send 5% → 25% → 100% of traffic to the new version while watching metrics | safest for risky changes; needs traffic splitting (Istio, Argo Rollouts) |
All need backward-compatible APIs and DB migrations (expand → migrate → contract), plus graceful shutdown so in-flight requests finish.
How do you structure CI/CD for many microservices?
- One pipeline per service (repo-per-service, or path filters in a monorepo): build → unit tests → contract tests → image build → push → deploy to staging → smoke tests → prod.
- Immutable images tagged with the git SHA; config injected per environment (ConfigMap, Secrets, Config Server).
- Semantic versioning for APIs and events; never deploy a breaking change before all consumers support it.
- GitOps (Argo CD) to sync K8s manifests from Git; automated rollback on failed health checks.