Careeroza

Distributed Tracing with Micrometer Tracing

Traces and Spans

A trace is the whole journey of one request. It has one trace ID (32 hex characters).
A span is one unit of work inside it (an HTTP call, a DB query, a Kafka send). Each span has its own span ID, a parent span ID, a start time and a duration.
The trace context travels between services in an HTTP header. The modern standard is W3C traceparent: 00-<traceId>-<spanId>-01; older Zipkin setups use X-B3-TraceId headers.

Example: How a Trace Is Built

Example

This mini tracer shows the idea behind Micrometer Tracing: the gateway starts the root span, sends traceparent to order-service, which creates child spans under it. The trace ID stays the same everywhere; each span knows its parent. IDs are fixed so the output is stable:

import java.util.*;

public class Main {
    static final List<Span> spans = new ArrayList<>();
    static int nextId = 1;

    public static void main(String[] args) {
        Span root = start("4bf92f3577b34da6a3ce929d0e0e4736", null, "gateway POST /api/orders", 0, 180);
        String header = "00-" + root.traceId() + "-" + root.spanId() + "-01";
        System.out.println("gateway -> order-service  traceparent: " + header);

        // order-service reads the header and continues the same trace
        String[] parts = header.split("-");
        Span order = start(parts[1], parts[2], "order-service placeOrder", 5, 175);
        start(order.traceId(), order.spanId(), "order-service SELECT product", 10, 25);
        Span pay = start(order.traceId(), order.spanId(), "payment-service POST /charge", 30, 160);
        start(pay.traceId(), pay.spanId(), "payment-service Razorpay API", 40, 150);
        start(order.traceId(), order.spanId(), "order-service kafka send", 162, 170);

        System.out.println();
        System.out.println("trace " + root.traceId() + "  (1 char = 10 ms)");
        print(null, 0);
    }

    static Span start(String traceId, String parentId, String name, int startMs, int endMs) {
        Span s = new Span(traceId, String.format("%016x", 0xa1b2c3d400L + nextId++), parentId, name, startMs, endMs);
        spans.add(s);
        return s;
    }

    static void print(String parentId, int depth) {
        for (Span s : spans) {
            if (!Objects.equals(s.parentId(), parentId)) continue;
            String bar = " ".repeat(s.startMs() / 10) + "#".repeat(Math.max(1, (s.endMs() - s.startMs()) / 10));
            System.out.printf("%-34s %3d ms |%-18s|%n", "  ".repeat(depth) + s.name(), s.endMs() - s.startMs(), bar);
            print(s.spanId(), depth + 1);
        }
    }
}

record Span(String traceId, String spanId, String parentId, String name, int startMs, int endMs) {}

Output

gateway -> order-service  traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-000000a1b2c3d401-01

trace 4bf92f3577b34da6a3ce929d0e0e4736  (1 char = 10 ms)
gateway POST /api/orders           180 ms |##################|
  order-service placeOrder         170 ms |################# |
    order-service SELECT product    15 ms | #                |
    payment-service POST /charge   130 ms |   #############  |
      payment-service Razorpay API 110 ms |    ###########   |
    order-service kafka send         8 ms |                # |

Reading the waterfall: the Razorpay call takes 110 of the 180 ms. That is exactly the kind of answer tracing gives you in seconds.

Setup in Spring Boot 3

Spring Cloud Sleuth was removed for Spring Boot 3. Tracing now lives in Micrometer Tracing, auto-configured by Actuator. Add these to every service:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-tracing-bridge-brave</artifactId>   <!-- or micrometer-tracing-bridge-otel -->
</dependency>
<dependency>
    <groupId>io.zipkin.reporter2</groupId>
    <artifactId>zipkin-reporter-brave</artifactId>             <!-- sends spans to Zipkin -->
</dependency>
<!-- only if you use OpenFeign: -->
<dependency>
    <groupId>io.github.openfeign</groupId>
    <artifactId>feign-micrometer</artifactId>
</dependency>

Configuration

application.yml

spring:
  application:
    name: order-service          # shown as the service name in Zipkin and in logs
  kafka:
    template:
      observation-enabled: true  # trace context goes into Kafka record headers
    listener:
      observation-enabled: true

management:
  tracing:
    sampling:
      probability: 1.0           # trace 100% in dev; use 0.1 (10%) in busy production
  zipkin:
    tracing:
      endpoint: http://localhost:9411/api/v2/spans

Example

Start Zipkin, place an order, and look at the logs. Spring Boot 3.2+ adds [traceId-spanId] to every log line automatically:

docker run -d --name zipkin -p 9411:9411 openzipkin/zipkin
curl -X POST localhost:8080/api/orders -H "Content-Type: application/json" -d '{"userId":7,"items":[{"productId":1,"quantity":2}]}'

Example output

# order-service log
2026-10-14T21:42:10.118+05:30  INFO 1 --- [order-service] [nio-8082-exec-3] [6650f1e2a1b3c4d5e6f708192a3b4c5d-e6f708192a3b4c5d] c.s.order.web.OrderController : Placing order for user 7
# payment-service log (same trace id, different span id)
2026-10-14T21:42:10.164+05:30  INFO 1 --- [payment-service] [nio-8084-exec-1] [6650f1e2a1b3c4d5e6f708192a3b4c5d-0d1e2f3a4b5c6d7e] c.s.payment.PaymentService : Charging Rs.2998.00 for order 1001

# Zipkin UI: http://localhost:9411 -> search traceId 6650f1e2... -> 5 spans, 3 services, 182 ms

Propagation Rules (Important)

Client	Traced automatically?
`RestClient` / `RestTemplate` / `WebClient` built from the auto-configured Builder bean	Yes
`RestClient.create()` or `new RestTemplate()`	No: the trace breaks here
OpenFeign	Yes, with `feign-micrometer` on the classpath
Kafka (KafkaTemplate / @KafkaListener)	Yes, with `observation-enabled: true`
`@Async` / your own ExecutorService	Only if the executor is wrapped with context propagation (Boot's task executor is)

@Configuration
public class HttpClients {
    @Bean
    RestClient paymentClient(RestClient.Builder builder) {   // inject the Builder: tracing + metrics included
        return builder.baseUrl("http://payment-service").build();
    }
}

// A custom span around business logic (Observation API):
Observation.createNotStarted("inventory.reserve", observationRegistry)
        .lowCardinalityKeyValue("warehouse", "BLR-1")
        .observe(() -> inventory.reserve(orderId, items));

Tip: Sampling 100% of production traffic is expensive. Use about 10% (0.1) and make sure error traces are kept; tail-based sampling in an OpenTelemetry Collector can keep all slow or failed traces.