Prometheus and Grafana

Exporting to Prometheus

<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>

Example

curl -s localhost:8081/actuator/prometheus | grep shopease_orders

Example output

# HELP shopease_orders_placed_total Orders placed
# TYPE shopease_orders_placed_total counter
shopease_orders_placed_total{application="shopease",channel="web"} 1532.0

Scraping and Dashboards

ShopEase app (/actuator/prometheus)
        ^  scrape every 15 s
        |
   Prometheus (stores time series, evaluates alert rules) ---> Alertmanager ---> Slack / email / PagerDuty
        |
        v
     Grafana (dashboards: requests/sec, p95 latency, error rate, JVM heap, DB pool)

Example

prometheus.yml

scrape_configs:
  - job_name: shopease
    metrics_path: /actuator/prometheus
    static_configs:
      - targets: ['shopease-app:8081']

Example

Add a common tag so every metric is labelled with the app name:

application.yml

management:
  metrics:
    tags:
      application: shopease

Useful Queries

Panel	PromQL
Requests per second	`sum(rate(http_server_requests_seconds_count[1m]))`
5xx error rate	`sum(rate(http_server_requests_seconds_count{status=~"5.."}[5m]))`
p95 latency	`histogram_quantile(0.95, sum by (le) (rate(http_server_requests_seconds_bucket[5m])))`
Active DB connections	`hikaricp_connections_active`

Tip: For the p95 query, turn on histograms: management.metrics.distribution.percentiles-histogram.http.server.requests=true.