EDI monitoring stack: OpenTelemetry + Grafana in practice

Monitoring of a 2026 EDI hub rests on three pillars: OpenTelemetry for unified instrumentation, an open backend (Prometheus + Grafana stack or equivalent), and dashboards aligned with the RED and USE methods.

Why OpenTelemetry?

OpenTelemetry (OTel, CNCF graduated project in 2024) emerged from the merger of OpenTracing and OpenCensus. It provides a backend-independent instrumentation standard for the three observability signals (traces, metrics, logs) and a universal export protocol format (OTLP). It has become the de facto standard for cloud-native observability in 2026.

For an EDI hub, OTel solves three structural problems: (1) avoid vendor lock-in (Datadog, New Relic, Dynatrace) — instrumentation remains portable; (2) unify export of traces (Jaeger, Tempo), metrics (Prometheus, Mimir) and logs (Loki, Elasticsearch) with a single SDK; (3) obtain native correlation between the three signals (a trace ID links a log and a metric to the same business event).

Three signals to instrument

• Traces: one trace per EDI message processed. Span per step (reception, parsing, validation, transformation, routing, ACK). Semantic attributes: edi.message.type, edi.message.id, edi.partner.id, edi.standard, edi.protocol. Allows instant answer to "why did this invoice take 4 minutes instead of 30s?".
• Metrics: temporal aggregates — count per message type, p50/p95/p99 latency, success rate, in-flight messages, queue depth. Controlled cardinality (never a label per message ID!). Prometheus /metrics exposition or OTLP push every 15-60 seconds.
• Logs: discrete events, structured (JSON), with trace ID for correlation. ERROR level only for what requires human action; WARN for situations to monitor; INFO for audit; DEBUG disabled in prod. Retention configurable per criticality (90 days operational, 7 years fiscal audit).

Target 2026 stack

Several valid combinations. Self-hosted open stack:

• Application → OpenTelemetry SDK → OTel Collector (sidecar or daemon)
• OTel Collector → Prometheus (metrics) → Mimir (long-term, multi-tenant)
• OTel Collector → Jaeger or Tempo (traces, with tail-based sampling)
• OTel Collector → Loki (structured logs)
• Grafana for visualisation of the three correlated signals
• Alertmanager for alert routing to PagerDuty/OpsGenie/Slack

Managed stack: Grafana Cloud, Datadog, New Relic, Dynatrace, Elastic Observability. Key advantage: no ops on the monitoring backend (retention, scaling, upgrades). Cost: cardinality- or ingestion-volume-based billing that can exceed the hub's compute cost itself if uncontrolled.

Hyperscaler-native stack: AWS CloudWatch + X-Ray, GCP Cloud Monitoring + Cloud Trace, Azure Monitor + Application Insights. Often easier to start with but less portable (lock-in).

EDI instrumentation patterns

Partner trace ID propagation

The native OTel trace ID is unknown to the external partner. To tie incidents end- to-end, propagate the partner functional identifier (RFF+ON for EDIFACT, BT-1 for UBL, ISA13 for X12) in span attributes. On a partner complaint "my invoice INV12345 is not received", this allows tracing the complete path in seconds.

Span per acknowledgement

Model the acknowledgement (CONTRL, 997, MDN, AS4 NRR) as a derived span from the parent span of the message. Attributes: edi.ack.type, edi.ack.code, edi.ack.timestamp. An abnormal acknowledgement latency (typically > 4h) triggers a per-partner alert.

RED metrics per partner

For each partner: Rate (msg/s), Error rate (errors/total), Duration (p50/p95/p99 latency). Combined with per-partner SLO ("99.5% of messages acknowledged within 4h"), generate an error budget and alert on exhaustion.

Tail-based sampling for traces

On a high-throughput hub (millions of messages/day), tracing 100% is too costly in storage. Head-based sampling (random 1% for example) loses interesting traces (errors, high latency). Tail-based sampling (OTel Collector or Tempo) keeps complete traces after evaluation: 100% of errors, 100% of long tail > p95, 1% of the rest. Storage divided by 10-20 without loss of useful information.

RED + USE for SLO dashboards

RED (Rate, Errors, Duration) — proposed by Tom Wilkie (Grafana) in 2015 — for request-oriented services. USE (Utilization, Saturation, Errors) — proposed by Brendan Gregg in 2012 — for physical resources (CPU, RAM, disk, network). Both methods are complementary.

Typical dashboards for an EDI hub:

• "Business" dashboard: messages/day per type, success rate per partner, top business errors, OTIF per partner.
• "SLO" dashboard: remaining error budget for each public SLO, burn rate, exhaustion projection.
• "RED" dashboard: for each microservice (parser, validator, router), Rate/Errors/Duration aggregated over 5 min/1 h/24 h.
• "USE" dashboard: CPU/memory/disk IOPS/network per node, Kafka queue depth, consumer lag.
• "Partner SLA" dashboard: per-partner view with latency, success rate, recent incidents, contractual SLA compliance.

Monitoring cost

For an EDI hub processing 10 million messages/month:

• Self-hosted (Prometheus/Mimir/Tempo/Loki/Grafana): typically 800-2 500 EUR/month of compute + storage (dedicated k8s cluster), plus 0.2-0.5 FTE ops. Advantage: cost almost linear with volume.
• Grafana Cloud: ~1 500-5 000 EUR/month depending on ingestion (metrics, logs, traces volumes). Free tier to start.
• Datadog: typically 3 000-15 000 EUR/month depending on host cardinality and custom metrics. Inflation risk without governance.

Further reading

• EDI metrics: SLO, error budgets, KPIs.
• AI-augmented EDI ops — LLM-assisted root cause analysis builds on these signals.
• OpenTelemetry — Semantic Conventions. opentelemetry.io
• Google SRE Workbook — chapter 4 "Service Level Objectives". sre.google
• Wilkie T. — The RED Method: How To Instrument Your Services, Grafana 2018.