Service Mesh
The L7 network fabric making mTLS, retries and observability invisible: once enabled, application code does not know it exists.
Problem
A modern EDI hub counts 40 internal services: EDIFACT parsers, X12 parsers, PEPPOL validators, AS2 and AS4 brokers, archive, audit, ML scoring, Factur-X signatures. If every service embeds its mTLS, retry, observability libraries, versions multiply, coordinated rollouts fail, and nobody knows who talks to whom. At 40 services the n × n matrix is unmanageable. Combining Sidecar/Ambassador manually per Pod works but explodes at 200 Pods.
Forces
- Service count explodes. Past 20 internal services the network-matrix complexity exceeds human limits.
- Network patterns are identical everywhere. mTLS, retries, circuit breaker — not service-specific. Factor them out.
- Policies change fast. A new crypto standard, a new trace header — pushing to 40 services at once is risky.
- The mesh has a cost. Per-Pod CPU/memory (Envoy ~50 MB), added latency (~1 ms), operational complexity of the control plane.
Solution
Deploy a service mesh — Istio, Linkerd, Consul Connect, AWS App
Mesh, Kuma. A central control plane distributes configuration
(mTLS policies, routes, retry policies) via xDS APIs. Each Pod
automatically receives (sidecar injection via mutating webhook) an
Envoy proxy intercepting all traffic via iptables. Application code
calls http://parser-edifact:8080 without TLS, without
retry, without tracing: the mesh adds everything. Policies: managed
as YAML or Kubernetes CRD, versioned in git.
┌─────────────────────────────────────────────────────────┐
│ Control plane (Istiod / Linkerd controller) │
│ ◇ mTLS policy ◇ retries ◇ canary routes │
└────────┬──────────┬─────────────┬────────────────┬──────┘
│ xDS │ xDS │ xDS │ xDS
▼ ▼ ▼ ▼
Pod parser Pod broker Pod archive Pod portal
┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐
│ app │ │ app │ │ app │ │ app │
│ + Envoy │ │ + Envoy │ │ + Envoy │ │ + Envoy │
└─────────┘ └─────────┘ └─────────┘ └─────────┘
◀── mTLS / retries / observability automatic ──▶
EDI implementation
Concrete case: Kubernetes EDI hub with Istio. The EDIFACT parser
service calls http://validator-peppol:8080/check. The mesh
intercepts: checks the PeerAuthentication (strict mTLS
required), applies the DestinationRule (5s timeout, 3
retries on 503, circuit breaker at 50% errors), propagates W3C traceparent headers, exports RED metrics (rate,
errors, duration) to Prometheus. If a v2 of the validator with a
new rule is deployed, a VirtualService sends 5% of
traffic in canary — without modifying the caller. SPIFFE
certificate rotation for mTLS: automatic every 24h, no redeploy.
Anti-patterns
- Mesh by default on a small system. 5 services: the mesh's operational complexity outweighs its benefits. Manual sidecar or application library suffice.
- Business logic in Envoy filters. Tempting to put
business routing (by
partnerId) in a WASM filter: escapes code governance, undebuggable. - Mesh as an excuse not to test. If the app panics the mesh will not make it work. The code stays responsible for business correctness.
- Uncoordinated multi-mesh. Istio on cluster A, Linkerd on cluster B: non-trivial inter-mesh bridge. If multi-cluster, pick one mesh.
Related patterns
- Sidecar — building block: every Pod in the mesh embeds an Envoy sidecar.
- Ambassador — the ambassador generalised at cluster scale.
- Circuit Breaker — natively implemented by the mesh.
- Bulkhead
— implemented via
connectionPoolin aDestinationRule. - Retry + backoff — retry policy configured at mesh level.
Sources
- Istio — What is Istio? The dominant mesh. istio.io
- Linkerd — A service mesh for Kubernetes. The mesh aiming for lighter footprint (Rust linkerd2-proxy). linkerd.io
- CNCF — Service Mesh Interface (SMI). Attempt to standardise mesh APIs, eventually merged into Gateway API. gateway-api.sigs.k8s.io
- Calabro F. — The Service Mesh: What Every Software Engineer Needs to Know, O'Reilly Report, 2022. oreilly.com — Service Mesh Report
- Envoy — Architecture overview. The data plane of every major mesh. envoyproxy.io