Message Store

A broker is not long-term storage. The Message Store persists every message — inbound, outbound, intermediate — in an indexed object store for audit, replay, history and fiscal proof.

Problem

A message broker (Kafka, RabbitMQ, Service Bus) is designed for short residence times: hours to days. But EDI requires preservation for 5 to 10 years (invoices), 3 to 5 years (orders, dispatches), sometimes longer in litigation. And the business wants to replay what happened a month ago — to answer an audit, understand a discrepancy, replay a lost day. Without Message Store, those capabilities do not hold: the broker has already deleted.

Forces

• Decouple lifetime. The broker keeps for processing time; the store keeps for legal time.
• Index for search. Finding "all INVOIC from Carrefour in March 2026" needs an index (Elasticsearch, OpenSearch, Postgres + GIN, etc.).
• Replay without contamination. To replay an INVOIC in 2031, read from the store, reinject into the pipeline — do not contaminate the broker's history.
• Fiscal proof. The store is the evidence produced during an audit. Hash, timestamp, detached signature when possible.

Solution

EIP §555 (Hohpe & Woolf, 2003) defines the Message Store as a component that listens on every pipeline channel and persists each message — often via a Wire Tap so as not to perturb the flow. The typical store combines: (a) an object store for the binary payload (S3, Azure Blob, GCS), (b) a relational or Elasticsearch index for metadata and search, (c) a retention policy driven by message type. The store serves three use cases: audit (forensic read), replay (reinjection into the pipeline), history (portal lookup).

┌───────┐         ┌───────┐         ┌────────┐
   │ src   │ ───▶   │ pipe  │ ───▶    │ dest   │
   └───────┘         └───────┘         └────────┘
                        │
                        │ tap (copy)
                        ▼
                   ┌──────────┐
                   │ message  │
                   │  store   │  ──▶ replay, audit, search
                   └──────────┘
                   (object store + index)

EDI implementation

The typical EDI Message Store stores each message at three stages: raw (as received from partner), canonical (after parsing & validation), outbound (as delivered to partner or ERP). That answers any audit question from any angle.

# Typical Message Store record
{
  "storeId": "01HVF8K3X1ABCDE0123456789",
  "messageId": "inv-202605140001",
  "interchangeRef": "INV202605140001",
  "type": "INVOIC",
  "format": "EDIFACT D96A",
  "stage": "inbound.raw",                # or inbound.canonical, outbound.partner
  "tenantId": "ediverse-prod",
  "from": "5798000000122",
  "to":   "7300010000001",
  "receivedAt": "2026-05-14T12:00:00.123Z",
  "sizeBytes": 4218,
  "sha256": "9a4b1c2def8b6a5e8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b",
  "storageUri": "s3://ediverse-store/2026/05/14/inv-202605140001.edi",
  "ackChain": [
    { "type": "CONTRL", "receivedAt": "2026-05-14T12:00:05Z" },
    { "type": "APERAK", "receivedAt": "2026-05-14T12:00:42Z" }
  ],
  "retentionUntil": "2036-05-14T00:00:00Z",
  "legalHold": false
}

Reading: each record carries the technical identifier (messageId), business identifier (interchangeRef), pipeline stage (stage), hash and payload URI. The ACK chain (ackChain) is attached to the history to reconstruct the full status. retentionUntil is computed at record time from the type and jurisdiction (10 years for French invoices).

• Fiscal audit. On audit, the operator must restore every emitted INVOIC over 10 years, with cryptographic proof of content and sending.
• Customer dispute. "You never sent that invoice" — the store responds with the exact signed copy, and the MDN/AS2 or NRR/AS4 chain proves reception.
• Replay after bug. A translator bug mis-translated 200 INVOIC over 2 days. Identify affected messages via index, reinject via store pointing to the fixed translator version.
• Migration. When changing integration hub, the store feeds history to the new hub without losing traceability.

Retention and cost

Three levers:

• Storage tiers. Hot ≤ 30 days, Warm 30-365 days, Cold (S3 Glacier, Azure Cool Blob) beyond. 10× cheaper in cold, read latency in hours — acceptable for occasional audit.
• Compression. EDIFACT, X12, UBL compress to 10-20% of original size (repetitive XML, fixed structures). Enable gzip / zstd by default.
• Deduplication. Hash each payload (SHA-256) and store once; share via reference. Especially effective for catalogues that repeat the same lines.

Anti-patterns

• Using the broker as a store. Keeping Kafka in infinite retention may seem economical, but you lose search and tiering capabilities. Keep the broker short, delegate to the store.
• No index. An object store without an index is a cemetery. Search capability defines store value; without it, no audit answer is possible.
• No hash. Without cryptographic fingerprint, legal proof is weak. Always SHA-256 minimum, ideally timestamped by a trusted third party for fiscal flows.
• Uniform retention. Not every message type has the same obligation. Configure by type + jurisdiction.

Related patterns

• Wire Tap — the tool to feed the store without disturbing the flow.
• Message History — the chain of traversed components, stored in the store.
• Claim Check — shares the same object-store + index infrastructure.
• Dead Letter Channel — the DLQ also archives in the Message Store.

Sources

• Hohpe G., Woolf B. — Enterprise Integration Patterns, Message Store (§555). enterpriseintegrationpatterns.com — Message Store
• French General Tax Code, article L102B. Mandatory retention of electronic invoices for 10 years.
• EU — Directive 2014/55/EU and implementing regulation on B2G e-invoicing, imposing archival and proof capabilities over 10 years.
• AWS Architecture Blog — Archival patterns with Amazon S3 Glacier. Modern reference for cold + WORM implementations. aws.amazon.com/glacier