Change Data Capture (CDC)

The pattern that turns a DB into an event source without modifying the application — the key to EDI integration without intrusion.

Problem

A legacy ERP cannot be modified to publish events on every change (order created, status updated). How to feed the EDI hub in real time without touching application code?

Forces

• Modifying the ERP is expensive and risky (dense business rules, few tests).
• A periodic DB poll misses rapid changes and stresses the DB.
• The DB transaction log already contains every change — not reusing it is waste.
• CDC must handle initial snapshot + incremental without missing a change between the two.

Solution

Plug a CDC connector onto the DB transaction log. The connector reads the log, transforms each entry into a structured event, and publishes to a broker. For PostgreSQL: `wal2json`, Debezium PostgreSQL connector. For MySQL: Debezium MySQL via binlog. For SQL Server: Change Tracking + Debezium. The pattern is non-intrusive (zero application change) and guarantees at-least-once delivery on the broker.

EDI implementation

In EDI, CDC is the reference path to integrate a non-Kafka-native ERP into a modern hub. Typical case: SAP ERP on Oracle 19c → Debezium Oracle connector → Kafka topic `sap.orders.cdc` → Camel transformation → EDIFACT ORDERS publish to the partner. The pattern also works reverse to materialise read models. Combines well with Outbox (Outbox on the outbox table + CDC on WAL = atomic application write, broker publish derived).

Anti-patterns

• CDC on tables without a primary key — Debezium cannot guarantee ordering / uniqueness.
• CDC with a missing snapshot — pre-existing rows are absent from the broker.
• CDC on a large table without throttling — the broker saturates.
• CDC exposing PII fields without filtering — GDPR leak.

Related patterns

• Outbox (architectural) — CDC on the outbox table is the standard idiom.
• Message Translator — transformation of CDC events into EDI messages.
• Event Message — message style produced by CDC.
• Guaranteed Delivery — at-least-once CDC guarantee.

Sources

• Debezium — Distributed CDC platform documentation. debezium.io/documentation/
• Microsoft SQL Server — Change Data Capture overview. learn.microsoft.com/en-us/sql/relational-databases/track-changes/about-change-data-capture-sql-server
• Confluent — Kafka Connect CDC source connectors. docs.confluent.io/platform/current/connect/index.html