Polling Consumer

The pull-at-interval consumer — legacy EDI SFTP/FTPS polling.

Problem

How to receive messages from a system without push notification? How to integrate a legacy EDI system that only drops files?

Forces

• Push is elegant but requires an exposed endpoint — not always feasible behind a firewall.
• Polling is simple to implement and debug — a cron + a script.
• Polling consumes resources even with no messages.
• Polling adds latency (poll interval) — not suited to real-time.

Solution

A looping consumer: (1) poll the channel/folder, (2) process new messages, (3) sleep N seconds, back to 1. Simple, predictable, easy to monitor (poll latency, items processed). Typical interval: 30s in batch EDI, 5s in near-realtime EDI.

EDI implementation

In EDI, Polling Consumer is the standard mode for SFTP: a CRON job or Camel route `file://?delay=30000` browses the SFTP folder every 30 seconds, processes each new file. Also used for OFTP polling and legacy VANs. Modernised: Polling Bridge transforms SFTP polling into Kafka push.

Anti-patterns

• Polling interval < 1s with expensive processing — useless overhead.
• Polling without timeout — a massive folder (1000+ files) takes hours per cycle.
• Polling without lockfile — two instances process the same file in parallel.

Related patterns

• Event-Driven Consumer — push alternative.
• Message Endpoint — parent concept.

Sources

• Hohpe G., Woolf B. — EIP, Polling Consumer (p. 494). www.enterpriseintegrationpatterns.com/patterns/messaging/PollingConsumer.html