ediverse

Type to search across standards, messages, tools and glossary.

ediverse Explore the platform

Spotlight PEPPOL BIS Billing 3.0 The EU e-invoicing mandate is here — France Sept 2026, Belgium Jan 2026, Germany 2025.

— 01

Foundations

— 02

Standards

— 03

Flagship messages

— 04

Tools

Token Bucket Rate Limiter

The Rate Limiter algorithm allowing controlled bursts without giving up on a steady mastered throughput: the detail turning policy into a readable number.

Problem

The basic Rate Limiter limits per fixed window ("100 requests per minute"). Problem: at the window edge, one can fire 100 requests at 12:00:59 and 100 more at 12:01:00, that is 200 in one second — saturation. Another problem: enforcing "never more than 2 per second" rejects a legitimate batch of 50 INVOIC in a second that averages perfectly over the day. An algorithm distinguishing sustained rate and burst, without temporal edge effect, is needed.

Forces

  • Legitimate peaks exist. Month-end, morning synchronisation, partner batches.
  • Sustained throughput must be bounded. Over 24h we do not want to exceed X million requests.
  • The computation must be simple. The Rate Limiter runs before any processing, must be O(1).
  • Partner SLA must be computable. "You can burst to 500, sustained rate 100/s" is contractual.

Solution

Implement a token bucket of maximum capacity N (allowed burst size), refilled at rate R tokens/sec (sustained rate). Each request consumes 1 token (or k tokens depending on cost). If the bucket is empty, the request is rejected immediately (HTTP 429, Retry-After). The calculation on a request at time t: tokens = min(N, tokens_prev + (t - t_prev) × R), then tokens -= 1 if allowed. O(1) algorithm, only two floats stored per counter. Implementations: Bucket4j (Java), Cloudflare Edge Workers, Kong, NGINX limit_req_zone, AWS API Gateway. 

   Automatic refill
   R = 100 tokens/sec
         │
         ▼
   ┌──────────────┐
   │  Bucket N=500│  max capacity (burst)
   │  ●●●●●●●●●● │  current tokens
   │  ●●●●●●●●●● │
   └──────────────┘
         │ consume 1 per request
         ▼
   request to partner
         │
   if bucket empty:
   ┌──────────────────────────┐
   │ HTTP 429 Too Many Req    │
   │ Retry-After: N           │
   └──────────────────────────┘

EDI implementation

Concrete case: an EDI hub REST API receives INVOIC from a Walmart partner. The contract says: 100 INVOIC/sec sustained, burst 500. On Kong (API Gateway), plugin rate-limiting-advanced in sliding window mode or a Lua token-bucket module: limit=500, rate=100/s, identifier=consumer.partner_id. Walmart sends a 480-message batch at 14:32:00: all pass (500-token bucket, 480 consumed). At 14:32:01, 20 + 100 refilled = 120 remain, etc. If Walmart pushes 600 at 14:32:00, the last 100 get HTTP 429 Retry-After: 1. On the Kafka producer: quota.producer.byte.rate per client.id, token bucket equivalent on bytes. On PEPPOL: OpenPEPPOL standard imposes a maximum rate per Access Point — typically token-bucket implemented.

Practical parameters

  • R (refill rate): the desired sustained per-second average. Must match downstream capacity.
  • N (bucket size): how much burst is tolerated. Often N = 5×R to 10×R.
  • Identifier: per partnerId, per apiKey, per IP. Pick the contractual key.
  • Storage: Redis cluster for multi-instance sharing (with Lua SCRIPT for atomicity), local memory if single-node.
  • Response on reject: HTTP 429, Retry-After: N (RFC 6585), header X-RateLimit-Remaining.

Anti-patterns

  • Token bucket without persistence. Service restart → bucket reset → free burst.
  • Multi-instance token bucket without coordination. Each instance has its bucket: the partner can exceed k×R by hitting k instances. Centralise via Redis.
  • Burst N too large. If N=10000 for R=100, the pattern has no effect — the max burst always saturates.
  • Partner-independent R. A single global rate limit for everyone: a chatty partner consumes everyone's quota.
  • Rate Limiter — the parent pattern.
  • Back-pressure — the rate limiter is a downstream contractual back-pressure.
  • Bulkhead — paired: one token bucket per compartment.
  • Circuit Breaker — complementary: if downstream fails, do not consume tokens.

Sources