Step Functions (managed declarative workflow)

Describe a workflow in declarative JSON (Amazon States Language) and let a managed service (AWS Step Functions, Google Cloud Workflows) execute, retry, parallelise. The infra-free workflow pattern.

Problem

Maintaining a Temporal cluster, a Camunda cluster or a self-hosted Airflow demands a dedicated ops team: monitoring, upgrades, backup, scaling, security. For moderately complex EDI workflows (10-30 steps), operational overhead exceeds value. One would like to simply define the workflow and let the cloud provider run it.

Forces

• Maintaining a workflow engine has a cost: install, monitor, upgrade, scale.
• EDI workflows can be many but simple: 10-30 steps, without sophisticated logic.
• Native cloud integration is valuable: Lambda, S3, SQS, SNS chain naturally.
• Declarative format eases audit: ASL JSON reads, versions, diffs cleanly.
• Usage-based billing: no cost without flow, horizontal scaling managed.

Solution

AWS Step Functions defines a workflow in Amazon States Language (ASL), a typed JSON subset. Each state is a Task (Lambda, ECS, etc. call), Wait (timer), Choice (branching), Parallel (parallel execution), Map (fan-out), Pass, Succeed, Fail. The AWS runtime executes, manages retries (with declarative policy), timeouts, persistence. Google Cloud Workflows offers the equivalent with a similar YAML. Azure Logic Apps is a cousin with visual UI.

ASL JSON definition of an EDI workflow:

{
  "StartAt": "ParseInvoice",
  "States": {
    "ParseInvoice": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:...:parse-invoice",
      "Next": "ValidateInvoice"
    },
    "ValidateInvoice": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:...:validate-invoice",
      "Retry": [{ "ErrorEquals": ["States.ALL"],
                  "IntervalSeconds": 2,
                  "MaxAttempts": 3,
                  "BackoffRate": 2.0 }],
      "Catch": [{ "ErrorEquals": ["ValidationError"],
                  "Next": "SendApErak" }],
      "Next": "RouteToErp"
    },
    "SendApErak": { "Type": "Task", ..., "End": true },
    "RouteToErp": { ..., "Next": "AwaitErpAck" },
    "AwaitErpAck": { "Type": "Wait", "Seconds": 3600,
                      "Next": "CheckAck" },
    "CheckAck": { "Type": "Choice", ... },
    ...
  }
}

The AWS Step Functions runtime executes this JSON,
manages retries, timeout, parallelism, parallel branches.

EDI implementation

Concrete case: incoming INVOIC processing workflow. Each step is a Lambda: parse-invoice, validate-invoice, route-to-erp, await-erp-ack, archive-to-s3. The ASL JSON defines the sequence, retries (3 attempts exponential backoff), error catches (validation error → SendAperak → Notification). Everything is versioned in GitOps. Deployment is via Terraform with aws_sfn_state_machine resource. Cost: ~25¢ for 1000 state transitions (Standard workflow) or 1¢ for 1000 transitions (Express workflow). For 100k invoices/month with 10 transitions each, cost ~$250/month in Standard. The AWS Step Functions console visualises each execution with detailed timeline. For more complex workflows (rich conditional logic, custom code), Temporal stays preferable. For simple cloud-native workflows, Step Functions is unbeatable in TCO.

Anti-patterns

• Workflow too voluminous: 5000-line ASL JSON for a 200-state workflow → unmanageable, unreadable. Split into sub-workflows.
• Business logic in the workflow: ASL is not a programming language. Put logic in Lambdas, keep the workflow pure orchestration.
• No versioning: modifying in the console without Git → impossible audit, painful rollback.
• Standard cost for short workflows: Standard bills each transition dearly for workflows < 5min. For these cases, Express Workflows.
• Cloud lock-in: ASL is AWS-proprietary, Google Cloud Workflows and Azure Logic Apps have different syntaxes. Cloud-agnostic architecture difficult.
• No local tests: hard to test a workflow without deploying. Use Step Functions Local for testing.

Related patterns

• Process Manager — parent EIP pattern.
• Temporal Workflow — more powerful code-first variant.
• Process Engine — parent pattern, here in managed SaaS mode.
• Saga Orchestration — pattern naturally implemented in Step Functions.
• Compensating Action — pattern explicitly coded via Catch in ASL.

Sources

• AWS Step Functions Developer Guide. Official reference. docs.aws.amazon.com — Step Functions
• Amazon States Language Specification. ASL formal grammar. states-language.net — spec
• Google Cloud Workflows Documentation. Google Cloud equivalent. cloud.google.com — workflows
• Azure Logic Apps Documentation. Microsoft Azure equivalent. learn.microsoft.com — logic-apps
• AWS Architecture Center — Serverless workflow patterns. Step Functions usage patterns for cloud orchestration. docs.aws.amazon.com — serverless-workflow-patterns
• Richardson C. — Microservices Patterns, Manning 2018. Chapter 4 on Sagas and their orchestration implementation; Step Functions is treated as an implementation.