APACHE-ICEBERG

Data lake table format with ACID + time-travel.

Definition

Iceberg structures its tables as metadata.json → manifest list → manifests → data files. Spec v2 adds row-level deletes, v3 (draft 2025) DataFusion column statistics. Catalogs: REST, Hive, Glue, Nessie, Snowflake Polaris. Compatible with Spark, Trino, Flink, DuckDB.

Origin

Created by Netflix in 2017 ; donated to Apache Software Foundation in May 2020 ; top-level project in November 2020.

Example in context

Apple stores 100 PB of analytics in S3 as Iceberg with a Nessie catalog to ensure analytical reproducibility.

Related terms

• Delta Lake — Databricks competitor.