Databricks co-creator unveils Apache Iceberg V4 metadata
Apache Iceberg V4 proposes a major redesign of its open table format metadata to improve scalability, support subsecond commits, and handle new file and vector data types for AI and search workloads. Presented by co-creator Daniel Weeks at QCon San Francisco, the update simplifies the metadata tree structure to reduce I/O overhead and introduces structured statistics and strict schema constraints. However, Weeks confirmed that V4 metadata will not be backward compatible with older readers. Open article →