Amazon S3 Tables Alternatives for Production Iceberg

Amazon S3 Tables is a managed service that combines Iceberg storage and catalog into a single solution. It offers a built-in Iceberg REST Catalog, automatic compaction, snapshot expiration, and orphan cleanup. S3 Tables also supports intelligent tiering for cost optimization and integrates with SageMaker Lakehouse for unified analytics.

For teams with straightforward requirements, such as AWS-native stacks using Athena, EMR, and Redshift, S3 Tables can save weeks of infrastructure setup. However, in production Iceberg estates, the constraints of S3 Tables become apparent.

One major limitation is the lack of visibility into S3 Tables' maintenance operations. Users cannot see what compaction has done, as there are no detailed compaction logs or correlation with query latency. This opacity can pose risks for streaming workloads with freshness SLAs. Additionally, S3 Tables runs maintenance operations continuously, which can lead to commit contention and increased latency for frequent writers.

Snapshot management also has its challenges. If any user-defined tag or branch exists on a table, snapshot management fails for the entire table, requiring deletion of all tags and branches before snapshots can be expired or removed. Table properties for Iceberg snapshot retention can also cause silent failures. While S3 Tables offers higher write throughput compared to general-purpose S3 buckets, it lacks certain features like ALTER TABLE RENAME, which can break dbt's standard near-zero-downtime table swap.