Apache Iceberg Governance: Access Control, Policies, and Audit for Open Lakehouses

In an Apache Iceberg lakehouse, governance is not a built-in feature like in traditional data warehouses. Instead, it is an architecture that must be assembled intentionally. While vendors handle access control, policy enforcement, and audit trails in warehouse solutions such as Snowflake, BigQuery, and Redshift, Iceberg takes a different approach.

The Apache Iceberg table format is open, engine-agnostic, and deliberately governance-unaware. This means any engine can read any table, and no single system owns access control. This openness creates a governance vacuum that must be filled with purpose-built tools, which is one of the most challenging operational problems in production lakehouses.

The Iceberg community made a deliberate choice to keep governance separate from the table format. Governance is someone else's responsibility. They made this choice for three reasons: preserving data portability, enabling evolution of governance requirements, and ensuring verification of enforcement. If Iceberg metadata included access control lists (ACLs) and masking rules, every engine would need to implement the same governance model, making portability impossible.

Additionally, governance requirements change faster than storage formats, and encoding governance in metadata would require metadata migration for every policy change. Lastly, when rules live in the format, enforcement depends on the reader, which can vary and may bypass security measures altogether.

To address these challenges, production Iceberg lakehouses need a three-layer governance model. The first layer is the catalog, which provides data portability and structural integrity. Apache Iceberg itself defines the table format and carries structural metadata that governance systems can leverage but never interprets as rules.

The catalog is the mandatory intermediary between engines and data, enforcing governance mechanisms. The second layer is the catalog enforcement and coordination system, which enforces governance rules defined by the policy engine. The catalog should not be the rule-authoring system; instead, it focuses on enforcing policies. The policy engine defines the rules, versioning, auditing, and distribution of access policies.

This separation allows for independent evolution of each layer, making open lakehouses viable. The third layer is operational governance, which handles table health, maintenance policies, and lifecycle management to ensure governed data is also operationally sound. By understanding the interaction between access governance and operational governance, teams can design effective governance architectures in Iceberg lakehouses.