I Said Isolation Was Structural. Then Tenancy Shipped and Proved Me Right the Hard Way

In the latest installment of the v0.1.0 series, a statement was made that isolation is structural or it is imaginary. This assertion was later proven to be accurate through rigorous testing. The first multi-tenancy release, v0.2.0, was shipped for HivePlane. Subsequently, an adversarial suite was run against the system, revealing seven findings, or nine if the combined findings are considered. These findings confirmed that the system was inferring tenancy rather than asserting it.

Each instance of inferred tenancy was due to a place where the system incorrectly used a client-supplied value as an identity, rather than enforcing isolation at the storage primary key and the authenticated principal. The vulnerabilities included a worker hijack, where a workload registered under one tenant could be accessed by another tenant using a worker ID alone.

Another issue involved a spoofable header used for rate limiting, which could be manipulated by an attacker to consume another tenant's resources. Additionally, approvals, security events, and incident states were found to default to a default tenant, allowing cross-tenant access to sensitive information.

The source also highlights that default tenants are not a safe default and can lead to silent cross-tenant leaks. To address these issues, composite primary keys should be used, TenantScopeError should be raised on cross-tenant writes, and adversarial field tests should be conducted to attack each boundary instead of testing only happy paths. The article emphasizes that inference is not enforcement and that composite primary keys, proper error handling, and rigorous testing are the keys to ensuring true tenancy isolation.