Multi-tenant agent memory isolation is a filtering problem, not a ranking problem

Multi-tenancy in agent memory often appears to be just a configuration issue. You add a tenant_id, apply a filter, and move on. However, filtering and vector search are fundamentally different operations, and the sequence in which they occur determines whether isolation remains deterministic. Approximate nearest neighbor (ANN) search returns the k closest vectors that fit within a given time budget.

If an index contains every tenant's memories and a tenant relies on its own rows ranking highest, isolation hinges on statistical tendencies. This holds true until two customers in the same vertical write nearly identical facts, such as primary datastore, deployment location, and on-call rotation schedule. These facts embed nearly identically in the vector space, making it impossible for the ranking system to distinguish between tenant A's copy and tenant B's.

Relational databases addressed this problem two decades ago through row-level security. However, embedding embeddings does not alter the core issue; it merely shifts the enforcement point from the database to an index that was never designed for security purposes. The control mechanism for memory read operations involves a fixed set of steps: embedding the query, applying the tenant (and optionally user) filter, executing the ANN search, optional reranking, and injecting the result into the prompt.

The filter must precede the search; otherwise, a leak occurs when tenants share similar characteristics. There are two primary places to apply the tenant filter: post-filtering runs the ANN search across the entire index, selects the global top-k results, and subsequently eliminates rows from other tenants. Two issues arise: shorter results if the global top ten includes three of your tenant's rows, leading to inferior answers rather than error messages, and more significant security concerns - tenant A's request reads and scores other tenants' vectors before discarding them.

Pre-filtering incorporates the constraint directly into the search, ensuring the candidate set exclusively comprises rows the caller can access. Most vector stores support pre-filtering: Watch out for Qdrant, which uses extra graph edges on payload-indexed fields to prevent filtered HNSW from deteriorating; Pinecone offers namespaces, a hard partition instead of a filter, requiring one namespace per tenant; pgvector provides a filtered HNSW, but earlier versions (before 0.8) might return fewer than k rows in a filtered scan; iterative scans offer a trade-off between predictable latency and correct row counts.

Regardless of the chosen strategy, enforce the implementation in the retrieval layer itself rather than within the agent. If scoping depends on the model incorporating the correct filter in a tool call, any change to the prompt becomes a change to your security posture, and prompts do not undergo the same rigorous review as schema migrations.

There are three ways to partition tenants and their respective trade-offs: a single index with a tenant filter, whose strength depends solely on the applied filter; indexing or collections per tenant, which provides stronger boundaries but demands more resources (creating, monitoring, and migrating thousands of indexes); and hash-sharding tenants across N indexes, with filtering occurring within each shard.

The choice depends on the number of tenants you manage. Once decided, document the decision, as altering it later necessitates re-embedding all data. Deleting a person from the system is not as straightforward as deleting a tenant. When a tenant is removed, derived data such as summaries, profiles, and analytics may not be automatically purged, leading to lingering data remnants.

Deleting one person within a tenant is even more complex, as the data's provenance may no longer explicitly reference the individual. In such cases, relying on lineage to identify and delete the relevant data or conducting a manual review becomes necessary, which may not effectively address GDPR Article 17 requests. Implementing a soft delete followed by a hard purge on a scheduled basis offers a more robust approach.

Start with a soft delete, which allows recoverability and caching of tombstones, before proceeding with a hard purge. Addressing potential failure modes is crucial: isolating tenants based on ranking can lead to breaches when two tenants share similar attributes, post-filtering defaults to returning short result sets that inadvertently incorporate other tenants' vectors, and a lack of provenance on writes hinders the ability to trace back to the originating tenant.

To achieve a secure and reliable system, the tenant boundary must always function as a pre-filter before ranking occurs. Ensure that tenant_id is mandatory, apply it as a pre-filter within the search, maintain raw client data within a retrieval layer inaccessible to the agent, retain sufficient provenance for lineage-based deletions, and incorporate a CI test that seeds look-alike tenants to verify that no cross-tenant data is present.

These measures align with extending database tenancy principles to vector storage systems and should be implemented proactively before scaling to prevent future complications.