Permission Filters and pgvector: Why Your Most Restricted Users Get the Fewest Answers

When building AI search over documents with restricted access, permission filters play a crucial role. In PostgreSQL with pgvector, this usually involves a WHERE clause based on the caller's groups and an ORDER BY on vector distance. However, when an HNSW index is added for speed, a subtle issue arises: users with the fewest permissions get the fewest answers, without any errors or data leakage. The mechanism is measured, and two fixes are provided.

Chunked documents have associated embeddings, and documents carry the groups allowed to read them. A search for a specific caller looks like this: SELECT c.id FROM chunk c JOIN document d ON d.id = c.document_id WHERE d.allowed_principals && ARRAY[group:hr] ORDER BY c.embedding = $1 LIMIT 20; The && operator checks if the caller holds any one of the document's groups. Without a vector index, pgvector performs an exact nearest neighbor search, providing perfect recall and returning 20 rows whenever 20 readable chunks exist.

With an HNSW index, filtering is applied after the index is scanned. The scan produces a fixed list of candidates, sized by hnsw.ef_search, which defaults to 40. The WHERE clause then runs on this list. If the caller can read only 10% of the rows, approximately 4 of the 40 candidates survive. This was measured on 100,000 chunks across 10,000 documents, with 10% readable by group:hr and the remaining 90% by group:staff.

Using random 64-dimensional vectors and an HNSW index with default settings, the same 40 queries were run for each caller, with LIMIT 20.

The key difference lies in who asks the query. Developers typically test with broad accounts, which receive full results. In contrast, restricted users, such as new hires or contractors, receive shorter lists. This short list may not appear as an error to AI assistants or users, potentially leading to incorrect information being provided. To address this issue, two fixes are proposed.

The first fix involves iterative index scans. Since pgvector 0.8.0, this feature has been added. When the filter leaves too few rows, the scan continues through the index instead of stopping at the candidate list. By enabling iterative scans, a caller with limited permissions retrieves more results. However, this comes at the cost of increased execution time.

For example, the median query time increased from 1.7 ms to 27.8 ms for the HR group using strict_order, and from 1.7 ms to 18.1 ms for the HR group using relaxed_order.

The second fix involves giving narrow permissions an exact path. A third group, group:legal, was added, readable on only one document in a thousand (0.1% of the chunks). By default, the planner opts for the most efficient path, using the GIN index on allowed_principals to find readable documents, joining their chunks, and sorting them by exact distance.

This yields 20 of 20 results on every query, with a median of 33 ms. However, when the HNSW index is used with iterative scans and default limits, the median drops to 16 of 20, with the worst query returning only 10 results.

To prevent this issue, an exact index on the permission column is recommended. With this index in place, the planner has a complete and efficient path for narrow callers. This can be confirmed with an EXPLAIN query, using the most restricted real user as the test subject rather than a broad administrator account. Including cases where the retrieval engine runs as a restricted group in test sets can help catch this problem early.