Navigating Multi-Tenant RAG in SaaS: An Essential Handbook

Multi-tenant RAG systems require careful isolation of tenant data to prevent leaks and ensure security. While partitioning the vector store is a common practice, it is essential to understand that isolation is a system property, not just a database setting. Customer data can reside in various places beyond the vector store, including agent memory, semantic caches, model-provider logs, observability traces, and evaluation datasets. Without proper controls at each of these layers, there is a risk of one customer’s information being exposed to another.

Isolation strategies should consider all the places where tenant data can reside, not just the vector store. Before selecting a partitioning strategy, it is crucial to identify all the stores where tenant content ends up. This includes agent memory, semantic caches, evaluation sets, and fine-tuned adapters, among others.

Three common isolation patterns for multi-tenant systems are Pool, Bridge, and Silo. Pool involves filtering data at query time, Bridge uses shared infrastructure with structural boundaries, and Silo offers dedicated resources for each tenant. It is common for SaaS companies to implement a combination of these patterns to cater to different types of tenants.

The recall problem in post-filtering shared indexes can lead to a significant decrease in retrieval quality, especially with a large number of tenants. It is important to test the filtering behavior of the retrieval engine and verify empirically to ensure accurate results for each tenant.

Provider caches can pose a risk of data leakage between tenants if not scoped properly. It is essential to understand the cache sharing policies of API providers and take measures to ensure that tenant data is isolated at the organization level.

Isolation and deletion go hand in hand, as the ability to confidently delete all data belonging to a tenant demonstrates true isolation. It is crucial to have a comprehensive offboarding process that covers all stores where tenant data may exist.

To prove isolation in a security review, it is recommended to build a cross-tenant probe suite that tests for data leakage between tenants on every release. This suite should cover various read paths and assert on content rather than status codes.

Handling noisy neighbors in a multi-tenant vector database requires implementing rate-limiting per tenant and monitoring query latency to identify potential issues. It is important to attribute model spend per tenant from the beginning to ensure cost visibility and scalability.

In conclusion, designing isolation in a multi-tenant RAG system is essential for maintaining data security and preventing leaks. By understanding where tenant data resides, implementing appropriate isolation strategies, and testing for data leakage, SaaS companies can ensure the integrity of their AI systems.