Enterprise RAG is rarely about quality. It's about access control, tenancy, and audit.
The three things every enterprise RAG must enforce
- Tenant isolation. A user from Org A must never retrieve a chunk owned by Org B.
- Document-level access control. Within a tenant, user permissions on the source doc must propagate to retrieval.
- Auditability. Every retrieval must be logged with (user, query, retrieved IDs, timestamp).
Get any of these wrong and your RAG is a data leak waiting to happen.
Tenant isolation patterns
| Pattern | How | Pros | Cons |
|---|---|---|---|
| Namespace per tenant | Pinecone namespace, Weaviate class, Qdrant collection | Clean separation, often best perf | Expensive at thousands of tenants |
| Filter by tenant_id | Metadata filter at query time | Cheap, flexible | Filter performance varies; bug → leak |
| Index per tenant | Separate index per tenant | Strongest isolation | Operational nightmare past 50 tenants |
For SaaS with hundreds-thousands of tenants: namespace per tenant is the production default. Filter-based works but is one bug from a leak.
Document-level ACLs
Hard problem. Three patterns:
A. Filter at retrieval
Each chunk has { tenant_id, doc_id, acl: [user_ids_or_groups] }. At query time, filter acl includes user_id_or_group. Works if your group memberships are stable and small.
B. Per-user index
Each user gets their own index containing only docs they can see. Update on permission changes. High write cost; trivial read security.
C. Re-rank by access
Retrieve broadly, then drop chunks the user can't see. Dangerous — leaks even existence of forbidden docs via timing/quota. Avoid.
Pattern A is the standard for IBM-scale enterprise. Use a fanout for group membership: cache user → groups, then tenant_id = X AND acl IN [user_id, *user_groups].
The hardest case: time-changing permissions
A user loses access to a doc after retrieving it. Standard answer: stale retrievals are acceptable in cache windows < N seconds; long-term, re-check ACL on every retrieval.
Multi-tenant embedding strategies
- Shared embedder, namespaced indices — usual answer.
- Per-tenant fine-tuned embedder — only when tenant data is so different it justifies the ops cost. Rare.
Audit logging
Log every (user_id, query, retrieved_chunk_ids, scores, final_answer, timestamp). For regulated industries, append to immutable storage (WORM). Retain per regulatory requirements (HIPAA: 6 years, GDPR: contract-driven).
Encryption
- At rest — your vector DB / metadata store encrypts.
- In transit — TLS, internal too.
- At query time — for highly sensitive: customer-managed keys (CMK) or homomorphic encryption (research-only at production scale today).
Practical IBM watsonx.ai pattern
- watsonx.data lakehouse for source docs with native ACL.
- Embeddings indexed in Milvus / Watson Discovery with ACL metadata.
- Granite for generation, with prompt log → watsonx.governance.
- All within OpenShift / Cloud Pak — no data leaves the cluster.
Common mistakes
- No ACL on chunks. Doc-level ACL exists, chunks inherit nothing → retrieval bypasses ACL.
- Tenant_id in prompt only. Model "promises" not to cross tenants. Trust based on prompt = guaranteed leak.
- No audit on retrieval. Only generation is logged → can't reconstruct what was leaked when there's an incident.