Security resource
RAG Security Audit Checklist - Multi-Tenant Retrieval Safety
Most RAG implementations ship with retrieval vulnerabilities-tenant data leakage, embedding inversion risks, and document poisoning vectors. This checklist covers the security controls we audit before launching RAG-powered features.
Pre-audit scope definition
Define what's in scope before auditing.
- Document sources: file uploads, integrations, APIs, scrapers, user-generated content.
- Vector store technology: pgvector, Pinecone, Weaviate, Qdrant, Chroma, or custom.
- Embedding model: OpenAI, Cohere, local models-and where embeddings are computed.
- Multi-tenant model: shared collection with metadata filtering, tenant-specific collections, or hybrid.
- Retrieval flow: who can query, how results are filtered, what context reaches the LLM.
Vector database security
- Tenant isolation: Metadata filtering applied BEFORE similarity search, not after.
- Collection/namespace separation: High-sensitivity tenants in dedicated collections.
- Access control: API keys scoped to tenant, not shared across the platform.
- Encryption at rest: Vector data encrypted, not just source documents.
- Backup isolation: Tenant data doesn't leak across backup boundaries.
Retrieval safety controls
- Pre-retrieval filtering: tenant_id, access_level, classification enforced before vector search.
- Result validation: Retrieved chunks validated against user permissions before LLM context.
- Chunk metadata integrity: Metadata can't be spoofed or modified by uploaders.
- Cross-tenant query testing: Adversarial queries attempting to retrieve other tenants' data.
- Cache isolation: Retrieval caches keyed by tenant, not shared globally.
Document ingestion security
- Source validation: Documents from trusted sources only, with provenance tracking.
- Content scanning: Malware, hidden text, steganography detection before embedding.
- Metadata sanitization: User-supplied metadata validated and escaped.
- Poisoning detection: Anomaly detection for documents that contradict existing corpus.
- Deletion propagation: Source document deletion removes all derived chunks and embeddings.
Embedding security
- Inversion risk assessment: Can embeddings be reversed to reconstruct source text?
- PII in embeddings: Sensitive data stripped before embedding, not just from display.
- Model versioning: Re-embedding strategy when switching models.
- Embedding API security: Rate limiting, authentication, tenant scoping on embedding endpoints.
- Local vs. remote: Sensitive documents embedded locally, not sent to third-party APIs.
Multi-tenant RAG controls
- Row-Level Security: RLS policies on vector tables enforcing tenant_id at database level.
- Middleware enforcement: tenant_id injected from authenticated session, never from user input.
- Query logging: All retrieval queries logged with tenant, user, query, and results.
- Noisy neighbor protection: Per-tenant rate limits and resource quotas.
- Tenant offboarding: Complete data deletion including embeddings, chunks, and logs.
Prompt injection via RAG
- Indirect injection testing: Upload documents with hidden instructions, verify they don't execute.
- Context isolation: Retrieved content clearly delimited from system instructions.
- Citation validation: LLM responses cite actual retrieved chunks, not hallucinated sources.
- Output filtering: Responses checked for PII, credentials, or system prompt leakage.
- Retrieval limiting: Cap on retrieved chunks prevents context flooding attacks.
Monitoring & observability
- Retrieval telemetry: Query latency, result count, similarity scores logged per request.
- Anomaly detection: Alerts on unusual query patterns, cross-tenant access attempts.
- Cost tracking: Embedding and retrieval costs attributed to tenants.
- Audit trail: Who queried what, when, and what was returned-immutable logs.
- Drift detection: Retrieval quality degradation alerts (precision/recall metrics).
Compliance & evidence
- Data residency: Vector stores and embeddings comply with geographic requirements.
- Retention policies: Embeddings and chunks follow same retention as source documents.
- Right to deletion: GDPR/CCPA deletion requests propagate to all derived data.
- Third-party risk: Embedding API providers assessed for security and compliance.
- Penetration test evidence: RAG-specific test cases documented with findings.
Download the RAG security workbook
Get the editable PDF/Notion version with severity scoring, owner assignments, and evidence templates.
Ungated content above. Email optional for template. No spam.
Related resources
- RAG Architecture Guide - retrieval patterns and semantic search.
- AI Platform Security Guide - multi-tenant architecture and controls.
- Multi-Tenant SaaS Architecture - RLS and tenant isolation patterns.
- AI Security Testing Checklist - broader platform security coverage.
- Penetration Testing Service - hire us to execute and remediate.
Need help auditing your RAG system?
I build RAG systems and audit them for security. If you want a senior engineer to run this checklist against your implementation and deliver remediation guidance, let's talk.
FAQ
What is this for?
Teams launching RAG-powered features who need to audit retrieval security, tenant isolation, and data pipeline safety before production or enterprise deals.
FAQ
Do I need to hire you to use it?
No. It's ungated so your internal teams can execute. Work with us if you want a senior engineer to run the audit and deliver remediation.
FAQ
How is this different from the AI Security Checklist?
This checklist focuses specifically on RAG systems-vector stores, retrieval pipelines, embedding security, and document ingestion. The AI Security Checklist covers broader platform security.