Security resource
AI Security Testing Checklist - OWASP Plus LLM-Specific Coverage
Most pen tests miss prompt injection, retrieval poisoning, and agent abuse. This checklist captures the test plan we run for AI platforms so you can prove coverage before audits, launches, or enterprise deals.
Pre-test alignment
Clarify scope, owners, and evidence before you start.
- Targets: UI, API, chat endpoints, uploads, RAG pipeline, vector stores, MCP servers, tools.
- Tenants/roles in scope and how RLS is asserted; test accounts prepared.
- Data sensitivity (PII/PHI/PCI) and redaction/retention expectations for logs and traces.
- Evidence format: scripts, payloads, screenshots, CVSS/rubric scoring, control mapping.
- Fix deadlines and retest expectations per severity; who approves closure and communicates to customers.
Baseline OWASP testing
- Authentication: MFA bypass, session fixation, token replay, device management.
- Authorization/IDOR: tenant boundary attempts across APIs, admin panels, download endpoints.
- Input validation: SQLi, NoSQLi, command injection, template injection for classic surfaces.
- CSRF/XSS testing, CSP validation, header hardening, TLS review.
- Dependency scanning, container/IaC scanning, secret exposure detection.
Prompt injection & RAG testing
- Direct, indirect, and multi-turn prompt overrides via UI, API, and uploads.
- Retrieval poisoning to leak other tenants' data or insert malicious citations.
- Context flooding/stuffing to force models to ignore instructions or choose attacker payloads.
- Eval harness replay of curated adversarial corpora in CI and pre-prod.
- Logging verification: prompts/responses stored securely with redaction + access controls.
Agent & MCP abuse testing
- Tool gating bypass attempts, privilege escalation through tool arguments.
- Rate limiting and quota enforcement for high-frequency agents.
- Sandbox escapes for code execution tools (Docker, Firecracker, Vercel Edge).
- Audit log completeness: every tool invocation correlated with user session + outcome.
- Session hijacking across chained agents, especially in multi-user workflows.
Compliance & evidence
- Documented severity scoring (CVSS or internal rubric) mapped to remediation owners.
- Proof-of-concept payloads captured (screenshots, Postman collections, CLI scripts).
- Regression automation (ZAP, Nuclei, custom prompt suites) wired into CI/CD.
- Control mapping for questionnaires (SOC-style templates, vendor security reviews).
- Retest plan including acceptance criteria before closing findings.
Evaluation & automation
- Offline evals on golden sets for accuracy/citations; thresholds gate releases.
- Adversarial prompt suites (jailbreaks, data exfil) replayed in CI and pre-prod.
- Semgrep/Playwright/Nuclei/ZAP pipelines for regression; false-positive triage process.
- Telemetry validation: logs/traces include tenant, request ID, model, tool, and document IDs.
- Alerting: anomalies on cost, latency, retrieval patterns, and agent/tool usage.
Runbooks & containment
- Leak response: revoke keys, disable indexes, invalidate caches, notify tenants.
- Poisoning response: quarantine sources, re-embed clean data, add approval gates.
- Hallucination response: degrade to retrieval-only answers, tighten prompts, citations-only mode.
- Cost overrun response: cap K/context, switch to cheaper models, batch rerankers.
- Who pages who: security, SRE, product, and customer comms owners listed.
Download the testing workbook
Get the editable PDF/Notion version with severity scoring, owner assignments, and evidence links.
Ungated content above. Email optional for template. No spam.
Related resources
- Penetration Testing AI Platforms — methodology and expectations.
- LLM Security Guide — model-specific threats and mitigations.
- AI Platform Security Guide — platform-wide controls and observability.
- Penetration Testing Service — hire us to execute and remediate.
Need help running these tests?
We build the platforms and conduct the penetration tests. If you want a senior engineer to execute this checklist and deliver remediation guidance, let's talk.
FAQ
What is this for?
Teams that need a clear test plan before launching AI products-covering both OWASP classics and AI-specific attack vectors.
FAQ
Do I need to hire you to use it?
No. It's ungated so your internal teams can execute. Work with us if you want a senior engineer to run the suite and deliver remediation.
FAQ
Can I get a PDF template?
Yes-opt in below for the PDF + Notion version with severity scoring, owner columns, and evidence links.