Audit template
ai-llm
Audit your LLM features the way an attacker and an honest user both would.
Maps to: OWASP LLM Top 10OWASP Top 10 for LLM Applications — the reference list of the most critical LLM security risks. · NIST AI RMFNIST AI Risk Management Framework — the U.S. standard for governing AI system risk.
specialists, in parallel
Each finding is evidence-bound and survives ≥2-of-3 adversarial skeptics.
How this audit works
Provider- and framework-agnostic, this audit first maps every place an LLM is called and every trust boundary where untrusted input — user text, retrieved documents, tool results — enters a prompt. Twelve specialists then probe prompt injectionAn attack that smuggles instructions into model input to override or hijack the system prompt., jailbreaksA prompt crafted to bypass a model's safety guardrails and make it produce restricted output., system-prompt and secret leakage, insecure output handling, tool/agent agency, RAGRetrieval-Augmented Generation — giving an LLM retrieved documents at query time. grounding, hallucinationA model output that states fabricated or unsupported information as if it were fact., evalsRepeatable test suites that measure an LLM feature's quality and safety on known cases., and cost, each finding mapped to the OWASP LLM Top 10OWASP Top 10 for LLM Applications — the reference list of the most critical LLM security risks. and severity-scored P0–P3. Every claim is traced to a concrete artifact and survives independent skeptics before it ships.
Use it when
Shipping a RAG support bot
Your assistant answers from internal docs and a shared knowledge base. The audit traces whether retrieval respects per-user permissions, whether the bot abstains or hallucinates when there is no good context, and whether instructions hidden in a retrieved document can override the system prompt (indirect injectionPrompt injection where the malicious instructions arrive via retrieved content rather than the user.).
Giving an agent real tools
Your agent can email, query the database, or call internal APIs on model say-so. The audit checks each tool's blast radiusThe set of systems and data affected when one component fails or is compromised. — can it take a destructive or irreversible action without a human gate? — validates the arguments the model generates, and traces every output sink for XSSCross-Site Scripting — injecting attacker-controlled script that runs in another user's browser., SQL, or eval injectionFeeding untrusted text into a code-evaluation call so it executes as program code..
Before scaling a public LLM endpoint
You are about to open an AI feature to untrusted traffic. The audit looks for the per-user and global cost caps that stop runaway spend and abuse, the evalsRepeatable test suites that measure an LLM feature's quality and safety on known cases. that guard your high-stakes paths, and the provider retention and PIIPersonally Identifiable Information — data that can identify a specific individual. handling you need before user data leaves your machine.
What you get
A per-dimension scorecard, a trust-boundary and data-flow map, and verified findings filed as priority-sorted GitHub issues — each with OWASPOpen Worldwide Application Security Project — the reference catalog of web-application security risks. LLM mapping, a redacted repro, and a concrete before/after fix.