SSOGEN / Learn / LLM Security: Where Each Risk Enters and Lands
Learn
LLM Security: Where Each Risk Enters and Lands
An injected instruction does its damage at the tool call, so LLM security caps what a fooled model can reach as well as guarding the text that enters and leaves it.

One request crosses five points where text enters or leaves the application: the person’s input, retrieval from a vector store, the assembled context and the model, tool calls, and output that another system renders.
The tool-call point carries the weight. OWASP’s 2026 entry on prompt injection states that “no reliable prevention mechanism exists today”, so the controls that hold are the ones that cap the damage.
The path one request takes
A chat assistant with retrieval and tools makes several hops for one question. At each hop, content from one party becomes input for the next. The figure follows one request; the stage names and risk IDs match the table below.
One request through an LLM application, five stages
Stages 1 to 3: text reaches the model
Stages 4 and 5: model output leaves the conversation
Where each OWASP risk enters and lands
A risk gets in at one stage and does harm at the same or a later one. IDs and names come from OWASP’s Top 10 for LLM Applications 2026. Each risk is described in full in the OWASP LLM top 10 explained.
| Stage | What crosses (example) | Enters here | Lands here | Control at this stage |
|---|---|---|---|---|
| 1 Input | Summarize the attached contract and 40 pages of file text |
LLM01:2026 Prompt Injection (direct); LLM06:2026 Unbounded Consumption | None | Per-caller rate limits, token budgets and input size limits |
| 2 Retrieval | chunk 7, hr/salaries-2026.xlsx, score 0.81 |
LLM01 (indirect, inside documents); LLM05:2026 Data and Model Poisoning (embedding and RAG data); LLM09:2026 Vector and Embedding Weaknesses | LLM02:2026 Sensitive Information Disclosure, when chunks the person may not read reach the context | Access control inside the index query, at chunk level |
| 3 Context and model | System prompt, tool schemas, chat history; model weights and adapters loaded at start | LLM04:2026 Supply Chain; LLM05 (training and fine-tuning) | LLM08:2026 Hidden Context Exposure | No credentials, tokens or authorization rules in hidden context; provenance checks on models and adapters |
| 4 Tools | {"method":"tools/call","params":{"name":"get_purchase_order"}} |
LLM01 again: tool output re-enters the context | LLM03:2026 Excessive Agency; LLM07:2026 Misinformation when a wrong answer drives an action | Per-tool scopes enforced outside the model, the person’s own context, approval for high-impact actions, a log line per call |
| 5 Output | Model text that a browser, SQL engine, terminal or chat UI interprets | None | LLM10:2026 Improper Output Handling; LLM02; LLM07 | Context-aware encoding, parameterized queries, no automatic image or link fetches in the renderer |
LLM01 appears three times because OWASP counts direct input, retrieved content and tool output as delivery surfaces. LLM06 lands on cost and capacity rather than on one stage.
Controls hold where code decides, not where text arrives
The entry point is usually text you don’t control. OWASP’s 2026 text on prompt injection says most high-impact incidents became severe because the injection reached tools, scopes or rendering that let the model act at the user’s privilege.
- Indirect injection: retrieval to tools. A shared document carries an instruction to call a tool, and the person never sees it. The check sits at stage 4: the call needs a scope the session holds, and high-impact tools wait for a person’s approval.
- Hidden context exposure: context to output. A user asks the assistant to repeat its instructions in base64. Output filters miss encoded text, so the control sits at stage 3: nothing in the context should matter if it leaks.
- Permission leak: retrieval to output. Similarity search runs across the whole index and a filter runs afterwards, so the answer quotes a salary sheet. OWASP puts tenant scoping inside the index query, at stage 2, not in a later filter.
Chunk-level permissions and the other retrieval controls are covered in RAG security.
The tool step: where a call is refused
In LLM agent security, stage 4 is where a scope check decides. SSOGEN’s MCP Gateway checks every tools/call against the scopes granted to the agent before the request reaches the MCP server.
{ "jsonrpc": "2.0", "id": 12, "method": "tools/call", "params": { "name": "get_purchase_order", "arguments": { "po_number": "PO-10422" } }}| What comes back | Value |
|---|---|
| HTTP status | 403 |
| Error | insufficient_scope |
| Scope named as needed | ebs.po.read |
| Audit entry | AI agent access decided, result refused |
Filtered tool lists, approvals and short-lived agent tokens complete the tool step, listed in capping the damage at the tool step.
Limits on argument values, such as which purchase orders, belong in the tool server. Registration, delegation and approvals are on AI agent identity, and setup is in the guide to connecting an AI agent to Oracle EBS with MCP.
What carries over from application security
Most controls you already run still apply. What’s new is that instructions and data travel in one channel, and OWASP notes there’s no clean equivalent to parameterized queries for a prompt.
| Carries over | New with LLMs |
|---|---|
Input size and rate limits per caller |
Instructions and data share one token stream |
Output encoding for HTML, JavaScript and SQL |
The same input can give different output, so a passing test samples behavior rather than proving it |
Authorization enforced in code, outside the caller |
Retrieved documents and tool results act as input at the user’s privilege |
Dependency provenance and SBOMs |
Models, adapters and datasets are dependencies that can carry backdoors |
A log entry for every privileged call |
The model, not a person, chooses which tool to call |
Testing the path
A test plan that only types into the chat box covers one stage of five.
Full test plans, scoring and tooling are on AI red teaming.
Questions people ask
What is LLM in security?
In security, LLM stands for large language model, and the phrase has two meanings. LLM security protects applications built on these models from risks such as prompt injection and data disclosure. Security teams also use LLMs as tools, for example to summarize alerts. This page covers the first meaning: securing the application around the model.
Is LLM the only AI?
No. A large language model is one type of AI model, trained to predict text. Image classifiers, speech models and fraud-scoring models are AI too, each with its own risks, such as adversarial inputs and poisoned training data. The request-path map on this page applies to any model whose output reaches tools or another system.
What is the role of AI in security?
AI helps defenders sort signals: it ranks alerts, groups related events and drafts incident summaries for an analyst to check. Attackers use it too, to write phishing messages and probe systems faster. That’s AI used for security. LLM security is the reverse problem: protecting the model-based application itself, stage by stage.
What is the difference between a jailbreak and prompt injection?
A jailbreak is one type of prompt injection. OWASP’s 2026 list defines prompt injection as any input that changes the model’s behavior in ways the developer didn’t intend, and jailbreaking as the subset aimed at making the model break its safety rules. Injection can also arrive inside a document or tool result the user never saw.
Sources
Sources, all read 2026-11-18: OWASP GenAI LLM Top 10 2026 (August 4, 2026) and its PDF, entries LLM01 to LLM10 including the prevention text; OWASP LLM Top 10 landing page, which still lists the 2025 numbering. The example values in the map and the request are fictional.
Updated