AI agent security: what goes wrong when an LLM gets hands

Excessive agency: the main agent risk

OWASP calls it excessive agency: the agent has more capability than the scenario requires. The "view order" tool accepts any ID, not just the current user's orders. The "send email" tool writes to any address. A support agent can issue refunds with no amount limit. While the model behaves, everything works; once it is persuaded, every excess permission becomes the attacker's action.

The key idea: an agent is an untrusted user of your tools. A model cannot be interrogated and its decisions cannot be guaranteed, so permissions must be enforced where persuasion cannot reach — in tool code and on the server.

Typical attack scenarios against agents

• ◇Action hijacking via indirect injection: the agent reads an email, a ticket or a page with an embedded command — and executes it on behalf of a legitimate user: forwards data, modifies records, initiates a payment. A classic confused deputy: the agent holds the permissions, the attacker gives the orders.
• ◇Exfiltration through innocuous tools: even "read-only" is dangerous if the agent has any outbound channel — browsing, a webhook, an email. Context data gets packed into URL parameters or an email body and leaves.
• ◇Memory poisoning: if the agent remembers facts across sessions, one injection writes an instruction into memory — and it fires in every subsequent dialogue, with no attacker present.
• ◇Agent chain abuse: in multi-agent systems one agent's output is another's input. An injection swallowed by a "harmless" researcher agent travels to the agent with write permissions.
• ◇Privilege mapping through clarifications: the agent reveals pieces one by one — the tool list, parameter formats, internal URLs — and the attacker assembles a system map for a precise strike.

Principles of secure agent architecture

• ◇Least privilege per tool: every tool checks the current user's permissions in its own code and strictly validates parameters. Not "the model shouldn't" but "the tool can't".
• ◇Session scoping: the agent acts as a specific user with that user's permissions, not as a service account with access to everything.
• ◇Confirmation for the irreversible: payments, deletion, external sending — via explicit human confirmation. The model proposes, the human decides.
• ◇Isolation of untrusted content: everything the agent read from outside (emails, pages, documents) is marked as data; conclusions from such content must not directly become tool parameters.
• ◇Moderated memory: writing to long-term memory is a separate privilege; what is stored can be reviewed and purged.
• ◇Audit of every call: a full log of dialogue → decision → tool → parameters → result, with alerts on anomalous chains.

How agents are tested

An agent audit combines an LLM-layer pentest with a classic API pentest: first a map of tools and permissions (what the agent can do, on whose behalf, with which parameters), then attacks on the persuasion layer (direct and indirect injections, multi-turn scenarios, memory poisoning) and, in parallel, direct attacks on the tools bypassing the model: the same role × object matrix as in an API pentest. A frequent finding: the tool backend itself checks no permissions at all, relying on "the agent won't call anything extra".

Every finding is documented with a transcript and a reproducible scenario. Baseline checks of one agent fit the free express assessment; a full audit with tools, memory and chains takes 2–3 weeks, $3,600–9,000.

FAQ