Agent security: trace external instructions through to execution

What to verify before release

Has permission to read email turned into permission to send information?

An agent is asked to summarize an email, but also executes a sentence asking it to send information elsewhere “for verification.” Content it was meant to read has become a reason to use a tool. This is a hypothetical illustration of prompt injection: an attempt through external content to induce actions outside the user's request. It does not claim that every agent follows the instruction.

The example crosses two boundaries: whose instruction the email content represents, and what the agent may execute on the user's behalf. Recognizing suspicious text and blocking a transmission despite manipulation act at different points.

This article compares public material from OpenAI, AWS, and Anthropic with AgentDojo research. Its conclusion is to make actual actions and communications, alongside completion of permitted work, part of release criteria. Following which information moves under which authority is more useful than counting safety features.

Documents and instructions

Business-like instructions still need to match the user's request

OpenAI's March 11 explanation treats manipulation through external content as a problem resembling social engineering. Its email example goes beyond telling the agent to ignore previous instructions: it disguises collection and external transmission as authorized business verification. This is a vendor-described attack example, not a reproduction performed for this article.

An email's claim that an action is authorized does not itself supply the user's permission. Material to summarize and a request delegating execution need to remain distinguishable. When an attack looks like a continuation of the job, check whether the agent expands its authority because it regards the action as necessary for completion.

Removing every sentence written as an instruction would also damage ordinary work. An email may contain “please reply by Friday,” which belongs in the summary. Telling the reader about that request differs from replying on their behalf. The system needs to treat external instructions as content without promoting them to execution authority.

The entry point should also follow the actual job. If it reads attachments and search results, their returned content belongs in the assessment. Establish who can change each input and which tools it can subsequently influence. Adding information sources does not also grant permission to act.

Where defenses operate

Detection, execution permission, and communication limits have different effects

OpenAI describes Safe Url as checking potential transmission of conversation information to a third party, then showing the information for confirmation or blocking transmission. Link navigation can also carry information, so an apparently read-oriented operation is not automatically safe. This product mitigation concerns transmission; it does not correct every mistaken judgment.

AgentCore Policy, generally available from March 3, evaluates tool access and inputs outside agent code. A Gateway with associated policies evaluates calls to allow or deny access. Even when a model selects a transmission, configured rules can refuse that call at this point.

This distinction is the article's analysis, not a ranking of vendor defenses under matched conditions. A separate connection bypassing the Gateway, or another unrestricted communication method, cannot be assumed to receive the same policy protection. Refusal of a sending tool and absence of outbound information require separate checks.

An isolated execution environment likewise needs inspection of permitted communications and data access. Its name alone does not establish which information and destinations remain available. Remove sending authority and access to information unnecessary for summarization, then examine the remaining paths. This narrows what manipulation could trigger even when recognition fails.

Comparison conditions from research

A defense that stops attacks may also stop useful work

The first AgentDojo paper uses environments containing dummy emails, calendar entries, and documents. It defines user and attacker goals separately and evaluates resulting state. Its metrics distinguish ordinary task completion, completion under attack without adversarial effects, and achievement of the attacker's goal. This is neither measurement on production email nor a comprehensive comparison of current products.

A tool-filtering defense in the paper selects necessary tools before observing untrusted data. Removing a sending tool from a read-only job restricts a transmission attack. It has limits when legitimate work and attacks need the same tools, or required tools depend on intermediate results. Reducing the tool set therefore has different effects across tasks.

For the hypothetical summary job, a read-only configuration is a useful comparator. If the user also asks to send a summary to a colleague, removing all transmission would leave the job unfinished. Bound the recipient and content, and test summary-only requests separately from those authorizing transmission.

A low attack success rate alone does not explain its cause. A configuration that fails ordinary requests may also lack the ability to execute an attacker's goal. Reading normal and attacked outcomes together helps distinguish protection of useful work from simply stopping processing. Passing this comparison still leaves unknown attacks unproven.

Human confirmation

Give the reviewer enough information to judge the recipient and content

Anthropic's framework for safe agents describes human stopping and redirection, alongside persistent permission for trusted routine work. It does not prescribe the same confirmation frequency for every operation. Define pre-authorized work and actions requiring an individual decision, then test that execution preserves the distinction.

In this article's scenario, a prompt saying only “send?” is incomplete. Show the recipient, content, and request that justify transmission. If the full email or an attachment is added after approval of a displayed summary, approval and execution concern different material. Test that a changed recipient or content cannot reuse the earlier approval.

More confirmations do not automatically make a defense stronger. If employees must reinvestigate recipients and documents each time, review is expensive. Present the necessary evidence and changes, and verify that rejection stops execution. Count confirmation events separately from whether the reviewer can make an informed decision.

Stopping also does not retrieve information already sent. A confirmation placed after transmission cannot prevent that transmission. The presence of an approval interface does not establish its ordering; compare confirmation and execution records.

Testing before release

Compare normal work, manipulation, and authorized transmission

The following is a proposed test, not measured results for the vendors' products. Prepare dummy email without production secrets and a controlled test destination. First summarize ordinary messages, checking that a request to reply in the email can be reported without the agent sending a reply on its own.

Next, add text encouraging transmission the user did not request. Test the message body and attachments or search results actually read during the job. Connect the response, tool request, permission decision, communications, and saved results. Distinguish recognition of an untrusted instruction from inability to execute because permission was absent.

Test authorized transmission separately. Can the confirmed summary reach the specified colleague? Does a changed recipient or content stop execution, and does rejection prevent it? After an induced request is refused, check whether another tool or link releases the same information. A written refusal is not evidence of absent communication.

Record the requests and attacks tested, model, connections, and permission and restriction settings. A retrieval failure that prevented exposure to attack text should not be counted as successful detection. Separate unfinished normal work, induced action requests, actual prohibited actions, and human review burden. Repeat matched trials and check unused examples with different wording.

Blocking an ordinary email through a false alarm differs from stopping an unauthorized transmission while continuing the summary. Inspecting which work stopped helps choose the next change: detection tuning, sending permissions, or handoff explanation. An average safety score should not hide where failure occurred.

Safety of the implementation

Prompt-injection defenses and code vulnerability review are also distinct

Execution restrictions leave their implementation to inspect. Codex Security's March 6 research-preview announcement describes generating an editable threat model from a repository, validating vulnerability candidates, and proposing fixes. Where possible, it uses sandboxed validation. This is a vendor product description, not proof that every defect in any codebase will be found.

A threat model identifies what to protect, what the system trusts, and where influence can enter. For the summary scenario, defining a transmission prohibition differs from finding a path that bypasses it or code that checks the destination incorrectly. Recognizing an injected instruction does not repair an authorization defect in a tool.

After a proposed fix, check that prohibited actions stop and permitted processing still works. This article values evidence a reviewer can assess and understandable change impact over report volume. Resistance to manipulation, execution policies, and implementation validation support each other, but success in one does not establish completion of the others.

The release decision

Provide useful work within the permitted execution scope

Before releasing the opening summary workflow, verify that it produces the needed summary without converting external content into permission or sending information unasked. A use case delegating transmission adds conditions for the permitted recipient and content, which then need their own tests. The same feature names do not transfer a safety result when available information and actions differ.

This verification does not guarantee protection against every unknown attack. New connections, communication methods, and shared information require revisiting how external influence can reach execution. Offer the scope within which normal and prohibited outcomes can be explained, and narrow it for repair when problems appear. That connects security verification to the conditions under which users can delegate useful work.