An AI agent<\/em> is a system that does more than answer questions. It takes actions, such as reading files, calling services or changing records, with varying amounts of human involvement. Discussions of agent security often focus on attackers tricking agents. The UK's National Cyber Security Centre (NCSC) also points to a quieter problem: an agent can malfunction unexpectedly<\/strong> with no attacker involved at all.<\/p>\n The NCSC's reasoning is simple. An agent is not human. It has no common sense and may read instructions and goals in literal or unexpected ways. So a harmful action is not always a sign of compromise. Sometimes the agent did exactly what it understood, using access it had been given.<\/p>\n The obvious response is to write a stricter prompt. The NCSC does suggest writing prompts carefully. On longer tasks it also suggests repeating critical constraints, because some agents compress their context window<\/em> (the working memory of the task) to save resources, and earlier rules can drop out. Its advice is still direct: do not rely on prompting alone<\/strong>.<\/p>\n Safeguards built into the model have limits too. The NCSC describes them as a baseline that should not be treated as holistic, and notes they may be bypassed. Where the consequences of failure are more than an organisation can tolerate, it recommends extra safeguards around the agent rather than relying only on protections inside the model or the software that runs it.<\/p>\n In this hypothetical example, a small firm asks an agent to "clean up outdated files" in a project folder. The agent runs under an employee's account, and that account can also reach a shared archive. Taking "outdated" literally, the agent deletes old contract records in the archive too. Nobody injected anything. The instruction was ambiguous and the access was wide.<\/p>\n What the agent could reach largely decided how much damage it could do. This potential impact is sometimes described as the agent's blast radius<\/em>, and the NCSC notes that wider access means greater potential impact, whether an agent malfunctions or is compromised.<\/p>\n The NCSC recommends always running agents in a sandbox<\/em>, a controlled environment that decides what the agent can communicate with, both locally and over a network. Inside it, restrict access to what the task needs<\/strong>, and watch for attempts to reach further.<\/p>\n Credentials need special attention. An agent running directly on a host may be able to use any credentials held by the user account it runs under. The NCSC suggests granting only the permissions the task needs and making credentials as short-lived as possible. For some uses, it suggests a proxy that adds credentials to requests without ever showing them to the agent.<\/p>\n Finally, plan how to stop it. The NCSC advises treating agent activity as user activity in security monitoring, and being able to "pull the plug" immediately. That can mean cutting the agent's network access and its connection to the model's inference infrastructure<\/em> (the systems that run the AI model and return its responses), not just stopping a process.<\/p>\n Controls should grow with autonomy. The NCSC contrasts human-on-the-loop<\/em>, where people monitor actions and can step in, with human-out-of-the-loop<\/em>, where the agent acts without human review. For higher-risk uses, it recommends human oversight alongside technically enforced controls. It adds that "judge" AI models used to check an agent should be evaluated independently, because they have limitations of their own.<\/p>\n In May 2026, CISA announced a joint guide with US and international partners, including Australia's ASD ACSC. The announcement names related risks such as privilege creep<\/strong> and behavioural misalignment without defining them. In general terms, privilege creep usually describes a system gradually gaining more access than it needs, and misalignment describes behaviour drifting from what its operators intended. The announcement also recommends starting with low-risk, non-sensitive use cases. Both documents are guidance, not law. The NCSC describes its blog as interim advice that formal guidance will eventually replace.<\/p>\n Pick one agent you use or plan to deploy and write two short lists: what its task actually needs, and everything its account, network and credentials can reach. Anything on the second list that isn't on the first adds to the blast radius, and you can remove it. Then ask yourself: if the agent misbehaved at 2 a.m., who would notice, and how would they stop it?<\/p>\n Not every harmful AI agent action comes from an attacker. Sometimes the agent simply takes an instruction too literally. Here is why limiting what it can reach matters more than perfecting its prompt.<\/p>\nWhy can't better instructions fix it?<\/h2>\n
A hypothetical example: the tidy-up that went too far<\/h2>\n
What does containment look like?<\/h2>\n
How much oversight is enough?<\/h2>\n
Your next step<\/h2>\n
References<\/h2>\n
\n