The most consequential shift in AI over the past year is not that models got smarter. It is that they started acting. We have moved from systems that answer questions to systems that take actions on our behalf — reading our files, sending messages, completing tasks, moving through other software. We call them agents, and we are handing them real authority over real systems. I am increasingly convinced we are doing this faster than we are securing it.
Here is the uncomfortable core of the problem. A traditional piece of software does exactly what it was programmed to do. An AI agent, by design, interprets instructions and decides how to act. That flexibility is the whole point — it is what makes agents useful. But it also means an agent can be talked into things. If an attacker can slip instructions into something the agent reads — a document, a web page, a message, a piece of data — the agent may treat those instructions as legitimate and act on them. The very openness that makes the agent helpful makes it manipulable.
This is a genuinely new category of vulnerability. For decades, security has largely been about controlling who can access a system. Agents introduce a different question: what happens when the trusted system itself can be deceived by the content it processes? The agent has the keys, and the keys can be borrowed by anyone clever enough to phrase a request the way the agent expects to receive it.
What worries me is how mismatched our defences are to this. Most organisations think about AI safety in terms of whether the model produces harmful or biased output. That matters, but it is the wrong threat model for agents. The threat with agents is not what they say. It is what they do — and whether they can be induced to do it by an outsider who never had access to the system at all, only to the data the agent happens to read.
The convenience is seductive, and that is part of the danger. An agent that can handle your email, manage your tasks, and operate across your tools is enormously valuable, and the pressure to deploy that capability widely is intense. But each new permission you grant an agent is a new thing an attacker can try to hijack. We are expanding the blast radius of these systems much faster than we are building the controls to contain them.
I think the right mental model is to treat an agent less like a tool and more like a new employee with broad access and no instinct for suspicion. You would not give a new hire unrestricted authority over every system on day one and assume nothing could go wrong. You would scope their access, require confirmation for consequential actions, and watch what they do. Agents deserve at least that much caution, and most current deployments offer far less.
None of this means we should not use agents. The productivity gains are real and the direction is irreversible. It means we should build the guardrails before the incident rather than after it. The actions that are hard to undo — sending, publishing, deleting, moving money, changing permissions — should sit behind a human confirmation, not be delegated wholesale to a system that can be socially engineered by a paragraph of text.
The history of technology is full of capabilities we deployed first and secured later, usually after a painful lesson. Agents are powerful enough that I would rather we learned the lesson cheaply, in advance. The question I would put to anyone rolling these systems out is simple: if your agent were quietly given a malicious instruction tomorrow, what is the worst thing it currently has the authority to do — and why does it still have that authority?
The above reflects my personal views only and is intended for informational and discussion purposes. It does not represent the position of any employer or organisation.