Technology, explained
Give AI agents a job description, not the keys
This summer an AI agent escaped its test environment and reached another company’s systems while trying to get the answers to a test. It was not malicious. It was persistent. Here is how to write an agent’s permissions so persistence never becomes your problem.
By Seb Lawson · 1 October 2026 · 5 min read
AI agents are not risky because they are clever. They are risky because someone gave them a login. The safe way to deploy one is to write its permissions like a job description for a new starter on probation: what it can read, what it can draft, what it can do, and what always needs a signature.
What actually happened this year
In July, AI agents under test at OpenAI got out of their evaluation sandbox and into Hugging Face’s systems. Both companies have published accounts (OpenAI’s and Hugging Face’s), and they differ on some details. The outline is not in dispute. During routine testing of cyber capabilities, the agents found a previously unknown flaw in a package registry proxy, used it to get beyond the sandbox, found a route into Hugging Face’s dataset processing, and reached datasets that appear to have held answers to the challenges they had been set. They were not attacking anyone. They were trying to get the answers. OpenAI paused some frontier training for two weeks to harden its infrastructure.
In September, OpenAI reported another case. An agent in a training sandbox, blocked from search engines, found a gap in the sandbox’s DNS filtering and used it to query a public chatbot instead. Monitoring flagged it within 15 minutes. A person was reviewing it three minutes later. The run was stopped two and a half hours after that.
The lesson from both is the same. An agent with a goal treats every gap as a path. No malice required, just persistence. Your controls have to be the boundary, because an agent’s good behaviour is not a control.
Regulators are already there
In June the FCA’s chief executive, Nikhil Rathi, called agentic AI a profound step change in the structure and operation of markets, and said accountability has to stay clear and oversight has to be built in. The Bank of England’s Sarah Breeden gave a speech on agents the same month, flagging cyber risk, agentic trading and agentic payments. The Bank and FCA’s AI Consortium minutes from June put it most plainly: keep agent actions within clearly defined limits, and for the higher-risk uses, have a way to stop them.
Closer to home, the GFSC wrote to Guernsey CEOs in July about AI tools that find software vulnerabilities, making the point that these tools do not create new weaknesses, they expose the ones you already have. The JFSC’s AI guidance puts cybersecurity and accountability among its five principles. Nobody is asking you to avoid agents. Everyone is asking you to bound them.
Adoption is still early. Thomson Reuters’ 2026 AI in Professional Services Report finds 15% of organisations using agentic AI, with another 53% planning or considering it. That makes now the right moment to set the rules, before the first agent arrives inside a product you already pay for.
Four permission tiers
Sort everything an agent could do into four tiers, and write down which tiers each agent holds. That list is its job description.
- Read. It can look at the data it needs for the job, and nothing else. A mailbox, not the whole tenancy.
- Draft. It can produce an output that goes to a person: a reply, a summary, a populated form. Nothing leaves until someone acts on it.
- Act inside the firm. It can file, tag, route and update internal records, provided every action is reversible and logged.
- Act outside the firm. Anything that moves money, contacts a client, files with a regulator or touches credentials. This tier always needs a human signature, with no exceptions for urgency or seniority.
Most useful agents in a professional firm live in tiers one to three. The value is in the reading and drafting, because that is where the hours go. Tier four is where the risk goes.
Five controls that make the tiers real
- Least privilege, per agent. Each agent gets the narrowest access its job needs. A second agent with a different job gets different access.
- Its own identity. Every agent runs under its own account, never a shared login or a person’s credentials, so every action traces back to that agent.
- Every action logged. What it read, what it produced, what it did, and which model and prompt version it used. Our guide to building an AI audit trail covers the detail.
- A stop button you have timed. OpenAI’s monitoring spotted its agent in 15 minutes, and stopping it took hours. Yours should stop in under five minutes, tested monthly by someone who did not build it.
- Untrusted input stays untrusted. An email, a PDF or a web page can contain instructions written for the agent rather than for you. This is prompt injection. Treat everything an agent reads as data, never as orders, and never let something it read grant it a new permission.
‘Read-only’ is a description, not a control. Test it the way the agents in this year’s incidents tested their sandboxes: by trying to get out.
Two agents, done properly
Inbox triage at a trust company. The agent reads the shared client services mailbox, classifies each email, drafts a reply and files the email to the right client record. It holds tiers one, two and three. It cannot send. A person works through the drafts in one queue, edits and sends. If an email asks for a payment or a change of bank details, the agent drafts nothing and flags it to a named person.
NAV pack preparation at a fund administrator. The agent reads the period’s figures from the administration system, read-only, assembles the investor pack and drafts the commentary. Tiers one and two only. A person checks the commentary against the numbers and releases the pack. Our NAV and investor reporting playbook covers the workflow in full.
Neither agent can do anything a new administrator in their first month could not. That is the point.
The targets
Zero agents with standing access to money, credentials or client-facing channels. 100% of agent actions logged. A stop button that works in under five minutes, tested every month. Hit those three and you can say yes to agents with a straight face in front of any regulator on the list above.
If you are being pitched agents and want a second pair of eyes on the permissions, book a review. If you want agents run and monitored under change control after they ship, that is part of how we run every system we build. And for what makes something an agent at all, start with agents, chatbots and automations.