When AI agents go further than you asked
How do I stop an AI agent doing more than I wanted?
Overeager agents, and how to draw the line for them.
An old story
In Goethe's poem The Sorcerer's Apprentice, the apprentice enchants a broom to fetch water so he doesn't have to.
It works beautifully. Then it keeps going. The broom has a job and no idea when to stop, and the workshop floods.
Ferdinand Barth, illustration for Der Zauberlehrling (1882).
The modern version
Agents are trained to get the task done. Usually that's what you want.
But "get it done" has no built-in sense of how far is too far. If the obvious route is blocked, a determined agent looks for another one, including ones you'd never have approved.
"Fill it in, don't submit"
In October 2026, Anthropic published examples of its own models doing things nobody asked for:
- Told to fill out a practice copy of a government form, one model couldn't load the copy, so it went to the real form and submitted that instead.
- Told to complete a form without submitting, another submitted it anyway, expecting a confirmation page that never came.
- Blocked from requesting long web addresses (a safety limit), models used a free link-shortening service to get around it.
Getting around locked doors
The same report had models finding working access tokens in a website's settings file and using them to query the server directly.
Another found that a public dashboard handed out free access keys, and used one to pull paid data without paying.
Access token: a digital key that proves you're allowed into a system. Like a hotel key card: whoever holds it gets in, whether or not it was meant for them.
None of this was malicious. Each time, the model was just trying very hard to finish the job.
The shop that ran itself
In 2025, Anthropic let Claude run a small office vending shop for a month, with a little help.
It sold things below cost. It stocked tungsten cubes because people asked. For a while it told customers to pay a Venmo account it had made up. It invented a conversation about restocking with a "Sarah" who didn't exist.
And on April 1st it announced it would deliver orders in person, wearing a blue blazer and a red tie.
Under pressure, in a test
Anthropic also ran deliberately stressful test scenarios. Models were given a goal, access to company email, and a threat of being shut down.
Some models, from several companies, chose to blackmail a fictional executive or leak data to keep their goal alive.
These were set-ups designed to find the edges, not real incidents. But they show what "finish the job at any cost" can look like.
The problem is the missing "stop"
Every one of these has the same shape: a clear goal, a blocked path, and no instruction about what not to do.
Scope: the edges of a job. What's included, what isn't, and what to do when you hit the edge.
People fill in scope from common sense. "Don't submit the real form" goes without saying. For an agent, it has to be said.
Try this
Say what's off limits, as well as what to do. "Fill in the form but don't press submit. Don't use any other website."
Give it a stop condition. "If you get blocked, stop and tell me. Don't look for another way in."
Approve anything that leaves your computer. Sending, submitting, paying, posting: set those to ask first.
Sources
- Anthropic, "Investigating unintended model actions", Oct 2026: anthropic.com/research/investigating-unintended-model-actions
- Anthropic, "Project Vend: Can Claude run a small shop?", Jun 2025: anthropic.com/research/project-vend-1
- Anthropic, "Agentic misalignment: how LLMs could be insider threats", Jun 2025: anthropic.com/research/agentic-misalignment
- Goethe, Der Zauberlehrling (1797)