How big a job can I hand an AI agent without checking in?
Task length, reliability, and when to check in.
You'd send a new hire to grab coffee without a second thought. You might trust them with a two-hour report. You wouldn't give them a three-week project on day one and walk away.
Agents are the same. The question isn't "is it smart?" It's "how long a job can it finish before something goes wrong?"
METR, a research group that tests AI systems, timed how long a set of tasks took skilled people. Then they checked which of those tasks AI agents could complete.
In early 2025, agents succeeded almost every time on tasks that take a person under 4 minutes, and less than 10% of the time on tasks that take a person more than about 4 hours.
Time horizon: the length of task (measured in how long it takes a person) that an agent finishes successfully half the time.
In March 2025, the best model's time horizon was about an hour.
Half the time is a coin flip. It's a way to compare models, not a level of reliability you'd sign off on.
Over six years of models, METR found that time horizon doubled about every 7 months.
So the size of job an agent can take on is growing fast. Whatever the limit is today, it'll be different in a year.
A job it pulls off half the time isn't a job you can walk away from.
If you need it to work most of the time, the safe task length is a lot shorter than the headline number. METR's own charts show the horizon shrinking sharply when you ask for 80% success instead of 50%.
Long jobs give small errors time to pile up.
Each step might be 99% right. A hundred steps in a row is a different story.
Made-up example.
Match the leash to the job. Minutes-long tasks: let it run. Hours-long tasks: plan check-ins.
Ask for a plan with checkpoints. "Break this into steps. Stop after each one and show me what you did."
Keep a progress file. For long jobs, ask it to keep a running list of what's done, what's next and what it learned, so a fresh session can pick up cleanly.