Is a long, polished AI research report as good as it looks?
What "deep research" is good at, and where it slips.
You ask a question, click "Deep research" and go and make a coffee.
When you come back there's a long report waiting: headings, an executive summary, a table, forty citations.
It looks like a week of an analyst's time.
Deep research: a mode in ChatGPT, Claude, Gemini and others where the AI runs dozens of web searches over several minutes, reads the pages, and writes a long report.
Every step is the same kind of model you already know, doing more steps. Errors can slip in at any of them.
OpenAI's own documentation for its deep research tool says it can make "convincing mistakes".
Simon Willison, a programmer who writes a lot about these tools, put it this way: a multi-page document with headings and citations "can give the misleading impression of a PhD level research assistant."
In February 2025, the analyst Benedict Evans looked closely at one of OpenAI's own showcase reports, about smartphone use around the world.
It said Japan was 69% iPhone, 31% Android, citing two websites. One of them measures web traffic, not who owns which phone. The other mostly collects other people's numbers.
The research firm behind the underlying data showed roughly the opposite: about 63% Android. A Japanese government survey put it at about 53% Android.
"If there are mistakes in the table, it doesn't matter how many there are. I can't trust it."
One wrong number in a table you have to use means you have to check every number. At that point the report has saved you less time than it seemed to.
Wrong facts can at least be checked. A gap is harder to spot.
If the report leaves out the one study that disagrees, or the regulation that changed last month, nothing on the page tells you. It reads as complete either way.
It works best as a starting map. It's risky to treat it as the finished answer.
Read the sources list before the report. Are these primary sources (the regulator, the company's own filings, the study) or blogs quoting blogs?
Check the three numbers you'd actually use. Open the source and find each one (see "AI's sources need checking too").
Ask what's missing. "What evidence points the other way? What couldn't you find?"