Can I trust the sources AI gives me?
A citation from AI is a claim, not proof.
New York, 2023. A lawyer used ChatGPT to find earlier court decisions to support his client's case, Mata v. Avianca.
It gave him several. He asked ChatGPT whether they were real. It said yes, and that they could be found in the major legal databases.
They didn't exist. The judge fined the lawyers $5,000.
A researcher, Damien Charlotin, keeps a public list of court decisions where AI-invented material turned up in legal filings.
As of October 2026 it has over 2,200 entries.
Most involve people representing themselves, but nearly 900 involve lawyers.
That's better, but it doesn't solve it.
In 2025, Columbia University's Tow Center tested eight AI search tools. They gave each one a passage from a real news article and asked where it came from.
Writing a citation is just more next-word prediction (see "AI isn't looking anything up"). It knows what a citation looks like. Whether this one points to something real is a separate question, and it can't tell the difference from the inside.
Even when it does search, it can read a page and then summarise it loosely, mix up two sources, or attach the right link to the wrong sentence.
The lawyer asked, "Are these cases real?"
The same process that wrote the citations answered the question, so it gave the same confident yes.
For anything that matters:
If you can't check it, don't repeat it as fact.
Prefer tools that show you what they searched and which source supports which sentence.
If it can't show you where a claim came from, treat the claim as unsourced.