Is there someone in there?
Why AI feels like a person, and why it isn't one.
You say "thanks". You feel a bit bad being rude to it. When it says "I understand how frustrating that must be", part of you believes it.
That's normal. Humans are wired to see minds everywhere: faces in clouds, moods in cars, intent in a printer that jams right before a deadline.
Joseph Weizenbaum, a computer scientist at MIT, built a simple chatbot called ELIZA. It played a therapist by turning your sentences back into questions.
You: I'm worried about my mother.
ELIZA: Tell me more about your family.
It had no understanding at all. People confided in it anyway. Weizenbaum later wrote that his own secretary, who knew exactly how it worked, asked him to leave the room so she could talk to it in private.
ELIZA effect: our tendency to read understanding and feelings into a machine that produces human-sounding words.
In a 2024 study, about two-thirds of people said ChatGPT has at least some degree of consciousness. People who used it more often were more likely to say so.
The better it sounds, the stronger the pull.
Andrej Karpathy, one of the founders of OpenAI, has a good way to put it.
An animal's mind was shaped by millions of years of staying alive. An AI model was shaped by imitating the text people left behind. He calls these models "summoned ghosts": echoes of human writing, not creatures.
So it can write like a person who's grieving without having lost anything. It can say "I'm excited" without being excited. The words are real. Nothing underneath them is feeling them.
If you picture a person, you'll expect person-like things:
Most of the surprises in this course come from that gap.
AI also isn't smart the way a person is.
Karpathy points out that these models are sharpest at things where answers can be checked, like maths puzzles and code, because that's what they've been trained hardest on. They can be oddly weak at things any child gets right.
A person who could write working code in seconds would never fail an easy riddle. These models sometimes do. (More on this in "Why AI is great at hard things and bad at easy ones".)
How we talk about it shapes what we expect. Small swaps help:
| Instead of | Try |
|---|---|
| It knows | It was trained on |
| It thinks | It generated |
| It lied | It produced something false |
| It understood me | It gave a relevant answer |
You don't have to talk like a robot. Just notice when the words make it sound more human than it is.
Say thanks if you like. It's harmless. Just don't let politeness turn into trust.
When it shows feelings, read it as style. "I'm so sorry" tells you about its training, not its state of mind.
Ask "what would a text-predictor do here?" It's a surprisingly good way to guess its next mistake.