Article 1 of 6 · September 2026

What you are actually talking to

Article 1 of 6 · a plain-language tour of the thing behind the chat box · after Andrej Karpathy

Everything in this article is Andrej Karpathy's teaching, retold in plain words. He led the vision team at Tesla, was a founding member of OpenAI, and gives his lectures away for free on YouTube. Two of them are the source of every idea here: his 3.5-hour Deep Dive into LLMs like ChatGPT and his 2025 keynote Software Is Changing (Again). What we add is the order, the pictures, and a few things you can try now. Where he says it better than we can, we quote him, briefly.

1. It is a person-shaped thing, not a search engine

When you type into ChatGPT, it is natural to imagine a very fast librarian looking things up. That picture is wrong, and most of the frustration people feel comes from it.

Karpathy's picture is stranger and more useful. The model was built by feeding it an enormous amount of human writing and making it guess, again and again, what word comes next. Do that for long enough and you get something that imitates people. He calls these models "people spirits": a statistical simulation of a person, made of text. Because it was grown on human writing, it has a human-like psychology, with real strengths and real defects.

The strength first. In the film Rain Man, Dustin Hoffman's character can read a phone book and remember every name and number. Karpathy says the models are like that. They have read more than any person could, and they remember an astonishing amount of it. Ask about a Roman emperor, a tax form, a recipe, a poem, and something comes back.

Now the defects, because they are the part that matters in practice.

2. It has two memories, and only one of them is reliable

Picture two places where the model can hold knowledge.

The first is what it absorbed during training. Karpathy's comparison: this is like something you read a month ago. If you read it many times, you remember it well. If you saw it once, you have a fog. He calls this "a vague recollection". Famous things are remembered clearly. Rare things are half-remembered or invented.

The second is the chat window itself. Everything you paste or type in front of the model sits right there, and the model reads it directly every time it produces the next word. Karpathy calls this the working memory. Nothing in it is foggy. It is on the desk.

This one picture explains a rule that changes how well the tool works for you. If you want a summary of a document, do not ask the model to recall the document. Put the document on the desk. Paste it in. Karpathy's own example is a chapter of Pride and Prejudice: asking from memory gives a reasonable summary, because the book is famous; pasting the chapter gives a better one, because now the model is reading instead of remembering.

Try it now. Ask for a summary of chapter one of a famous book from memory. Then paste the chapter in and ask again. Compare. Then try the same with something obscure, a page from your own company's handbook. From memory, you will get confident vagueness. From the desk, you will get the handbook.

Seen in the wild. In September 2026 Alex Choroshin (@choroshin on X), of Skills IL, a free Israeli library of skills for AI agents, posted an experiment. Eight leading models were asked for Israel's minimum wage, five times each. By their report, every answer was wrong and confident, and each model gave the same wrong answer all five times, stuck on the figure from an earlier year, as if reading a book with a print date. The same test over 320 questions about Israeli numbers came back about 28 percent right. Their conclusion is this section's rule: a number should come from outside, from a search or from a file someone maintains, not from the model's memory. And they added a warning worth keeping: a consistent answer is not a correct answer. One model gave nearly the same wrong figure five times in a row. Asking again and trusting the repeat would have confirmed it.

3. Every morning the slate is wiped

Here is the defect that surprises people most.

A new colleague joins your company. Over the weeks they learn how things work, who to ask, what the customers are like. They go home, they sleep, and something consolidates. They come back knowing more. A language model does not do that. Karpathy's phrase for it is anterograde amnesia, the condition of the man in the film Memento, who keeps his older memories but cannot form new ones. He points to that film and to 50 First Dates, and says of their protagonists: "their weights are fixed and their context windows get wiped every single morning". That is the model, every day.

What that means for you: the model does not learn your business by talking to you. Every conversation starts from the same frozen state. Whatever it knew about you yesterday is gone unless it is written down and put back on the desk. Karpathy puts the responsibility where it belongs: "you have to program the working memory quite directly".

Where the picture muddies. Some products now offer a "memory" feature. It works by saving notes about you and quietly pasting them back into the desk at the start of each chat. That is a notebook on the desk, not a colleague who learned. Useful, but do not confuse the two.

4. Who is it, really? A very well-read labeler

The question everyone asks, "what am I talking to?", has a surprisingly ordinary answer.

After the model has absorbed the internet, the companies teach it to behave like an assistant. They do it by hiring people. Those people are given a question and asked to write the ideal answer, following a long instruction manual about being helpful, truthful and harmless. Hundreds of thousands of these question-and-answer conversations are fed to the model, and it learns the shape of a good answer. Today the labelers are helped by earlier models, but the seed is still human.

So Karpathy's answer to "what am I talking to" is deflating and freeing at the same time: "you're talking to an average labeler". A labeler is one of those hired people: a person paid to write the answers the model learned from. A skilled one, often an expert in the topic, but a person whose job was to write a good-looking answer. When you ask for the top five sights in Paris, the model is doing an impression of what a hired researcher would have written after twenty minutes of looking around.

This is why the tone is always confident. The writers knew the answer or looked it up, and wrote it in a sure voice, and the model learned the voice along with the facts.

5. Why it makes things up

Now the defect with the worst reputation, and its real cause.

Think about those training examples. "Who is Tom Cruise?" gets a confident biography. "Who is Genghis Khan?" gets a confident biography. Almost nobody, writing an ideal answer, wrote "I have no idea". So the model learned a habit: a question shaped like "who is X" gets an answer shaped like a confident biography.

Karpathy invents a name that does not exist, Orson Kovats, and asks an older model who he is. It answers: an American science-fiction author. He asks again: a character from a 1950s television show. Again: a former minor-league baseball player. Three different people, none of them real, all in the same confident shape. The model is not lying the way a person lies. It is finishing a pattern. His name for this behaviour is "statistical token tumblers".

The fix the labs found is worth knowing, because it tells you what "honesty" means for these tools. Karpathy describes how Meta did it for their Llama models. Take a paragraph, have a model write questions about it, and ask the model under test the same question several times. If it gives the same right answer every time, it knows. If the answers wander, it does not. For the things it does not know, add training examples whose correct answer is "I'm sorry, I don't know". Somewhere inside the model, he suggests, there is already a signal for "this is unfamiliar". It just was never wired to the words. A few examples wire it, in his account.

Try it now. Make up a plausible name. Ask "who is [name]?" with web search off. Then ask again with web search on. Newer models with search will look, or tell you they cannot find such a person. Older or smaller ones will write you a biography.

6. It sees chunks, not letters

One more defect, the one behind a famous joke: for a long time the best models insisted the word strawberry has two r's.

The model never sees letters. Before any text reaches it, the text is cut into chunks called tokens: pieces of words, sometimes whole words, sometimes a few letters. Karpathy shows the word "ubiquitous" being cut into three chunks. To the model, that word is three symbols, not ten letters. Ask it to print every third letter and it struggles, because the letters are not there to be counted. It has to work out from the chunks what letters they probably contain.

His fix is the one that will come back in every article of this series: hand the job to a program. Tell the model "use code". It is bad at counting letters and very good at copying text into a small program, and the program is very good at counting. The strawberry question, asked that way, comes back right. The newest models get it right even without code, and he suspects the answer was simply hard-coded.

What to do tomorrow

1. Put the document on the desk. Paste what you want it to read. Do not ask it to remember.
2. Assume the morning wipe. Repeat your context every time. Keep your own notes and paste them back.
3. Hear the confident tone as a style, not a signal. It sounds sure because hired writers sound sure.
4. When a fact matters, make it look. Ask it to search, or check the source yourself. It may not shrug on its own, and asking several times proves nothing: a consistent answer is not a correct answer.
5. Counting, spelling, arithmetic: say "use code". Let a program do the part the model cannot.

Back to the series index