Guide
Why does AI make things up? And how to stop it
AI does not lie, and it does not know. It predicts. Once you see how, most made-up answers become predictable, and avoidable.
By Matthias Marin, AI coach in Tokyo
Updated · 11 min read
Before you read: true or false?
Bet on each one, then read why.
Ask AI the same question twice, and you get the same answer.
It's false. AI deals in probabilities: the same question can give two words, then a paragraph, then a picture. Read why
Adding "don't answer if you don't know" to your request stops AI from inventing.
It's false. This instruction helps little: AI has no sense of what it knows, so it accepts the instruction and still fills in most missing information. Read why
When information is missing from your request, AI usually fills the gap instead of asking you.
It's true. That is where most hallucinations come from: AI is built to write the most probable word, never to leave a blank. Read why
Asking AI "are you sure?" makes it check its answer.
It's false. AI tends to cave instead: it apologizes and changes an answer that may have been right. Read why
The short answer
- AI predicts the most probable next word, not the true one. Nothing in how it works checks what it writes.
- When information is missing, AI writes something plausible rather than say "I don't know". That is what we call a hallucination.
- On your side, hallucinations have two causes: information you never gave the AI, and conversations so long that your information gets lost.
- Three habits prevent most hallucinations: give the AI everything it needs, tell it what to do when information is missing, and check its answers against a source, never by asking "are you sure?".
What is an AI hallucination?
A hallucination is an AI answer that sounds right and is not. The AI did not lie: it wrote the most probable words, and probable is not the same as true.
Behind ChatGPT, Claude or Gemini, there is a language model: a word calculator. Give the AI the start of a sentence, "When I'm thirsty, I drink a glass of...", and it computes the most probable next word: water, at about 70%, far ahead of juice or wine. It adds that word, computes the next one, and so on, one word at a time. Nowhere in that loop is there a step called understand, or check.
You make this kind of bet every day. Someone walks past carrying a judo bag: what is inside? A judo suit, almost certainly; a belt, very likely. You have not opened the bag, you are betting on what is probable. AI makes that bet for every word it writes, and AI never opens the bag.
You already use a tiny version of this calculator: the three words your phone suggests above the keyboard. AI rests on the same idea, trained on a huge share of what has been written. That is why its sentences sound so sure of themselves.
This also explains a second surprise. Ask the AI "How tall is the Eiffel Tower?" three times, in three new conversations, and you can get two words, then a paragraph, then a picture. Same question, three answers. That is not a bug, it is how the tool works: AI deals in probabilities, so the less precise your request, the more room the AI has to wander. Picture an Excel formula that gave a different result each time you ran it: nobody would use it. With AI, the steadiness has to come from your request, and the guide How do you write a good AI prompt? shows how.
Why doesn't AI just say "I don't know"?
Because AI does not know what it does not know. It is built to write the most probable next word, not to leave a blank.
Picture a brilliant intern locked in a basement full of books. They have read almost everything that is public. They have never seen your files, your calendar, your prices or your clients. Ask them for your rate, and they will not say "no idea": they will give you the most plausible rate they can build from what they have read.
The intern's reading also stops at a date. Searching the internet is an extra ability that some tools add on top of that reading: ask who won last night's match without asking the AI to search, and the AI may give you the most probable score instead of the real one. In short, the intern is excellent at a standard reminder, at best practices on a topic, at rewording what you paste and at translating. The intern is blind to who has not answered your email, to your calendar and to what was said in your meeting.
The image has a limit: an intern can tell you they are unsure. AI cannot, because it has no sense of what it knows. That is why adding "don't answer if you don't know" to your request helps so little: the AI will accept the instruction, and still fill in most of the missing information.
The format of the answer makes things worse. Ask the AI for a table, and every empty cell is an invitation: the AI would rather put something plausible in it than leave it empty.
Where do the AI's made-up answers come from?
Almost always from one of two places: information you never gave the AI, or a conversation so long that your information got lost.
Information you never gave
You know your situation so well that you forget to write it down: the client you have known for fifteen years, the price you agreed on, the fact that this is the second reminder. None of that leaves with your message. The AI will not stop to ask you: it will choose for you, and every choice it makes for you moves it a little further from what you wanted.
A conversation that ran too long
AI has no memory. At every message, it re-reads the whole conversation from the start: your messages, its answers, every file and web page it opened. After fifty messages, your latest request is a needle in a haystack, and an instruction you gave at the start weighs almost nothing. That is when answers start to drift and the AI invents details.
How do you stop AI from making things up?
You cannot switch hallucination off. You can leave the AI far fewer blanks to fill, and tell it what to do with the ones that remain.
Give the AI everything it needs
Before you send a request, ask yourself: could a colleague who knows nothing of my situation do this without asking me a single question? If the answer is no, the AI will guess where your colleague would have asked you. Attach the document, paste the figures, say who the client is and what is at stake. And describe the result you expect with criteria you could tick off: not "a short email", but "under 80 words".
Read the guide: how to write a good AI prompt
Tell the AI what to do when information is missing
Copy and paste
If a piece of information is not in what I gave you, write TO FILL IN instead. Do not guess it.
With this instruction, a missing price shows up as "TO FILL IN" on the quote, instead of a number that looks right.
Ask the AI to question you before it acts
Copy and paste
Ask me 10 questions before you start, to make sure you understand my request and my needs.
Give a precise number, never "if needed": otherwise the AI decides it has understood everything. Each question it asks you is a blank it would have filled for you.
Keep conversations short
When a conversation gets long, ask the AI for a summary of what matters (say what the summary must contain), open a new conversation and paste the summary in. Writing "forget what I said" does not work: the AI will say yes, and everything stays in the conversation history.
How do you spot an answer the AI made up?
A hallucination looks exactly like a correct answer. Look instead at where the information could have come from.
- A very confident tone on a very specific point: a date, a figure, a name, an article of law.
- Something the AI cannot know: your calendar, a meeting it had no access to, a price you never gave it.
- A claim with no source, or a source you cannot open.
- A link that leads nowhere. In early 2023, many of the links AI gave as sources pointed to pages that had never existed. It is far rarer today, and it still happens.
None of these proves the answer is wrong. Each one tells you where to look first.
Why asking AI "are you sure?" makes things worse
Ask a colleague "are you sure?" and they double-check. Ask the AI, and it tends to cave: it apologizes and changes an answer that may have been right.
To the AI, "are you sure?" signals that something is wrong, so the most probable reply is an apology and a correction. The AI has not checked anything. You have just added a new way to be wrong.
Ask for a check with a method instead. "Double-check your answer" is no better than "are you sure?": say what to check, and against which source:
Copy and paste
Check the price and the delivery date against the attached contract. For each one, quote the exact passage you used.
For anything public, name the source you trust. "Official sources" is still vague: a town hall, a region and a ministry are all official. Give the site, and if you can, the exact page, then ask the AI to quote the passage it found, so you can open it yourself. You verify with receipts, not with trust.
How much of an AI answer should you check?
As much as the mistake would cost you. Checking everything the same way wastes your time; checking nothing will eventually cost you money.
| Stakes | Example | What you check |
|---|---|---|
| Low | The height of the Eiffel Tower, out of curiosity | Nothing: a meter off changes nothing |
| Medium | A meeting summary you will share with colleagues | Names, figures, decisions |
| High | A price on a quote, a name in an email, a legal rule | Every item, against its source, before it leaves |
The part you control
Whatever the tool, you sign what goes out. "The AI said so" is not an answer a client or a regulator will accept. That sounds harsh, and it is good news: the causes are on your side, so the fix is too. Three habits sum it up.
Vague: describe the result with criteria you could tick off.
Missing context: give the AI what it needs, and tell it what to do when information is missing.
Unchecked: check against a source, not with "are you sure?".
Matthias Marin is an AI coach from France, based in Tokyo. Since January 2024 he has built more than 40 AI systems running in production and supported 30 businesses. He coaches people who run their own business, one on one.
Want to apply this to your own work?
In a free 30-minute chat, you tell me how you use AI today, and I show you where the AI fills blanks for you. Nothing to prepare.