How to Stop AI From Making Things Up — 3 Verification Prompts

🔓 Free this week: This guide is normally Premium. It's unlocked through Sunday — read it now.

AI hallucination is the most misunderstood problem in AI. People treat it like a bug that will eventually get fixed. It won't — not completely. Hallucination is a fundamental property of how language models work. They predict plausible text. Sometimes plausible text is wrong.

The good news: once you understand why it happens, you can catch most of it before it causes a problem.

Why AI makes things up

Language models don't "know" facts the way a database does. They learn patterns from training data and generate responses that fit those patterns. When asked about something that isn't well-represented in training — a recent event, a specific statistic, a niche topic — the model fills in the gap with something that looks right. It doesn't flag uncertainty unless you ask it to.

The three types of hallucination

Factual hallucination: A specific claim that's simply false — a wrong date, a made-up statistic, a citation that doesn't exist.
Synthesis hallucination: Mixing information from multiple sources in a way that produces something plausible but wrong.
Confident extrapolation: The model doesn't have enough information, so it fills the gap with something that sounds authoritative but is largely invented.

Prompt 1: The uncertainty probe

Run this immediately after any response that contains specific claims, statistics, or citations:

Before I use your last answer, I need you to audit it.

1. Which specific claims are you most uncertain about?
2. Which facts could you have gotten wrong or confused with something similar?
3. Are there any statistics, dates, or citations I should verify independently?
4. What would change in your answer if any of those claims turned out to be wrong?

Be honest. I'd rather know you're uncertain than have you sound confident and be wrong.

Prompt 2: The source pressure test

For anything where accuracy matters — research, legal summaries, financial data:

For the factual claims in your last response:

- Identify the three claims that most need external verification
- For each one: (a) how confident are you, 1-10? (b) what's the most likely source I could check?
- Flag anything where your training data might be outdated or incomplete

If you're not sure of a source, say so. Do not fabricate citations.

Prompt 3: The second-opinion approach

Open a fresh conversation and paste:

I received the following answer from an AI assistant. I need you to act as a fact-checker, not a helpful assistant.

Find anything that might be inaccurate, misleading, or unverifiable. Be skeptical.

[paste the original AI response here]

For each concern: explain the issue, why it might be wrong, and what I'd need to check to verify it.

The 4-question pressure test

For any important AI output, ask yourself these four questions before acting on it:

Is there a specific statistic or date I should verify independently?
Is this a topic where Claude's training data might be outdated?
Am I relying on this for something where being wrong has real consequences?
Did Claude express any uncertainty, or does it sound uniformly confident?

If the answer to any of these is yes, verify the most consequential claims before acting.

The practical rule: Use AI to think and draft. Use primary sources to verify before you act. That division of labor eliminates most hallucination risk without slowing you down.