Why models make things up, and how grounding helps

Sometimes a model gives an answer that sounds completely confident and is completely wrong — an invented feature, a fake statistic, a policy that does not exist. This is often called a hallucination.

Why it happens

Remember what a model does: it predicts likely-sounding text. It is not checking facts against a source. When it does not actually know something — because it is private, recent, or rare — the most likely-sounding continuation can still be a confident, made-up answer. The model has no built-in sense of "I don't know" unless you design for it.

Common triggers:

• Questions about your company, product, or data, which the model never saw in training.
• Recent events after its knowledge cutoff.
• Very specific details — exact numbers, dates, quotes, links.

Grounding: give it the facts

The most effective fix is to stop relying on the model's memory. Instead, find the relevant information yourself and put it in the request, then tell the model to answer only from it. This is called grounding, and the most common way to do it is RAG — retrieval-augmented generation:

Copy code1. User asks:      "How long do refunds take?"
2. Retrieve:       search your help articles (often with embeddings) → "Refunds are processed within 7 days."
3. Generate:       send the question + that article, with the instruction:
                   "Answer only from the provided articles. If they don't contain the answer, say so."
4. Answer:         "Refunds are processed within 7 days." (source: Refund policy)

Make "I don't know" a valid answer

Copy code# academy-check
ARTICLES = {"refunds": "Refunds are processed within 7 days."}

def answer(question):
    found = [text for key, text in ARTICLES.items() if key in question.lower()]
    if not found:
        return {"answer": "I couldn't find this in our help articles.", "sources": []}
    # In a real app, a model writes the answer from `found`; here we just return it.
    return {"answer": found[0], "sources": ["refunds"]}

assert answer("How long do refunds take?")["sources"] == ["refunds"]
assert answer("Do you ship to Mars?")["sources"] == []

Two habits make grounded apps trustworthy:

• Show sources so users can check the answer.
• Allow "not found" instead of forcing an answer when the documents don't cover the question.

Grounding greatly reduces made-up answers, but does not remove them completely — a model can still misread a document. That is why serious teams also test their AI features on a set of known questions before shipping. You will build exactly this in the AI Engineer path.

Try it: ask an AI chat a question about a small local business. Then paste that business's opening hours into the chat and ask again, telling it to use only what you pasted. Compare.

Quick check: why does RAG reduce hallucinations? (The model answers from real documents you provide instead of guessing from memory.)