rag-hallucination-mitigation
verifiedce01221d-7550-41ae-aef1-12a5cb1ee453
Reduce hallucination in RAG answers — grounding checks, retrieval quality, citation, faithfulness evals, temperature/decoding, and refusal on low-confidence.
Metadata
Skill file
# RAG Hallucination Mitigation
Use when your retrieval-augmented app occasionally *confidently* answers things that
aren't supported by the retrieved sources — the highest-stakes failure in RAG. You
can't eliminate it, but you can drive it down systematically and fail loud when it
would happen.
## Where hallucinations come from
- **Retrieval failure** — the right source never made it into context (the top cause).
- **Context overload/confusion** — too many chunks or irrelevant chunks dilute the
model's attention; it leans on parametric memory instead of the sources.
- **The model filling gaps** — when the requested answer isn't in context, a
well-behaved LLM should say so; many don't, and instead pattern-complete.
- **Decoding/decoder temperature** — high temperature invites unfaithful elaboration.
- **Instructions that encourage guessing** — prompts that don't demand grounding.
## Layer 1 — fix retrieval first (biggest lever)
Hallucination is often *unanswerability* dressed up as a model flaw:
- Improve recall — hybrid vector + BM25, reranking, better chunking (see the semantic
chunking and reranking skills). If the answer isn't retrieved, no prompt will save you.
- Return **scores** and threshold: if top-k confidence is low, don't force an answer.
- Prefer **small, relevant** context over maximal context — cap chunk count and total
tokens so the model can actually attend to what matters.
## Layer 2 — ground the generation
- **Add a hard grounding directive** to the system/nugget prompt: "Answer ONLY from
the provided context. If the answer is not in the context, say you don't know."
Few-shot a refusal example so "unknown" is an available, natural output.
- **Require citations:** instruct the model to support claims with source references
(e.g. `[1]`, `[2]`), then post-validate that cited indices exist and actually
contain the claim. This converts silent fabrication into checkable structure.
- **Keep temperature low** (e.g. 0–0.3) for grounded factual Q&A; reserve high
temperature for creative tasks where faithfulness matters less.
## Layer 3 — post-hoc verification (guardrail)
- **Faithfulness check:** after generation, have a judge (rule-based or an LLM judge —
see the LLM-judge skill) compare each claim in the answer against the retrieved
context. Flag any claim not supported.
- **NLI-style scorers** can quantify groundedness; use them to gate output or route
low-faithfulness answers to a retry/fallback path.
- **Citation validation:** if the model cites `[3]`, verify source #3 is in the chunk
set and (roughly) matches the claim.
## Layer 4 — measure it (evals drive the fix)
Build a hallucination eval set and track it:
- **Faithfulness / groundedness metric** — fraction of answer claims supported by
context.
- **Answer relevance** — is the answer actually addressing the question.
- **Refusal rate on unanswerable items** — you *want* the model to decline when it
should; a high hallucination rate on unanswerable queries is the worst case.
- Compare metrics across retrieval changes, prompt changes, and temperature to make
the improvement measurable, not anecdotal. (Full harness: see the RAG-eval skill.)
## Pitfalls
- **Blame the model when the source is missing** — always check retrieval recall first;
a hallucination is frequently an *unanswerable query with a too-permissive model*.
- **Temperature too high** for factual questions — invite elaboration.
- **Forcing an answer every time** — refuse when confidence is low; an honest "not in
the sources" beats a confident lie.
- **Citations unchecked** — telling the model to cite but never validating gives a
false sense of grounding (it can fabricate citations too).
- **No faithfulness eval** — you can't manage what you don't measure; a small labeled
set catches regressions when you tweak the prompt or chunker.
## Verify
- On an unanswerable eval subset, the model refuses (or says "not in sources") instead
of inventing — measure the refusal/groundedness rate.
- Retrieved-then-generated answers cite sources that pass a consistency check.
- Faithfulness metric stays above your target across prompt/retrieval changes.
- A spot-check of high-risk queries (numbers, names, dates) shows no fabrications in
the cited passages.