English 繁體中文 简体中文

How can Retrieval-Augmented Generation (RAG) answer questions without hallucination?

Howard Pong Yuen Lam · AI Notes · November 3, 2026 · 8 min read

Ask a chat model a question and it generates an answer from what it learned during training. The writing is fluent, but some of the “facts” may be invented — this is called hallucination. Retrieval-Augmented Generation (RAG) turns the order around: first the program finds the relevant lines in a real document, then the model is asked to answer using only those lines.

The program in this note shows the difference in a single run. It asks DeepSeek the question “How many people get headaches at least once a year?” twice: first letting the AI generate an answer purely from its training, then grounding it with the three most relevant sentences from Panadol's headache webpage. The second answer even names the line number it relied on, so you can open the file and check it yourself — in 82 lines.

The idea in one minute

The name carries the three steps. Retrieval: every sentence of the source text is turned into an embedding — a list of numbers that captures its meaning (see the similar-sentence note below) — and the sentences closest in meaning to the question are picked out. Augmented: those sentences are pasted into the prompt as context, so the question arrives at the model with the facts attached. Generation: the model writes the answer, but the instruction tells it to use only the supplied context and to give the line number.

Getting a key first

The chat half of the program talks to DeepSeek, so it needs an API key — a secret string that works like a password. Sign up at https://platform.deepseek.com/ and create a key on the API keys page; a small balance is enough for trying things out. The key is then placed in a file named .env in the same folder, containing one line:

DEEPSEEK_API_KEY=paste-your-key-here

Create this file yourself after downloading the demo; it is not included in the zip, because a key is personal. The DeepSeek note below walks through keys and costs in more detail.

The program, piece by piece

import os
import re
import requests
from bs4 import BeautifulSoup
from dotenv import load_dotenv
load_dotenv()
from langchain_core.documents import Document
from langchain_core.vectorstores import InMemoryVectorStore
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_openai import ChatOpenAI

The first five lines bring in familiar tools: os reads settings stored outside the code, re cuts text with patterns, requests downloads webpages, BeautifulSoup picks out the readable parts, and load_dotenv() loads the .env key file immediately. The langchain_core lines bring in building blocks from LangChain, a library that snaps steps together with the pipe symbol |: Document is one piece of text with labels attached, InMemoryVectorStore is a meaning-based index kept in memory, ChatPromptTemplate is a message form with blanks to fill, StrOutputParser unpacks the reply into plain text, and RunnablePassthrough hands the question through unchanged. Finally, HuggingFaceEmbeddings turns text into embeddings, and ChatOpenAI is the chat connector — named after OpenAI's message format, which DeepSeek deliberately speaks too.

DEEPSEEK_API_KEY = os.getenv("DEEPSEEK_API_KEY")
my_model = ChatOpenAI(
    api_key=DEEPSEEK_API_KEY,
    base_url="https://api.deepseek.com",
    model_name="deepseek-chat"
)

embeddings = HuggingFaceEmbeddings(
    model_name="BAAI/bge-m3"
)

Two connections are prepared. my_model points the connector at DeepSeek's address, carrying the key that load_dotenv() loaded; model_name="deepseek-chat" picks which model answers, and names do change over time, so check the documentation when in doubt. embeddings loads BAAI/bge-m3, the same multilingual embedding model as the similar-sentence note — the first run downloads about 2 GB, after which the model is saved on the computer.

url = "https://www.panadol.com/en-gb/adult-symptoms/headache/"
headers = {"User-Agent": "Mozilla/5.0 (compatible; SkillBot/1.0)"}
resp = requests.get(url, headers=headers, timeout=15)
resp.raise_for_status()
resp.encoding = resp.apparent_encoding
soup = BeautifulSoup(resp.text, "html.parser")
for tag in soup(["script", "style", "noscript", "nav", "footer"]):
    tag.decompose()
lines = [" ".join(tag.get_text(" ").split()) for tag in soup.find_all(["h1", "h2", "h3", "p", "li"])]
lines = [line for line in lines if line != ""]

sentences = []
for line in lines:
    for sentence in re.split(r"(?<=[.!?])\s+(?=[A-Z])", line):
        sentences.append(sentence)
lines = sentences

with open("panadol.txt", "w", encoding="utf-8") as outfile:
    for line in lines:
        outfile.write(line + "\n")

This is the web-scraping note in miniature: the Panadol headache page is downloaded (with a polite User-Agent naming the program), the script, style, navigation and footer parts are thrown away, and the headings, paragraphs and list items are kept with their whitespace tidied. The pattern (?<=[.!?])\s+(?=[A-Z]) then cuts each line into sentences — split after a full stop, question mark or exclamation mark when a space and a capital letter follow — exactly the splitter from the sentence-splitting note. The result is saved to panadol.txt, one sentence per line. A peek inside:

Who gets headaches?
Headaches are very common.
Around seven in 10 people get them at least once a year.
Recurring headaches affect approximately half of adults worldwide.
chunked_text_with_metadata = []
for line_num, line in enumerate(lines, start=1):
    if line.strip() == "":
        continue
    content = f"line number {line_num}\n{line.strip()}"
    chunked_text_with_metadata.append(
        Document(
            page_content=content,
            metadata={"source": url, "line": line_num},
        )
    )

Every sentence is wrapped as a Document. Notice what goes into page_content: the words line number 128 (or whichever number) on one line, then the sentence itself. The metadata also records the source address and the line number. Putting the number inside the text the model will read is what lets the model quote it later — it literally sees the label above the sentence it uses.

vectorstore = InMemoryVectorStore.from_documents(chunked_text_with_metadata, embeddings)
retriever = vectorstore.as_retriever(search_kwargs={"k": 3})

from_documents turns every Document into an embedding and keeps the set in memory — in memory means the index is rebuilt on each run and vanishes when the program ends, which is fine for one webpage. as_retriever with "k": 3 creates a finder that, given a question, returns the three Documents closest in meaning to it.

print("Before RAG")
before_rag_template = "{topic}"
before_rag_prompt = ChatPromptTemplate.from_template(before_rag_template)
before_rag_chain = before_rag_prompt | my_model | StrOutputParser()
user_question = "How many people get headaches at least once a year?"
print(before_rag_chain.invoke({"topic": user_question}))

The first ask is deliberately plain: the template "{topic}" contains nothing but the question, so the model answers purely from its training. The chain reads left to right through the pipe symbol: fill the template, send it to DeepSeek, unpack the reply into text. invoke({"topic": user_question}) fills the blank and runs the chain. The reply is fluent and detailed — but there is no source to check it against.

print("After RAG")
after_rag_template = """Answer the question based only on the following context:
{context} and give line number as well
Question: {question}
"""
after_rag_prompt = ChatPromptTemplate.from_template(after_rag_template)
after_rag_chain = (
    {"context": retriever, "question": RunnablePassthrough()}
    | after_rag_prompt
    | my_model
    | StrOutputParser()
)
results = after_rag_chain.invoke(user_question)
print(results)

The second ask is the RAG one. The template now has two blanks and one firm instruction: answer based only on the following context, and give the line number. In the chain, {"context": retriever, "question": RunnablePassthrough()} fills the two blanks differently: the question is sent to the retriever, which pastes the three closest labelled sentences into {context}, while RunnablePassthrough() copies the question itself into {question} unchanged. Then template, model, plain text as before. The answer now comes from the retrieved lines — and names where it came from.

The whole program

import os
import re
import requests
from bs4 import BeautifulSoup
from dotenv import load_dotenv
load_dotenv()
from langchain_core.documents import Document
from langchain_core.vectorstores import InMemoryVectorStore
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_openai import ChatOpenAI

DEEPSEEK_API_KEY = os.getenv("DEEPSEEK_API_KEY")
my_model = ChatOpenAI(
    api_key=DEEPSEEK_API_KEY,
    base_url="https://api.deepseek.com",
    model_name="deepseek-chat"
)

embeddings = HuggingFaceEmbeddings(
    model_name="BAAI/bge-m3"
)

url = "https://www.panadol.com/en-gb/adult-symptoms/headache/"
headers = {"User-Agent": "Mozilla/5.0 (compatible; SkillBot/1.0)"}
resp = requests.get(url, headers=headers, timeout=15)
resp.raise_for_status()
resp.encoding = resp.apparent_encoding
soup = BeautifulSoup(resp.text, "html.parser")
for tag in soup(["script", "style", "noscript", "nav", "footer"]):
    tag.decompose()
lines = [" ".join(tag.get_text(" ").split()) for tag in soup.find_all(["h1", "h2", "h3", "p", "li"])]
lines = [line for line in lines if line != ""]

sentences = []
for line in lines:
    for sentence in re.split(r"(?<=[.!?])\s+(?=[A-Z])", line):
        sentences.append(sentence)
lines = sentences

with open("panadol.txt", "w", encoding="utf-8") as outfile:
    for line in lines:
        outfile.write(line + "\n")

chunked_text_with_metadata = []
for line_num, line in enumerate(lines, start=1):
    if line.strip() == "":
        continue
    content = f"line number {line_num}\n{line.strip()}"
    chunked_text_with_metadata.append(
        Document(
            page_content=content,
            metadata={"source": url, "line": line_num},
        )
    )

vectorstore = InMemoryVectorStore.from_documents(chunked_text_with_metadata, embeddings)
retriever = vectorstore.as_retriever(search_kwargs={"k": 3})

print("Before RAG")
before_rag_template = "{topic}"
before_rag_prompt = ChatPromptTemplate.from_template(before_rag_template)
before_rag_chain = before_rag_prompt | my_model | StrOutputParser()
user_question = "How many people get headaches at least once a year?"
print(before_rag_chain.invoke({"topic": user_question}))

print("After RAG")
after_rag_template = """Answer the question based only on the following context:
{context} and give line number as well
Question: {question}
"""
after_rag_prompt = ChatPromptTemplate.from_template(after_rag_template)
after_rag_chain = (
    {"context": retriever, "question": RunnablePassthrough()}
    | after_rag_prompt
    | my_model
    | StrOutputParser()
)
results = after_rag_chain.invoke(user_question)
print(results)

Running it on your own computer

Download the zip file and unzip it to a folder. The zip holds the program and a pyproject.toml project file. Add your own .env file with the key, as shown above. Then open the folder in Visual Studio Code and type these three lines in the terminal, one at a time:

uv python pin 3.12.10
uv sync
uv run rag_sentence01.py

The program downloads the webpage, writes panadol.txt, builds the meaning index, and prints both answers. The first run also downloads the embedding model (about 2 GB), so it needs the internet once more; after that the model is saved on the computer. A real run looks like this:

Before RAG
Roughly **half of all adults worldwide** — about **50%** — report having a headache at least once in a given year. That figure comes from global estimates (e.g., WHO and the Global Burden of Disease studies), which put the 1-year prevalence of any headache disorder at around 50% of adults.

Some context on the breakdown:
- **Any headache disorder:** ~50% of adults per year
- **Migraine:** ~14–15% of adults per year
- **Tension-type headache:** ~30–40% of adults per year
- **Chronic daily headache:** ~3–4% of adults

So in absolute terms, with ~8 billion people globally and adults making up roughly 75% of the population, that's on the order of **2.5–3 billion people** experiencing at least one headache per year.
After RAG
Based on the provided context, recurring headaches affect approximately **half of adults worldwide** (line number 128). The context does not specifically state "at least once a year," but this is the closest relevant information available.

Compare the two. The Before RAG answer is confident and detailed, names studies, and gives you nothing to open and verify. The After RAG answer says only what the retrieved lines say, points to line 128 of panadol.txt — open the file and the sentence is right there — and even admits that the lines do not literally say “at least once a year”. That honesty is the grounding working. Your own wording and line numbers may differ, because the model writes a fresh reply each time and the live webpage can change.

If you have not set up uv yet, start with How to install Visual Studio Code? and then the uv note for Windows or Mac.

Limitations

Further reading