How can Retrieval-Augmented Generation (RAG) answer questions without hallucination?
Ask a chat model a question and it generates an answer from what it learned during training. The writing is fluent, but some of the “facts” may be invented — this is called hallucination. Retrieval-Augmented Generation (RAG) turns the order around: first the program finds the relevant lines in a real document, then the model is asked to answer using only those lines.
The program in this note shows the difference in a single run. It asks DeepSeek the question “How many people get headaches at least once a year?” twice: first letting the AI generate an answer purely from its training, then grounding it with the three most relevant sentences from Panadol's headache webpage. The second answer even names the line number it relied on, so you can open the file and check it yourself — in 82 lines.
The idea in one minute
The name carries the three steps. Retrieval: every sentence of the source text is turned into an embedding — a list of numbers that captures its meaning (see the similar-sentence note below) — and the sentences closest in meaning to the question are picked out. Augmented: those sentences are pasted into the prompt as context, so the question arrives at the model with the facts attached. Generation: the model writes the answer, but the instruction tells it to use only the supplied context and to give the line number.
Getting a key first
The chat half of the program talks to DeepSeek, so it needs an API key —
a secret string that works like a password. Sign up at
https://platform.deepseek.com/
and create a key on the API keys page; a small balance is enough for trying things out. The key
is then placed in a file named .env in the same folder, containing one line:
DEEPSEEK_API_KEY=paste-your-key-here
Create this file yourself after downloading the demo; it is not included in the zip, because a key is personal. The DeepSeek note below walks through keys and costs in more detail.
The program, piece by piece
import os
import re
import requests
from bs4 import BeautifulSoup
from dotenv import load_dotenv
load_dotenv()
from langchain_core.documents import Document
from langchain_core.vectorstores import InMemoryVectorStore
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_openai import ChatOpenAI
The first five lines bring in familiar tools: os reads settings stored outside the
code, re cuts text with patterns, requests downloads webpages,
BeautifulSoup picks out the readable parts, and load_dotenv() loads
the .env key file immediately. The langchain_core lines bring in
building blocks from LangChain, a library that snaps steps together with the
pipe symbol |: Document is one piece of text with labels attached,
InMemoryVectorStore is a meaning-based index kept in memory,
ChatPromptTemplate is a message form with blanks to fill,
StrOutputParser unpacks the reply into plain text, and
RunnablePassthrough hands the question through unchanged. Finally,
HuggingFaceEmbeddings turns text into embeddings, and ChatOpenAI is
the chat connector — named after OpenAI's message format, which DeepSeek deliberately
speaks too.
DEEPSEEK_API_KEY = os.getenv("DEEPSEEK_API_KEY")
my_model = ChatOpenAI(
api_key=DEEPSEEK_API_KEY,
base_url="https://api.deepseek.com",
model_name="deepseek-chat"
)
embeddings = HuggingFaceEmbeddings(
model_name="BAAI/bge-m3"
)
Two connections are prepared. my_model points the connector at DeepSeek's address,
carrying the key that load_dotenv() loaded; model_name="deepseek-chat"
picks which model answers, and names do change over time, so check the documentation when in
doubt. embeddings loads BAAI/bge-m3, the same multilingual embedding
model as the similar-sentence note — the first run downloads about 2 GB, after which
the model is saved on the computer.
url = "https://www.panadol.com/en-gb/adult-symptoms/headache/"
headers = {"User-Agent": "Mozilla/5.0 (compatible; SkillBot/1.0)"}
resp = requests.get(url, headers=headers, timeout=15)
resp.raise_for_status()
resp.encoding = resp.apparent_encoding
soup = BeautifulSoup(resp.text, "html.parser")
for tag in soup(["script", "style", "noscript", "nav", "footer"]):
tag.decompose()
lines = [" ".join(tag.get_text(" ").split()) for tag in soup.find_all(["h1", "h2", "h3", "p", "li"])]
lines = [line for line in lines if line != ""]
sentences = []
for line in lines:
for sentence in re.split(r"(?<=[.!?])\s+(?=[A-Z])", line):
sentences.append(sentence)
lines = sentences
with open("panadol.txt", "w", encoding="utf-8") as outfile:
for line in lines:
outfile.write(line + "\n")
This is the web-scraping note in miniature: the Panadol headache page is downloaded (with a
polite User-Agent naming the program), the script, style, navigation and footer
parts are thrown away, and the headings, paragraphs and list items are kept with their
whitespace tidied. The pattern (?<=[.!?])\s+(?=[A-Z]) then cuts each line into
sentences — split after a full stop, question mark or exclamation mark when a space and a
capital letter follow — exactly the splitter from the sentence-splitting note. The result
is saved to panadol.txt, one sentence per line. A peek inside:
Who gets headaches?
Headaches are very common.
Around seven in 10 people get them at least once a year.
Recurring headaches affect approximately half of adults worldwide.
chunked_text_with_metadata = []
for line_num, line in enumerate(lines, start=1):
if line.strip() == "":
continue
content = f"line number {line_num}\n{line.strip()}"
chunked_text_with_metadata.append(
Document(
page_content=content,
metadata={"source": url, "line": line_num},
)
)
Every sentence is wrapped as a Document. Notice what goes into
page_content: the words line number 128 (or whichever number) on one
line, then the sentence itself. The metadata also records the source address and the line
number. Putting the number inside the text the model will read is what lets the model
quote it later — it literally sees the label above the sentence it uses.
vectorstore = InMemoryVectorStore.from_documents(chunked_text_with_metadata, embeddings)
retriever = vectorstore.as_retriever(search_kwargs={"k": 3})
from_documents turns every Document into an embedding and keeps the
set in memory — in memory means the index is rebuilt on each run and vanishes
when the program ends, which is fine for one webpage. as_retriever with
"k": 3 creates a finder that, given a question, returns the three
Documents closest in meaning to it.
print("Before RAG")
before_rag_template = "{topic}"
before_rag_prompt = ChatPromptTemplate.from_template(before_rag_template)
before_rag_chain = before_rag_prompt | my_model | StrOutputParser()
user_question = "How many people get headaches at least once a year?"
print(before_rag_chain.invoke({"topic": user_question}))
The first ask is deliberately plain: the template "{topic}" contains nothing but
the question, so the model answers purely from its training. The chain reads left to right through the
pipe symbol: fill the template, send it to DeepSeek, unpack the reply into text.
invoke({"topic": user_question}) fills the blank and runs the chain. The reply is
fluent and detailed — but there is no source to check it against.
print("After RAG")
after_rag_template = """Answer the question based only on the following context:
{context} and give line number as well
Question: {question}
"""
after_rag_prompt = ChatPromptTemplate.from_template(after_rag_template)
after_rag_chain = (
{"context": retriever, "question": RunnablePassthrough()}
| after_rag_prompt
| my_model
| StrOutputParser()
)
results = after_rag_chain.invoke(user_question)
print(results)
The second ask is the RAG one. The template now has two blanks and one firm instruction: answer
based only on the following context, and give the line number. In the chain,
{"context": retriever, "question": RunnablePassthrough()} fills the two blanks
differently: the question is sent to the retriever, which pastes the three closest labelled
sentences into {context}, while RunnablePassthrough() copies the
question itself into {question} unchanged. Then template, model, plain text as
before. The answer now comes from the retrieved lines — and names where it came from.
The whole program
import os
import re
import requests
from bs4 import BeautifulSoup
from dotenv import load_dotenv
load_dotenv()
from langchain_core.documents import Document
from langchain_core.vectorstores import InMemoryVectorStore
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_openai import ChatOpenAI
DEEPSEEK_API_KEY = os.getenv("DEEPSEEK_API_KEY")
my_model = ChatOpenAI(
api_key=DEEPSEEK_API_KEY,
base_url="https://api.deepseek.com",
model_name="deepseek-chat"
)
embeddings = HuggingFaceEmbeddings(
model_name="BAAI/bge-m3"
)
url = "https://www.panadol.com/en-gb/adult-symptoms/headache/"
headers = {"User-Agent": "Mozilla/5.0 (compatible; SkillBot/1.0)"}
resp = requests.get(url, headers=headers, timeout=15)
resp.raise_for_status()
resp.encoding = resp.apparent_encoding
soup = BeautifulSoup(resp.text, "html.parser")
for tag in soup(["script", "style", "noscript", "nav", "footer"]):
tag.decompose()
lines = [" ".join(tag.get_text(" ").split()) for tag in soup.find_all(["h1", "h2", "h3", "p", "li"])]
lines = [line for line in lines if line != ""]
sentences = []
for line in lines:
for sentence in re.split(r"(?<=[.!?])\s+(?=[A-Z])", line):
sentences.append(sentence)
lines = sentences
with open("panadol.txt", "w", encoding="utf-8") as outfile:
for line in lines:
outfile.write(line + "\n")
chunked_text_with_metadata = []
for line_num, line in enumerate(lines, start=1):
if line.strip() == "":
continue
content = f"line number {line_num}\n{line.strip()}"
chunked_text_with_metadata.append(
Document(
page_content=content,
metadata={"source": url, "line": line_num},
)
)
vectorstore = InMemoryVectorStore.from_documents(chunked_text_with_metadata, embeddings)
retriever = vectorstore.as_retriever(search_kwargs={"k": 3})
print("Before RAG")
before_rag_template = "{topic}"
before_rag_prompt = ChatPromptTemplate.from_template(before_rag_template)
before_rag_chain = before_rag_prompt | my_model | StrOutputParser()
user_question = "How many people get headaches at least once a year?"
print(before_rag_chain.invoke({"topic": user_question}))
print("After RAG")
after_rag_template = """Answer the question based only on the following context:
{context} and give line number as well
Question: {question}
"""
after_rag_prompt = ChatPromptTemplate.from_template(after_rag_template)
after_rag_chain = (
{"context": retriever, "question": RunnablePassthrough()}
| after_rag_prompt
| my_model
| StrOutputParser()
)
results = after_rag_chain.invoke(user_question)
print(results)
Running it on your own computer
Download the zip file and
unzip it to a folder. The zip holds the program and a pyproject.toml project file.
Add your own .env file with the key, as shown above. Then open the folder in
Visual Studio Code and type these three lines in the terminal, one at a time:
uv python pin 3.12.10
uv sync
uv run rag_sentence01.py
The program downloads the webpage, writes panadol.txt, builds the meaning index,
and prints both answers. The first run also downloads the embedding model (about 2 GB), so
it needs the internet once more; after that the model is saved on the computer. A real run
looks like this:
Before RAG
Roughly **half of all adults worldwide** — about **50%** — report having a headache at least once in a given year. That figure comes from global estimates (e.g., WHO and the Global Burden of Disease studies), which put the 1-year prevalence of any headache disorder at around 50% of adults.
Some context on the breakdown:
- **Any headache disorder:** ~50% of adults per year
- **Migraine:** ~14–15% of adults per year
- **Tension-type headache:** ~30–40% of adults per year
- **Chronic daily headache:** ~3–4% of adults
So in absolute terms, with ~8 billion people globally and adults making up roughly 75% of the population, that's on the order of **2.5–3 billion people** experiencing at least one headache per year.
After RAG
Based on the provided context, recurring headaches affect approximately **half of adults worldwide** (line number 128). The context does not specifically state "at least once a year," but this is the closest relevant information available.
Compare the two. The Before RAG answer is confident and detailed, names studies, and
gives you nothing to open and verify. The After RAG answer says only what the
retrieved lines say, points to line 128 of panadol.txt — open the file and
the sentence is right there — and even admits that the lines do not literally say
“at least once a year”. That honesty is the grounding working. Your own wording and
line numbers may differ, because the model writes a fresh reply each time and the live webpage
can change.
If you have not set up uv yet, start with How to install Visual Studio Code? and then the uv note for Windows or Mac.
Limitations
- It needs a key, the internet, and a little money. Without a valid key in
.env, or without a connection, the program stops with an error; DeepSeek charges per amount of text processed, and a question like this one costs a fraction of a cent. - Grounded does not mean guaranteed. The model can still misread or over-combine the retrieved lines. The defence is that every claim can be traced to a numbered line, so a wrong answer can be caught instead of believed.
- Only as good as the source page. If the page does not contain the answer, the three retrieved lines will not contain it either, and the reply will be thin or will say so.
- The scrape is live. If Panadol edits or restructures the page,
panadol.txtand its line numbers change, and a heavy redesign could break the scraping step altogether. - Model names change.
deepseek-chatis the name as of writing; older names may stop working, so check the documentation when in doubt.
Further reading
- The DeepSeek API documentation lists the current model names.
- The LangChain RAG tutorial shows the same retrieve-then-answer pattern with more options.
- How can a Python program use the DeepSeek API? explains the chat half of this program — key, connector, one call.
- How can a Python program find the sentence most similar to a question? explains the retrieval half, with no chat model involved.
- How does web scraping turn a webpage into a file of sentences? covers the scraping step that produces
panadol.txt. - How does a program split a text into sentences? explains the cutting pattern used above.
- How to mitigate AI hallucination with Retrieval-Augmented Generation (RAG) by providing a webpage? is an earlier version that hands the model webpage chunks instead of numbered sentences.
- Lam, Pong Yuen (2026), “Natural Language Processing: Foundations and Applications without Hallucination from Generative AI”, Working Paper. Available at SSRN: https://doi.org/10.2139/ssrn.7384558.