How to mitigate AI hallucination with Retrieval-Augmented Generation (RAG) by providing a webpage?
A chat model is trained on a very large body of text, and then that training stops at a cut-off date. From that day on, the model is frozen: it does not read new pages, and it does not see new facts.
Two gaps follow from this. The first is time: anything written after the cut-off date cannot be in its knowledge. The second is coverage: even for text written before that date, the training text cannot cover every website in the world. So, a personal page or a small company site may be absent.
If your webpage is in the second gap, ask the model for a fact that lives only on that page. It will either admit that it does not know, or, worse, invent a confident answer. This note shows the fix in a Python program which fetches the page, keeps its text, and puts the relevant part in front of the model before it answers.
The technique is called retrieval-augmented generation, RAG for short. The name describes the two halves: retrieval finds the text that matters, and generation is the model writing the answer from that text.
The same question, twice
The program asks one question of the biography page: “What is the Chinese name of Howard Lam?” First it asks the model directly, with no webpage attached. Then it asks again, this time with the webpage. The model and the question are identical in both runs, so any difference in the answer comes from the webpage.
Without the webpage, the answer is:
Howard Lam's Chinese name is 林浩然 (Lín Hàorán).
That is a fluent sentence, correctly formatted, and completely wrong. The webpage is simply not in the training text, so the model assembled something that looks like an answer from patterns it has seen. This is what people mean by a hallucination, and the confident tone is part of the problem.
With the webpage, the same model produced this:
林邦源
Short, plain, and correct. The model did not become smarter between the two runs. The model simply had the chunk of characters in front of it, taken from the webpage, and was told to generate an answer from that chunk of characters. So, the second run can give the correct answer.
Step 1: connect to the model
Two connections are needed, and they are different in kind. The chat model is a service you call over the internet; the embedding model runs on your own machine.
import os
from dotenv import load_dotenv
load_dotenv()
from langchain_openai import ChatOpenAI
DEEPSEEK_API_KEY = os.getenv("DEEPSEEK_API_KEY")
my_model = ChatOpenAI(
api_key=DEEPSEEK_API_KEY,
base_url="https://api.deepseek.com",
model_name="deepseek-chat"
)
The first three lines make the key available. import os brings in the standard
operating system module, whose only job here is to read values from the
environment. load_dotenv() reads a file called .env in the same folder
and copies each line of it, such as DEEPSEEK_API_KEY=..., into the environment.
os.getenv("DEEPSEEK_API_KEY") reads the value from the .env file, and
then assigns it to the variable in the Python program on the left-hand side of the
= sign, that is, DEEPSEEK_API_KEY. The key is a password that lets
your program use DeepSeek's service, and that service is an API, short for
application programming interface: the way one program asks another for something over the
internet.
The class is called ChatOpenAI although the model is DeepSeek's. DeepSeek offers an
interface that speaks the same language as OpenAI's, so the same class works; only
base_url and model_name change.
from langchain_huggingface import HuggingFaceEmbeddings
embeddings = HuggingFaceEmbeddings(
model_name="BAAI/bge-m3"
)
The second connection is the embedding model, and it is the one that makes search possible. It
converts text into a list of numbers in such a way that two pieces of text with similar meaning
are close together. bge-m3 is used here because it handles Chinese and
English in the same space, which is why a question in English can find a Chinese name. The
name in front of the slash, BAAI, is the Beijing Academy of Artificial Intelligence, the
group that published the model.
The first run downloads this model, and after that it works offline. There is no charge for using it.
Step 2: fetch the webpage and keep the text
urls = [
"https://pongyuenlam.github.io/biography.html",
]
docs = []
for url in urls:
html = requests.get(url).text
text = BeautifulSoup(html, "html.parser").get_text()
docs.append([Document(page_content=text, metadata={"source": url})])
docs_list = []
for sublist in docs:
for item in sublist:
docs_list.append(item)
urls is the list of web addresses to read, which is why it is written in square
brackets. One is enough here; add another line to the list and each address is fetched in turn.
requests.get(url) downloads the page, and the .text at the end keeps
the whole page as one long string, tags and all.
BeautifulSoup(html, "html.parser").get_text() throws the tags away and keeps only
what a reader would see. That matters: a search over HTML tags would waste its time on
<div> and <nav>. The second argument,
"html.parser", names the parser that BeautifulSoup uses to read the HTML; it ships
with Python, so there is nothing extra to install.
Document is a small container with two parts: the text, and a label recording where
it came from. The label is not used for the answer, but it is how you keep track of which page a
retrieved sentence belongs to once you have several pages.
Notice the square brackets around the Document on the last line: each page goes
into a small list of its own, so docs is a list of lists. The next four lines make
a simple list of documents. The first creates an empty list called docs_list. The
next two go through each sublist in docs, and then through each item in that
sublist. The last line puts each item into docs_list. In the end,
docs_list holds all the documents in one simple list, which is what the text
splitter in the next step needs.
Step 3: cut the text into chunks
text_splitter = RecursiveCharacterTextSplitter(chunk_size=300, chunk_overlap=100)
chunks = text_splitter.split_documents(docs_list)
The page is now one long string, and a long string is a poor unit for search: it covers many topics at once, so no single question matches it well. Cutting it into small pieces gives each idea its own entry.
chunk_size=300 means each piece is about 300 characters, roughly four or five
sentences. chunk_overlap=100 repeats the last 100 characters of one chunk at the
start of the next, so a sentence that is split between two chunks still appears whole in one of
them.
Recursive describes where it prefers to cut. It first tries blank lines, then single line breaks, then the spaces between words. The result keeps whole sentences together as far as the size allows.
split_documents also copies the label onto every chunk. With one webpage address,
the label is the same everywhere and adds nothing; it matters as soon as the list holds more
than one, because then a chunk found later must still say which webpage it came from.
Step 4: turn chunks into numbers
vectorstore = InMemoryVectorStore.from_documents(chunks, embeddings)
retriever = vectorstore.as_retriever(search_kwargs={"k": 1})
from_documents passes every chunk through the embedding model and keeps the
resulting numbers of each chunk, plus the original text, in memory. InMemory is
literal: nothing is written to disk, and the store disappears when the program ends.
as_retriever turns the store into something you can ask a question. Give it a
question, and it hands back the chunks whose meaning is closest to it.
search_kwargs holds the settings for that search, and k is how many
chunks it returns. k=1 asks for the single best match. One is enough here, because
it lets you see exactly what the model was given.
Step 5: ask without the page
print("Before RAG")
before_rag_template = "{topic}"
before_rag_prompt = ChatPromptTemplate.from_template(before_rag_template)
before_rag_chain = before_rag_prompt | my_model | StrOutputParser()
user_question = "What is the Chinese name of Howard Lam?"
print(before_rag_chain.invoke({"topic": user_question}))
The template here does nothing but pass the question through, which is the point: this is the model with nothing else to go on. It is included so that you can see the failure for yourself rather than take my word for it.
The vertical bars build a chain, read left to right. The prompt produces a message, the model
answers that message, and StrOutputParser() turns the answer object into plain
text so that print can show it.
invoke means “run this chain now”, with the input you give it. The input
is a dictionary, {"topic": user_question}, and its key matches the name of the blank
in the template, so {topic} is replaced by the question. That is the only thing the
model receives.
Step 6: ask with the page
print("After RAG")
after_rag_template = """Answer the question based only on the following context:
{context}
Question: {question}
"""
after_rag_prompt = ChatPromptTemplate.from_template(after_rag_template)
after_rag_chain = (
{"context": retriever, "question": RunnablePassthrough()}
| after_rag_prompt
| my_model
| StrOutputParser()
)
results = after_rag_chain.invoke(user_question)
print("Retrieved chunk:", retriever.invoke(user_question)[0].page_content)
print(results)
The triple quotes let the instruction run over several lines, and those line breaks are part of
the text the model receives, which is easier to read than one long line. Two placeholders now
need filling. {context} is filled by the retriever, which takes the question,
searches the store, and returns the best chunk. {question} is filled by
RunnablePassthrough(), whose job is to hand the input on unchanged.
The instruction is the second half of the fix: answer based only on the following context. Without it, the model may mix the chunk with whatever it remembers, and you are back to guessing. With it, an honest “the context does not say” becomes possible.
The last two lines print the retrieved chunk before the answer.
retriever.invoke asks the store the same question a second time, [0]
takes the first result (the only one, because k=1), and .page_content
takes its words. This is the most useful line in the whole program: when an answer looks wrong,
you look here first. If the right sentence was never retrieved, changing the prompt cannot
make up for it.
The whole program
The pieces above, put back together. This is the complete file, 67 lines in all:
import os
from dotenv import load_dotenv
load_dotenv()
import requests
from bs4 import BeautifulSoup
from langchain_core.documents import Document
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_core.vectorstores import InMemoryVectorStore
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnablePassthrough
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_openai import ChatOpenAI
DEEPSEEK_API_KEY = os.getenv("DEEPSEEK_API_KEY")
my_model = ChatOpenAI(
api_key=DEEPSEEK_API_KEY,
base_url="https://api.deepseek.com",
model_name="deepseek-chat"
)
embeddings = HuggingFaceEmbeddings(
model_name="BAAI/bge-m3"
)
urls = [
"https://pongyuenlam.github.io/biography.html",
]
docs = []
for url in urls:
html = requests.get(url).text
text = BeautifulSoup(html, "html.parser").get_text()
docs.append([Document(page_content=text, metadata={"source": url})])
docs_list = []
for sublist in docs:
for item in sublist:
docs_list.append(item)
text_splitter = RecursiveCharacterTextSplitter(chunk_size=300, chunk_overlap=100)
chunks = text_splitter.split_documents(docs_list)
vectorstore = InMemoryVectorStore.from_documents(chunks, embeddings)
retriever = vectorstore.as_retriever(search_kwargs={"k": 1})
print("Before RAG")
before_rag_template = "{topic}"
before_rag_prompt = ChatPromptTemplate.from_template(before_rag_template)
before_rag_chain = before_rag_prompt | my_model | StrOutputParser()
user_question = "What is the Chinese name of Howard Lam?"
print(before_rag_chain.invoke({"topic": user_question}))
print("After RAG")
after_rag_template = """Answer the question based only on the following context:
{context}
Question: {question}
"""
after_rag_prompt = ChatPromptTemplate.from_template(after_rag_template)
after_rag_chain = (
{"context": retriever, "question": RunnablePassthrough()}
| after_rag_prompt
| my_model
| StrOutputParser()
)
results = after_rag_chain.invoke(user_question)
print("Retrieved chunk:", retriever.invoke(user_question)[0].page_content)
print(results)
Running it on your own computer
Download the zip file and unzip it to
a folder. The zip holds the program, a requirements.txt list, and an empty
.env file. Open the folder in Visual Studio Code, then type these four lines in
the terminal:
uv python pin 3.12.10
uv init
uv add -r requirements.txt
uv run rag_webpage_01.py
uv init creates the project file, pyproject.toml, and
uv add -r requirements.txt reads the package list and installs each package into
the project. The first run also downloads the embedding model, so it takes longer than the rest.
Before the program will run, put your DeepSeek key into .env. The file is already
there with an empty value, so add your key after the = sign:
DEEPSEEK_API_KEY=your-key-here
If you keep the API key in .env, you can share your program without handing over
the key. Never share a .env file that holds a real key.
If uv complains about a missing Python version, follow the uv notes above: Windows or Mac.
What this is good for
Anything the model has not learned, provided it sits on a webpage.
What are its limitations?
chunk_size=300andchunk_overlap=100affect the result. An answer to a question may still be separated into two chunks, even with thechunk_overlapset between them.- The model itself has not learned anything. RAG changes what the model reads for one question; it does not move the cut-off date, and the next question without the page is back to guessing.
- One chunk is a small window.
k=1was chosen so you can see the retrieval, but a question whose answer spans two chunks will miss half of it. Raisingkto 3 or 4 is the usual fix. - Search matches meaning, not keywords. That is its strength and its weakness: a question phrased in a different vocabulary can still find the right chunk, but an exact identifier such as a course code may be missed.
- The page is fetched fresh each run. Editing the page changes the answers, which is convenient for a website and risky for an audit trail. And if the page is wrong, the answer will be wrong too.
What comes next
The same six steps work on a PDF instead of a webpage, and that is the next note: swap
requests and BeautifulSoup for a PDF reader, and everything after stays
as it is. The six steps are the method; the file type is only a detail.