How to use an AI agent to remove repeated sentences from a draft (Research, version 5)?
The previous version used three tools:
search_internet,
generate_draft and
write_draft.
This version adds a fourth tool,
enhance_draft, that reads the finished draft and
removes the sentences that add nothing new. Every sentence is still copied,
and every endnote still points to a page. There are simply fewer of both, and
each of the remaining sentences is there for a reason.
One search per run
Before the new tool, one small change in the search tool. It now keeps its
answer in a file called tavily_search.json, and
reads that file if it already exists.
if os.path.exists("tavily_search.json"):
with open("tavily_search.json", "r", encoding="utf-8") as f:
response = json.load(f)
else:
response = tavily_client.search(query, max_results=10)
with open("tavily_search.json", "w", encoding="utf-8") as f:
json.dump(response, f, ensure_ascii=False, indent=2)
os.path.exists asks a simple question: is this
file already in the folder? If it is, the saved answer is read back with
json.load and no search is made. If it is not,
Tavily is called and the answer is written out with
json.dump.
Near the bottom of the program, the file is deleted before the run starts, so the first search of every run is fresh:
if os.path.exists("tavily_search.json"):
os.remove("tavily_search.json") # start each run with a fresh search
indent=2 makes the file readable when you open
it, and ensure_ascii=False keeps non-English
characters as they are instead of turning them into escape codes. The file is
worth a look after a run: it holds everything Tavily returned, not just the
200 characters the screen shows.
Two more shared lists
The generate_draft tool already had to leave its
work where the next tool could find it. The
enhance_draft tool needs two more places:
draft_items = []
skipped_items = []
draft_items holds each sentence from the draft
together with the page it came from, which is what the new tool reads.
skipped_items holds the sentences that were
removed.
Tool four: enhance_draft
return_direct=True sits on the tool whose result
should be the final answer.
@tool(return_direct=True)
def enhance_draft(keep: list[int]):
"""
Write an enhanced draft from the draft made by write_draft.
keep: the endnote numbers of the sentences to keep. Sentence 1 is always kept.
Read sentences 2, 3, ... one by one, and keep a sentence only if it adds
useful new information that the sentences already kept do not give.
"""
The input is a list of whole numbers:
keep: list[int]. The model decides which
sentences survive — that is a judgement about meaning, and a model is
good at it. The code that follows does the exact part: renumbering the
endnotes, and recording what was dropped.
Inside the tool
skipped_items = []
s = ""
endnotes = ""
endnote_num = 1
for i, item in enumerate(draft_items, start=1):
if i != 1 and i not in keep:
skipped_items.append({"num": i, "sentence": item["sentence"], "url": item["url"]})
continue # skip sentences that add little new information
s = s + item["sentence"] + "[" + str(endnote_num) + "]\n"
endnotes = endnotes + f"[{endnote_num}] {item['url']}\n"
endnote_num = endnote_num + 1
enumerate(draft_items, start=1) walks the draft
and hands over two things each time: the position i
and the item itself. Positions start at 1, not 0, because the numbers have to
match the endnote numbers the reader sees.
The test is two conditions joined by and.
i != 1 protects the first sentence: it is kept
no matter what the model says. Then
i not in keep asks whether this sentence is one
the model chose to drop. Both conditions must be true before a sentence is
removed.
A removed sentence is written into skipped_items
with its old number, its text and its link, so the program can report it
later. continue then jumps to the next sentence,
which means the two lines below never run and the sentence gets no endnote
number. The counter only moves for sentences that are kept, so the new draft
is numbered [1],
[2], [3] …
with no gaps.
In the demo run the model keeps 5 sentences 1, 2, 4, 6 and 7, so they become sentences 1, 2, 3, 4 and 5 in the enhanced draft.
The AI Agent
agent = create_agent(
model=model,
tools=[search_internet, generate_draft, write_draft, enhance_draft],
system_prompt="""
You are a helpful assistant.
Always call search_internet, using the user's message exactly as the query.
Do not rephrase, shorten, translate, or add anything to it.
Then call generate_draft to select the useful results.
Then go through each sentence in each selected result, pick the one sentence
closest in meaning to the user's query, and copy it exactly.
A complete sentence has a subject and a verb and ends with a full stop;
table rows, headings, menus and fragments are not complete sentences.
If a result has no complete sentence, use an empty string "" for it.
Call write_draft with these sentences in the same order as the results.
Then read the draft from write_draft. Keep sentence 1.
Read the remaining sentences one by one, and keep a sentence only if it adds
useful new information that the sentences already kept do not give.
Skip a sentence that only repeats a similar figure already given.
Keep a sentence that adds a new fact, such as a ranking, a trend,
a breakdown, or an official government source.
Call enhance_draft with the endnote numbers of the sentences to keep.
""",
)
The tools list now holds
search_internet,
generate_draft,
write_draft and
enhance_draft. The prompt is what tells the
model which one to call, and in what order.
Two key sentences in the prompt are the following, Read the remaining sentences one by one, and keep a sentence only if it adds useful new information that the sentences already kept do not give. They can then help us reduce duplication of sentences.
Results
The following block prints the results from two tools,
write_draft and
enhance_draft.
for message in result["messages"]:
if message.type == "tool" and message.name in ["write_draft", "enhance_draft"]:
print(f"Tool: {message.name}\n")
print(message.content)
elif message.type == "tool":
response = json.loads(message.content)
print(f"Tool: {message.name}")
The following prints the sentences skipped by
enhance_draft for our reference:
print("Sentences skipped by enhance_draft:\n")
if skipped_items == []:
print("None\n")
for item in skipped_items:
print(f"[{item['num']} in write_draft] {item['sentence']}")
print(f" {item['url']}\n")
What comes out
The screen now prints four tool blocks —
search_internet,
generate_draft,
write_draft and
enhance_draft — and then the list of
sentences that were removed. Here is the enhanced draft:
Tool: enhance_draft
China, Hong Kong SAR 2026 population is estimated at 7,378,602 people at mid-year.[1]
As of 2026, Hong Kong has a total population of 7,378,602, ranking as the 107th most populous nation in the world.[2]
Total current population for Hong Kong in 2026 is 7,500,962, a 0.02% increase from 2025.[3]
Hong Kong's population was approximately 7.50 million in mid-2025.[4]
Hong Kong is a largely homogenous society, with about 91.6% of its people being Chinese (ethnically speaking, Han Chinese).[5]
Endnotes:
[1] https://www.worldometers.info/world-population/china-hong-kong-sar-population
[2] https://worldpopulationreview.com/countries/hong-kong
[3] https://www.macrotrends.net/global-metrics/countries/hkg/hong-kong/population
[4] https://www.gov.hk/en/about/abouthk/facts.htm
[5] https://www.had.gov.hk/rru/en/info/demographics.htm
Seven sentences became five, and the endnotes were numbered again from 1, so nothing is missing in the middle.
Then comes the part that makes the choice checkable. This is what the run removed:
Sentences skipped by enhance_draft:
[3 in write_draft] The population of Hong Kong is 7.5 million.
https://study.com/academy/lesson/hong-kong-facts-population.html
[5 in write_draft] In 2023, the estimated population of Hong Kong was 7.443 million.
https://ourworldindata.org/profile/population-demography/hong-kong
Both were dropped for the same reason: the figure was already covered. The study.com page says 7.5 million without a year, the Our World in Data page gives 7.443 million for 2023, and the draft already carried 7,500,962 for 2026 and 7.50 million for mid-2025.
The result is not perfect, and it is worth saying so. Sentences
[1] and [2] both
state 7,378,602, which is a repetition by any measure; the second survived
because it also adds the 107th-place ranking. A tool that removed everything
repeated twice would have taken the ranking away with it.
The surviving five still disagree with one another: 7,378,602, 7,500,962 and 7.50 million are three different answers to the same question. Removing repetition does not remove disagreement, and this version does not claim to.
We can enhance the prompt by adding one more tool, such as
generate_summary, to read the values in each
sentence and say, for example, an average or a median of the numbers found.
The whole program
import os
import json
from dotenv import load_dotenv
from tavily import TavilyClient
from langchain_core.tools import tool
from langchain_openai import ChatOpenAI
from langchain.agents import create_agent
load_dotenv()
DEEPSEEK_API_KEY = os.getenv("DEEPSEEK_API_KEY")
TAVILY_API_KEY = os.getenv("TAVILY_API_KEY")
tavily_client = TavilyClient(api_key=TAVILY_API_KEY)
last_search = {}
selected_results = []
draft_items = []
skipped_items = []
@tool
def search_internet(query: str):
"""
Search the internet for up-to-date information about the query
and return the full Tavily search response.
"""
global last_search
if os.path.exists("tavily_search.json"):
with open("tavily_search.json", "r", encoding="utf-8") as f:
response = json.load(f)
else:
response = tavily_client.search(query, max_results=10)
with open("tavily_search.json", "w", encoding="utf-8") as f:
json.dump(response, f, ensure_ascii=False, indent=2)
last_search = response
return response
@tool
def generate_draft():
"""
Go through the results from the latest search_internet call.
Keep results with a score of 0.5 or higher, but never keep Wikipedia.
Return the selected results.
"""
global selected_results
selected = []
for r in last_search.get("results", []):
if r["score"] >= 0.5 and "wikipedia.org" not in r["url"]:
selected.append(r)
selected_results = selected
return {"query": last_search.get("query"), "results": selected}
@tool
def write_draft(sentences: list[str]):
"""
Write a draft with endnotes from the results of generate_draft.
sentences: one sentence per result from generate_draft, in the same order.
Each sentence must be copied exactly from that result's content, and be
the sentence closest in meaning to the user's query.
Use an empty string "" for a result that has no complete sentence.
"""
global draft_items
draft_items = []
s = ""
endnotes = ""
endnote_num = 1
for sentence, r in zip(sentences, selected_results):
if sentence.strip() == "":
continue # skip results with no complete sentence
s = s + sentence + "[" + str(endnote_num) + "]\n"
endnotes = endnotes + f"[{endnote_num}] {r['url']}\n"
draft_items.append({"sentence": sentence, "url": r["url"]})
endnote_num = endnote_num + 1
s = s + "\nEndnotes:\n" + endnotes
return s
@tool(return_direct=True)
def enhance_draft(keep: list[int]):
"""
Write an enhanced draft from the draft made by write_draft.
keep: the endnote numbers of the sentences to keep. Sentence 1 is always kept.
Read sentences 2, 3, ... one by one, and keep a sentence only if it adds
useful new information that the sentences already kept do not give.
"""
global skipped_items
skipped_items = []
s = ""
endnotes = ""
endnote_num = 1
for i, item in enumerate(draft_items, start=1):
if i != 1 and i not in keep:
skipped_items.append({"num": i, "sentence": item["sentence"], "url": item["url"]})
continue # skip sentences that add little new information
s = s + item["sentence"] + "[" + str(endnote_num) + "]\n"
endnotes = endnotes + f"[{endnote_num}] {item['url']}\n"
endnote_num = endnote_num + 1
s = s + "\nEndnotes:\n" + endnotes
return s
model = ChatOpenAI(
model="deepseek-flash",
api_key=DEEPSEEK_API_KEY,
base_url="https://api.deepseek.com",
temperature=0,
)
agent = create_agent(
model=model,
tools=[search_internet, generate_draft, write_draft, enhance_draft],
system_prompt="""
You are a helpful assistant.
Always call search_internet, using the user's message exactly as the query.
Do not rephrase, shorten, translate, or add anything to it.
Then call generate_draft to select the useful results.
Then go through each sentence in each selected result, pick the one sentence
closest in meaning to the user's query, and copy it exactly.
A complete sentence has a subject and a verb and ends with a full stop;
table rows, headings, menus and fragments are not complete sentences.
If a result has no complete sentence, use an empty string "" for it.
Call write_draft with these sentences in the same order as the results.
Then read the draft from write_draft. Keep sentence 1.
Read the remaining sentences one by one, and keep a sentence only if it adds
useful new information that the sentences already kept do not give.
Skip a sentence that only repeats a similar figure already given.
Keep a sentence that adds a new fact, such as a ranking, a trend,
a breakdown, or an official government source.
Call enhance_draft with the endnote numbers of the sentences to keep.
""",
)
if os.path.exists("tavily_search.json"):
os.remove("tavily_search.json") # start each run with a fresh search
prompt = "What is the population of Hong Kong?"
result = agent.invoke({
"messages": [{"role": "user", "content": prompt}]
})
for message in result["messages"]:
if message.type == "tool" and message.name in ["write_draft", "enhance_draft"]:
print(f"Tool: {message.name}\n")
print(message.content)
elif message.type == "tool":
response = json.loads(message.content)
print(f"Tool: {message.name}")
print(f"Query: {response['query']}\n")
for i, r in enumerate(response["results"], start=1):
print(f"{i}. {r['title']} (score: {r['score']:.2f})")
print(f" {r['url']}")
print(f" {r['content'][:200].replace(chr(10), ' ')}...\n")
print("Sentences skipped by enhance_draft:\n")
if skipped_items == []:
print("None\n")
for item in skipped_items:
print(f"[{item['num']} in write_draft] {item['sentence']}")
print(f" {item['url']}\n")
Running it on your own computer
Download the demo program
(Folder-AI-Research05.zip). The zip holds three files:
research05.py (the full script), pyproject.toml and
README.md. This version calls a language model, so it
needs both a Tavily key and a DeepSeek key.
- Open the folder in Visual Studio Code.
-
Create a file named
.envin the folder with two lines:
A Tavily key is available from the Tavily dashboard, and a DeepSeek key from the DeepSeek console. Usage is billed by each provider, so check the current pricing terms there before you run it.TAVILY_API_KEY=your-tavily-key DEEPSEEK_API_KEY=your-deepseek-key - In the Terminal, run these three lines, one at a time:
uv python pin 3.12.10uv syncuv run research05.py - You should see four tool blocks in turn — ten results from the search,
seven from
generate_draft, a seven-sentence draft fromwrite_draft, then the shortened draft fromenhance_draft— followed by the list of skipped sentences. The run takes a few seconds.
If you have not set up uv yet, start with How to install Visual Studio Code? and then the uv note for Windows or Mac.
Limitations
-
Repetition is judged, not measured. There is no rule for how
similar two sentences must be before one is dropped, so a second run can keep
a different set.
temperature=0makes that less likely, not impossible. -
Some repetition survives. Sentences
[1]and[2]in the demo give the same figure twice, because the second also carries the ranking. The tool trades one kind of redundancy for a better draft. - The numbers still disagree. Three figures remain, and the draft cannot say which is right.
Further reading
- How to use an AI agent to write a draft with endnotes (Research, version 4)? — where the seven-sentence draft comes from.
- How to use an AI agent to select useful search results (Research, version 3)? — the first tool that makes a judgement about which results to keep.
- LangChain documentation — tools,
return_directandcreate_agent. - Tavily documentation — the
searchcall and its response fields.