English 繁體中文 简体中文

如何使用 AI 代理删除草稿中的重复句子(Research,版本 5)?

林邦源博士 · AI 笔记 · 2026 年 11 月 9 日 · 约 5 分钟阅读

上一个版本用了三个工具: search_internet、 generate_draft 和 write_draft。

这个版本加入第四个工具 enhance_draft,它读取已完成的草稿,删去没有 带来新东西的句子。每一句仍然是原文照抄,每一个尾注仍然指向一个网页。只是 两者的数目都少了,而剩下的每一句都有它留下的理由。

每次运行只搜索一次

在新工具之前,搜索工具有一点小改动。它现在把答案存进名为 tavily_search.json 的文件,如果文件已经存在, 就直接读取它。

    if os.path.exists("tavily_search.json"):
        with open("tavily_search.json", "r", encoding="utf-8") as f:
            response = json.load(f)
    else:
        response = tavily_client.search(query, max_results=10)
        with open("tavily_search.json", "w", encoding="utf-8") as f:
            json.dump(response, f, ensure_ascii=False, indent=2)

os.path.exists 问一个简单问题:这个文件是否 已经在文件夹里?如果已在,就用 json.load 读回 已保存的答案,不再搜索。如果不在,就调用 Tavily,并用 json.dump 把答案写出来。

在程序接近底部的位置,文件会在运行开始之前被删除,所以每次运行的第一次 搜索都是最新的:

if os.path.exists("tavily_search.json"):
    os.remove("tavily_search.json")  # start each run with a fresh search

indent=2 让你打开文件时看得清楚,而 ensure_ascii=False 让非英文字符保持原样,不会 变成转义码。运行完之后,这个文件值得打开看看:里面是 Tavily 返回的全部 内容,不只是屏幕上显示的 200 个字符。

另外两个共享的列表

generate_draft 工具本来就要把它的成果放在下 一个工具找得到的地方。enhance_draft 工具还需要 另外两个地方:

draft_items = []
skipped_items = []

draft_items 存放草稿中每一句,连同它来自的 网页,也就是新工具读取的内容。skipped_items 存放被删去的句子。

第四个工具:enhance_draft

return_direct=True 放在那个结果应该成为最终 答案的工具上。

@tool(return_direct=True)
def enhance_draft(keep: list[int]):
    """
    Write an enhanced draft from the draft made by write_draft.
    keep: the endnote numbers of the sentences to keep. Sentence 1 is always kept.
    Read sentences 2, 3, ... one by one, and keep a sentence only if it adds
    useful new information that the sentences already kept do not give.
    """

输入是一串整数:keep: list[int]。哪些句子 留下,由模型决定 —— 这是关于意思的判断,而模型擅长这件事。之后的代码 负责精确的部分:重新编号尾注,并记录被丢弃的句子。

工具内部

    skipped_items = []
    s = ""
    endnotes = ""
    endnote_num = 1
    for i, item in enumerate(draft_items, start=1):
        if i != 1 and i not in keep:
            skipped_items.append({"num": i, "sentence": item["sentence"], "url": item["url"]})
            continue  # skip sentences that add little new information
        s = s + item["sentence"] + "[" + str(endnote_num) + "]\n"
        endnotes = endnotes + f"[{endnote_num}] {item['url']}\n"
        endnote_num = endnote_num + 1

enumerate(draft_items, start=1) 逐项走过草稿, 每次都交出两样东西:位置 i 和该项本身。 位置从 1 开始而不是 0,因为编号必须与读者看到的尾注号码一致。

这个判断由两个条件以 and 连接。 i != 1 保护第一句:无论模型说什么,它都会 保留。然后 i not in keep 问这一句是否属于 模型选择舍弃的句子。两个条件都成立,句子才会被删去。

被删去的句子会连同原有编号、文字和链接写进 skipped_items,程序之后就可以报告它。 continue 然后跳到下一句,也就是下面两行不会 运行,这一句不会得到尾注号码。计数器只为保留下来的句子移动,所以新草稿的 编号是 [1]、 [2]、[3] ……,中间没有跳号。

在演示运行中,模型保留了第 1、2、4、6、7 句,共 5 句,所以它们在增强版 草稿中成为第 1、2、3、4、5 句。

AI 代理

agent = create_agent(
    model=model,
    tools=[search_internet, generate_draft, write_draft, enhance_draft],
    system_prompt="""
You are a helpful assistant.
Always call search_internet, using the user's message exactly as the query.
Do not rephrase, shorten, translate, or add anything to it.
Then call generate_draft to select the useful results.
Then go through each sentence in each selected result, pick the one sentence
closest in meaning to the user's query, and copy it exactly.
A complete sentence has a subject and a verb and ends with a full stop;
table rows, headings, menus and fragments are not complete sentences.
If a result has no complete sentence, use an empty string "" for it.
Call write_draft with these sentences in the same order as the results.
Then read the draft from write_draft. Keep sentence 1.
Read the remaining sentences one by one, and keep a sentence only if it adds
useful new information that the sentences already kept do not give.
Skip a sentence that only repeats a similar figure already given.
Keep a sentence that adds a new fact, such as a ranking, a trend,
a breakdown, or an official government source.
Call enhance_draft with the endnote numbers of the sentences to keep.
""",
)

工具列表现在有 search_internet、 generate_draft、 write_draft 和 enhance_draft。提示词就是告诉模型该调用哪一个 工具、以及先后顺序的东西。

提示词中两句关键的句子如下: “逐句阅读其余的句子,只有某句带来已保留句子没有的有用新信息时,才 保留它。”这两句可以帮我们减少句子重复。

结果

以下代码块印出两个工具的结果: write_draft 和 enhance_draft。

for message in result["messages"]:
    if message.type == "tool" and message.name in ["write_draft", "enhance_draft"]:
        print(f"Tool: {message.name}\n")
        print(message.content)
    elif message.type == "tool":
        response = json.loads(message.content)
        print(f"Tool: {message.name}")

以下印出被 enhance_draft 跳过的句子,供我们 参考:

print("Sentences skipped by enhance_draft:\n")
if skipped_items == []:
    print("None\n")
for item in skipped_items:
    print(f"[{item['num']} in write_draft] {item['sentence']}")
    print(f"   {item['url']}\n")

输出结果

屏幕现在印出四个工具区块 —— search_internet、 generate_draft、 write_draft 和 enhance_draft —— 然后是被删去的句子列表。 以下是增强版草稿:

Tool: enhance_draft

China, Hong Kong SAR 2026 population is estimated at 7,378,602 people at mid-year.[1]
As of 2026, Hong Kong has a total population of 7,378,602, ranking as the 107th most populous nation in the world.[2]
Total current population for Hong Kong in 2026 is 7,500,962, a 0.02% increase from 2025.[3]
Hong Kong's population was approximately 7.50 million in mid-2025.[4]
Hong Kong is a largely homogenous society, with about 91.6% of its people being Chinese (ethnically speaking, Han Chinese).[5]

Endnotes:
[1] https://www.worldometers.info/world-population/china-hong-kong-sar-population
[2] https://worldpopulationreview.com/countries/hong-kong
[3] https://www.macrotrends.net/global-metrics/countries/hkg/hong-kong/population
[4] https://www.gov.hk/en/about/abouthk/facts.htm
[5] https://www.had.gov.hk/rru/en/info/demographics.htm

七句变成五句,尾注从 1 重新编号,所以中间没有缺号。

接着是让这个选择可以被核对的部分。这次运行删去了这些:

Sentences skipped by enhance_draft:

[3 in write_draft] The population of Hong Kong is 7.5 million.
   https://study.com/academy/lesson/hong-kong-facts-population.html

[5 in write_draft] In 2023, the estimated population of Hong Kong was 7.443 million.
   https://ourworldindata.org/profile/population-demography/hong-kong

两句被丢弃的原因相同:这个数字已经有了。study.com 的页面说 750 万,没有 注明年份;Our World in Data 的页面给出 2023 年的 744.3 万;而草稿已经有 2026 年的 7,500,962 和 2025 年中的 750 万。

结果并不完美,这一点值得说明。第 [1] 句和 第 [2] 句都说 7,378,602,无论怎样衡量都是 重复;第二句之所以留下,是因为它同时加上了第 107 位的排名。如果工具把所有 出现两次的内容都删去,排名也会被一起拿走。

留下的五句仍然互相矛盾:7,378,602、7,500,962 和 750 万,是同一个问题的 三个不同答案。删去重复并不能消除分歧,这个版本也没有这样声称。

我们可以再加一个工具来改进提示词,例如 generate_summary,读取每句中的数值,并说出 例如所找到数字的平均值或中位数。

完整程序

import os
import json
from dotenv import load_dotenv
from tavily import TavilyClient
from langchain_core.tools import tool
from langchain_openai import ChatOpenAI
from langchain.agents import create_agent

load_dotenv()
DEEPSEEK_API_KEY = os.getenv("DEEPSEEK_API_KEY")
TAVILY_API_KEY = os.getenv("TAVILY_API_KEY")
tavily_client = TavilyClient(api_key=TAVILY_API_KEY)

last_search = {}
selected_results = []
draft_items = []
skipped_items = []

@tool
def search_internet(query: str):
    """
    Search the internet for up-to-date information about the query
    and return the full Tavily search response.
    """
    global last_search
    if os.path.exists("tavily_search.json"):
        with open("tavily_search.json", "r", encoding="utf-8") as f:
            response = json.load(f)
    else:
        response = tavily_client.search(query, max_results=10)
        with open("tavily_search.json", "w", encoding="utf-8") as f:
            json.dump(response, f, ensure_ascii=False, indent=2)
    last_search = response
    return response

@tool
def generate_draft():
    """
    Go through the results from the latest search_internet call.
    Keep results with a score of 0.5 or higher, but never keep Wikipedia.
    Return the selected results.
    """
    global selected_results
    selected = []
    for r in last_search.get("results", []):
        if r["score"] >= 0.5 and "wikipedia.org" not in r["url"]:
            selected.append(r)

    selected_results = selected
    return {"query": last_search.get("query"), "results": selected}

@tool
def write_draft(sentences: list[str]):
    """
    Write a draft with endnotes from the results of generate_draft.
    sentences: one sentence per result from generate_draft, in the same order.
    Each sentence must be copied exactly from that result's content, and be
    the sentence closest in meaning to the user's query.
    Use an empty string "" for a result that has no complete sentence.
    """
    global draft_items
    draft_items = []
    s = ""
    endnotes = ""
    endnote_num = 1
    for sentence, r in zip(sentences, selected_results):
        if sentence.strip() == "":
            continue  # skip results with no complete sentence
        s = s + sentence + "[" + str(endnote_num) + "]\n"
        endnotes = endnotes + f"[{endnote_num}] {r['url']}\n"
        draft_items.append({"sentence": sentence, "url": r["url"]})
        endnote_num = endnote_num + 1
    s = s + "\nEndnotes:\n" + endnotes
    return s

@tool(return_direct=True)
def enhance_draft(keep: list[int]):
    """
    Write an enhanced draft from the draft made by write_draft.
    keep: the endnote numbers of the sentences to keep. Sentence 1 is always kept.
    Read sentences 2, 3, ... one by one, and keep a sentence only if it adds
    useful new information that the sentences already kept do not give.
    """
    global skipped_items
    skipped_items = []
    s = ""
    endnotes = ""
    endnote_num = 1
    for i, item in enumerate(draft_items, start=1):
        if i != 1 and i not in keep:
            skipped_items.append({"num": i, "sentence": item["sentence"], "url": item["url"]})
            continue  # skip sentences that add little new information
        s = s + item["sentence"] + "[" + str(endnote_num) + "]\n"
        endnotes = endnotes + f"[{endnote_num}] {item['url']}\n"
        endnote_num = endnote_num + 1
    s = s + "\nEndnotes:\n" + endnotes
    return s

model = ChatOpenAI(
    model="deepseek-flash",
    api_key=DEEPSEEK_API_KEY,
    base_url="https://api.deepseek.com",
    temperature=0,
)

agent = create_agent(
    model=model,
    tools=[search_internet, generate_draft, write_draft, enhance_draft],
    system_prompt="""
You are a helpful assistant.
Always call search_internet, using the user's message exactly as the query.
Do not rephrase, shorten, translate, or add anything to it.
Then call generate_draft to select the useful results.
Then go through each sentence in each selected result, pick the one sentence
closest in meaning to the user's query, and copy it exactly.
A complete sentence has a subject and a verb and ends with a full stop;
table rows, headings, menus and fragments are not complete sentences.
If a result has no complete sentence, use an empty string "" for it.
Call write_draft with these sentences in the same order as the results.
Then read the draft from write_draft. Keep sentence 1.
Read the remaining sentences one by one, and keep a sentence only if it adds
useful new information that the sentences already kept do not give.
Skip a sentence that only repeats a similar figure already given.
Keep a sentence that adds a new fact, such as a ranking, a trend,
a breakdown, or an official government source.
Call enhance_draft with the endnote numbers of the sentences to keep.
""",
)

if os.path.exists("tavily_search.json"):
    os.remove("tavily_search.json")  # start each run with a fresh search

prompt = "What is the population of Hong Kong?"
result = agent.invoke({
    "messages": [{"role": "user", "content": prompt}]
})

for message in result["messages"]:
    if message.type == "tool" and message.name in ["write_draft", "enhance_draft"]:
        print(f"Tool: {message.name}\n")
        print(message.content)
    elif message.type == "tool":
        response = json.loads(message.content)
        print(f"Tool: {message.name}")
        print(f"Query: {response['query']}\n")
        for i, r in enumerate(response["results"], start=1):
            print(f"{i}. {r['title']}  (score: {r['score']:.2f})")
            print(f"   {r['url']}")
            print(f"   {r['content'][:200].replace(chr(10), ' ')}...\n")

print("Sentences skipped by enhance_draft:\n")
if skipped_items == []:
    print("None\n")
for item in skipped_items:
    print(f"[{item['num']} in write_draft] {item['sentence']}")
    print(f"   {item['url']}\n")

在自己的电脑上运行

下载演示程序 (Folder-AI-Research05.zip)。zip 里有三个文件: research05.py(完整脚本)、pyproject.toml 和 README.md。这个版本会调用语言模型,所以需要 Tavily 密钥和 DeepSeek 密钥。

  1. 在 Visual Studio Code 中打开这个文件夹。
  2. 在文件夹里创建一个名为 .env 的文件,内容有两行:
    TAVILY_API_KEY=your-tavily-key
    DEEPSEEK_API_KEY=your-deepseek-key
    Tavily 密钥可以从 Tavily 控制台获取,DeepSeek 密钥则可以从 DeepSeek 控制台获取。用量由各家提供商计费,所以运行之前,请先在那里查看当前的 收费条款。
  3. 在终端里逐行运行以下三行:
    uv python pin 3.12.10
    uv sync
    uv run research05.py
  4. 你应该会依次看到四个工具区块 —— 搜索的十个结果、 generate_draft 的七个结果、来自 write_draft 的七句草稿,然后是 enhance_draft 缩短后的草稿 —— 接着是 被跳过句子的列表。这次运行只需要几秒钟。

如果你还没有配置好 uv,请先看 如何安装 Visual Studio Code?,然后看 uv 笔记的 Windows 或 Mac 版本。

局限

延伸阅读