程式如何把一段文字分成句子(ReAct,版本 8)?
我們代理的版本 9 將會讀取網頁。網頁送來的時候是一大塊一大塊的文字 — 標題、段落、清單項目 — 如果先把文字切成句子,代理的表現會更好。真正 負責切分的只有一行 Python。這篇筆記就用我們自己編的三個文字區塊,逐步拆解 這一行,過程中不涉及任何網頁。
要學的那一行
sentences.extend(re.split(r"(?<=[.!?])\s+", block))
這一行同時做兩件事。re.split(...) 把一個文字區塊切成句子,
sentences.extend(...) 則把這些句子逐一加入一個清單。真正
值得理解的是切分的規則,而最容易的方法,是先試兩個更簡單的規則。以下是
我們要切開的三個區塊:
blocks = [
"Howard Lam, PhD", # a heading: no full stop at all
"I teach AI for marketing. Do you like it? Yes!", # a paragraph: three sentences
"My average meal cost is HK$60.10 in 2025.", # 60.10 has a "." but no space after it
]
三個區塊都是刻意挑選的。標題完全沒有句號。段落裡有三個句子。而價格 HK$60.10 中間的那一點,並不是句子的結尾。
第 1 步:按空格切 — 碎片太多
re.split(r"\s+", blocks[1])
['I', 'teach', 'AI', 'for', 'marketing.', 'Do', 'you', 'like', 'it?', 'Yes!']
\s 代表一個空格(或定位字元,或換行),\s+
代表連續一個或多個。在每個空格處切開,每個字都會變成獨立一片 —
實在太多。我們要的是三個句子,不是十片碎片。
第 2 步:按標點切 — 標點不見了
re.split(r"[.!?]\s+", blocks[1])
['I teach AI for marketing', 'Do you like it', 'Yes!']
[.!?] 代表一個 .、! 或
? 字元,緊隨其後的 \s+ 表示只有在標點後面有空格
時才切開。
'I teach AI for marketing.'變成
'I teach AI for marketing',少了那個
.,因為 . 後面有空格。
'Do you like it?'變成
'Do you like it',因為 ? 後面
有空格。
'Yes!'變成 'Yes!',
因為 ! 後面沒有空格可以切。
第 3 步:真正的那一行 — 保留標點
re.split(r"(?<=[.!?])\s+", blocks[1])
['I teach AI for marketing.', 'Do you like it?', 'Yes!']
(?<=[.!?]) 稱為後顧斷言:只在緊接 .、
! 或 ? 之後的空格處切開。它會查看標點,但不會
把標點切走,所以每個句子都保留自己的結尾。三個句子,標點完整。
第 4 步:逐個區塊處理 — 為甚麼 HK$60.10 不會被切開
['Howard Lam, PhD']
['I teach AI for marketing.', 'Do you like it?', 'Yes!']
['My average meal cost is HK$60.10 in 2025.']
標題原封不動地回來 — 裡面沒有 .、! 或
?,所以沒有可以切開的位置。HK$60.10 也沒有被切開:規則在
「一點加一個空格」處切,而 60.10 中的那一點後面沒有空格。價格就留在原來
的句子裡面。
第 5 步:把所有句子收集到一個編號清單
sentences = []
for block in blocks:
sentences.extend(re.split(r"(?<=[.!?])\s+", block))
1. Howard Lam, PhD
2. I teach AI for marketing.
3. Do you like it?
4. Yes!
5. My average meal cost is HK$60.10 in 2025.
這就是 extend 的用途:每個區塊的每個句子,都成為同一個清單中
的一個項目。版本 9 會用完全相同的方式為網頁的句子編號,這樣代理就可以用
編號指涉第 3 句或第 17 句。
完整程式
# Learn ONE line from ReAct09.py, step by step:
# sentences.extend(re.split(r"(?<=[.!?])\s+", block))
# It cuts a block of text into sentences.
# Here we make our own blocks (no web page), so we can see exactly what happens.
import re
# A few blocks, like the headings, paragraphs and list items ReAct09.py takes from a web page
blocks = [
"Howard Lam, PhD", # a heading: no full stop at all
"I teach AI for marketing. Do you like it? Yes!", # a paragraph: three sentences
"My average meal cost is HK$60.10 in 2025.", # 60.10 has a "." but no space after it
]
# ---- Step 1: split on spaces only -> cuts after EVERY word (too many pieces) ----
print("=== Step 1: re.split(r'\\s+', ...) ===")
print(re.split(r"\s+", blocks[1]))
# \s means one space (or tab, or line break); \s+ means one or more of them in a row
# ---- Step 2: split on ". ! ?" plus spaces -> the punctuation is LOST ----
print("\n=== Step 2: re.split(r'[.!?]\\s+', ...) ===")
print(re.split(r"[.!?]\s+", blocks[1]))
# [.!?] means ONE character that is . or ! or ?
# The cut happens ON the punctuation, so it disappears
# ---- Step 3: the real line -> keep the punctuation ----
print("\n=== Step 3: re.split(r'(?<=[.!?])\\s+', ...) ===")
print(re.split(r"(?<=[.!?])\s+", blocks[1]))
# (?<=[.!?]) is a "look-behind": cut only at spaces that come right AFTER . ! or ?
# It only LOOKS at the punctuation and does not cut it out, so the sentence keeps its "."
# ---- Step 4: every block -> why there is no cut inside HK$60.10 ----
print("\n=== Step 4: each block on its own ===")
for block in blocks:
print(re.split(r"(?<=[.!?])\s+", block))
# "Howard Lam, PhD" has no . ! ? so we get the whole block back as ONE sentence
# "HK$60.10" has a "." but no space after it, so it is not cut: the block stays ONE sentence
# ---- Step 5: put the sentences of ALL blocks into one list, and number them (as ReAct09.py does) ----
sentences = []
for block in blocks:
sentences.extend(re.split(r"(?<=[.!?])\s+", block)) # extend adds each sentence as its own item
print(f"\n=== Step 5: {len(sentences)} sentences ===")
for number, sentence in enumerate(sentences, start=1):
print(f"{number}. {sentence}")
# Try it: change a block to "Dr. Lam teaches AI." and run again.
# Why does it cut after "Dr."? (Hint: the rule only sees ". " — it does not know "Dr." is not a sentence end.)
在自己的電腦上執行
下載示範程式
(Folder-AI-ReAct08.zip)。zip 內有三個檔案:
ReAct08.py(完整程式)、pyproject.toml 和
README.md。毋須安裝任何東西 — re 是 Python
內置的 — 也不需要 API 金鑰。
- 在 Visual Studio Code 開啟這個資料夾。
- 在終端機逐行執行以下三條指令:
uv python pin 3.12.10uv syncuv run ReAct08.py - 你應該會看到五個步驟,最後是編號清單:
=== Step 1: re.split(r'\s+', ...) === ['I', 'teach', 'AI', 'for', 'marketing.', 'Do', 'you', 'like', 'it?', 'Yes!'] === Step 2: re.split(r'[.!?]\s+', ...) === ['I teach AI for marketing', 'Do you like it', 'Yes!'] === Step 3: re.split(r'(?<=[.!?])\s+', ...) === ['I teach AI for marketing.', 'Do you like it?', 'Yes!'] === Step 4: each block on its own === ['Howard Lam, PhD'] ['I teach AI for marketing.', 'Do you like it?', 'Yes!'] ['My average meal cost is HK$60.10 in 2025.'] === Step 5: 5 sentences === 1. Howard Lam, PhD 2. I teach AI for marketing. 3. Do you like it? 4. Yes! 5. My average meal cost is HK$60.10 in 2025.
如果你還未設定好 uv,請先看如何安裝 Visual Studio Code?, 然後看 Windows 或 Mac 的 uv 筆記。
限制
-
規則只看到一點和一個空格,僅此而已。把某個區塊改成
"Dr. Lam teaches AI."再執行一次:它會在"Dr."之後切開,因為規則無法知道"Dr."是個稱謂,不是句子的結尾。 - 它只適用於用空格的語言。像中文這類語言,標點後面沒有空格, 所以這條規則沒有可以切開的位置,需要另一套規則。
延伸閱讀
- Python re 模組文件 —
split、像\s這類特殊序列,以及後顧斷言。 - regex101 — 貼上規則和區塊,看它在哪裡切開。
- 如何在 AI 代理中使用 LangChain(ReAct,版本 7)? — 本系列的上一篇筆記。