English 繁體中文 简体中文

程式如何把一段文字分成句子(ReAct,版本 8)?

林邦源博士 · AI 筆記 · 2026 年 11 月 9 日 · 約 3 分鐘閱讀

我們代理的版本 9 將會讀取網頁。網頁送來的時候是一大塊一大塊的文字 — 標題、段落、清單項目 — 如果先把文字切成句子,代理的表現會更好。真正 負責切分的只有一行 Python。這篇筆記就用我們自己編的三個文字區塊,逐步拆解 這一行,過程中不涉及任何網頁。

要學的那一行

sentences.extend(re.split(r"(?<=[.!?])\s+", block))

這一行同時做兩件事。re.split(...) 把一個文字區塊切成句子, sentences.extend(...) 則把這些句子逐一加入一個清單。真正 值得理解的是切分的規則,而最容易的方法,是先試兩個更簡單的規則。以下是 我們要切開的三個區塊:

blocks = [
    "Howard Lam, PhD",                                    # a heading: no full stop at all
    "I teach AI for marketing. Do you like it? Yes!",     # a paragraph: three sentences
    "My average meal cost is HK$60.10 in 2025.",          # 60.10 has a "." but no space after it
]

三個區塊都是刻意挑選的。標題完全沒有句號。段落裡有三個句子。而價格 HK$60.10 中間的那一點,並不是句子的結尾。

第 1 步:按空格切 — 碎片太多

re.split(r"\s+", blocks[1])
['I', 'teach', 'AI', 'for', 'marketing.', 'Do', 'you', 'like', 'it?', 'Yes!']

\s 代表一個空格(或定位字元,或換行),\s+ 代表連續一個或多個。在每個空格處切開,每個字都會變成獨立一片 — 實在太多。我們要的是三個句子,不是十片碎片。

第 2 步:按標點切 — 標點不見了

re.split(r"[.!?]\s+", blocks[1])
['I teach AI for marketing', 'Do you like it', 'Yes!']

[.!?] 代表一個 .、! 或 ? 字元,緊隨其後的 \s+ 表示只有在標點後面有空格 時才切開。
'I teach AI for marketing.'變成 'I teach AI for marketing',少了那個 .,因為 . 後面有空格。
'Do you like it?'變成 'Do you like it',因為 ? 後面 有空格。
'Yes!'變成 'Yes!', 因為 ! 後面沒有空格可以切。

第 3 步:真正的那一行 — 保留標點

re.split(r"(?<=[.!?])\s+", blocks[1])
['I teach AI for marketing.', 'Do you like it?', 'Yes!']

(?<=[.!?]) 稱為後顧斷言:只在緊接 .、 ! 或 ? 之後的空格處切開。它會查看標點,但不會 把標點切走,所以每個句子都保留自己的結尾。三個句子,標點完整。

第 4 步:逐個區塊處理 — 為甚麼 HK$60.10 不會被切開

['Howard Lam, PhD']
['I teach AI for marketing.', 'Do you like it?', 'Yes!']
['My average meal cost is HK$60.10 in 2025.']

標題原封不動地回來 — 裡面沒有 .、! 或 ?,所以沒有可以切開的位置。HK$60.10 也沒有被切開:規則在 「一點加一個空格」處切,而 60.10 中的那一點後面沒有空格。價格就留在原來 的句子裡面。

第 5 步:把所有句子收集到一個編號清單

sentences = []
for block in blocks:
    sentences.extend(re.split(r"(?<=[.!?])\s+", block))
1. Howard Lam, PhD
2. I teach AI for marketing.
3. Do you like it?
4. Yes!
5. My average meal cost is HK$60.10 in 2025.

這就是 extend 的用途:每個區塊的每個句子,都成為同一個清單中 的一個項目。版本 9 會用完全相同的方式為網頁的句子編號,這樣代理就可以用 編號指涉第 3 句或第 17 句。

完整程式

# Learn ONE line from ReAct09.py, step by step:
#     sentences.extend(re.split(r"(?<=[.!?])\s+", block))
# It cuts a block of text into sentences.
# Here we make our own blocks (no web page), so we can see exactly what happens.
import re

# A few blocks, like the headings, paragraphs and list items ReAct09.py takes from a web page
blocks = [
    "Howard Lam, PhD",                                    # a heading: no full stop at all
    "I teach AI for marketing. Do you like it? Yes!",     # a paragraph: three sentences
    "My average meal cost is HK$60.10 in 2025.",          # 60.10 has a "." but no space after it
]

# ---- Step 1: split on spaces only -> cuts after EVERY word (too many pieces) ----
print("=== Step 1: re.split(r'\\s+', ...) ===")
print(re.split(r"\s+", blocks[1]))
# \s means one space (or tab, or line break); \s+ means one or more of them in a row

# ---- Step 2: split on ". ! ?" plus spaces -> the punctuation is LOST ----
print("\n=== Step 2: re.split(r'[.!?]\\s+', ...) ===")
print(re.split(r"[.!?]\s+", blocks[1]))
# [.!?] means ONE character that is . or ! or ?
# The cut happens ON the punctuation, so it disappears

# ---- Step 3: the real line -> keep the punctuation ----
print("\n=== Step 3: re.split(r'(?<=[.!?])\\s+', ...) ===")
print(re.split(r"(?<=[.!?])\s+", blocks[1]))
# (?<=[.!?]) is a "look-behind": cut only at spaces that come right AFTER . ! or ?
# It only LOOKS at the punctuation and does not cut it out, so the sentence keeps its "."

# ---- Step 4: every block -> why there is no cut inside HK$60.10 ----
print("\n=== Step 4: each block on its own ===")
for block in blocks:
    print(re.split(r"(?<=[.!?])\s+", block))
# "Howard Lam, PhD" has no . ! ? so we get the whole block back as ONE sentence
# "HK$60.10" has a "." but no space after it, so it is not cut: the block stays ONE sentence

# ---- Step 5: put the sentences of ALL blocks into one list, and number them (as ReAct09.py does) ----
sentences = []
for block in blocks:
    sentences.extend(re.split(r"(?<=[.!?])\s+", block))   # extend adds each sentence as its own item
print(f"\n=== Step 5: {len(sentences)} sentences ===")
for number, sentence in enumerate(sentences, start=1):
    print(f"{number}. {sentence}")

# Try it: change a block to "Dr. Lam teaches AI." and run again.
# Why does it cut after "Dr."? (Hint: the rule only sees ". " — it does not know "Dr." is not a sentence end.)

在自己的電腦上執行

下載示範程式 (Folder-AI-ReAct08.zip)。zip 內有三個檔案: ReAct08.py(完整程式)、pyproject.toml 和 README.md。毋須安裝任何東西 — re 是 Python 內置的 — 也不需要 API 金鑰。

  1. 在 Visual Studio Code 開啟這個資料夾。
  2. 在終端機逐行執行以下三條指令:
    uv python pin 3.12.10
    uv sync
    uv run ReAct08.py
  3. 你應該會看到五個步驟,最後是編號清單:
    === Step 1: re.split(r'\s+', ...) ===
    ['I', 'teach', 'AI', 'for', 'marketing.', 'Do', 'you', 'like', 'it?', 'Yes!']
    
    === Step 2: re.split(r'[.!?]\s+', ...) ===
    ['I teach AI for marketing', 'Do you like it', 'Yes!']
    
    === Step 3: re.split(r'(?<=[.!?])\s+', ...) ===
    ['I teach AI for marketing.', 'Do you like it?', 'Yes!']
    
    === Step 4: each block on its own ===
    ['Howard Lam, PhD']
    ['I teach AI for marketing.', 'Do you like it?', 'Yes!']
    ['My average meal cost is HK$60.10 in 2025.']
    
    === Step 5: 5 sentences ===
    1. Howard Lam, PhD
    2. I teach AI for marketing.
    3. Do you like it?
    4. Yes!
    5. My average meal cost is HK$60.10 in 2025.

如果你還未設定好 uv,請先看如何安裝 Visual Studio Code?, 然後看 Windows 或 Mac 的 uv 筆記。

限制

延伸閱讀