How strict should face matching be (Face recognition, version 9)?

Howard Pong Yuen Lam · AI Notes · November 13, 2026 · 6 min read

Version 8 handles speed. The last version handles a quieter problem: version 8 decides whether two faces are the same using a number nobody chose. The library's default is 0.6, which is a reasonable guess but not a decision. Version 9 makes it a decision, by naming the number and moving it into one place at the top of the file.

Two lines change from version 8. That is all. The change is small because the dial was always there; it was just hidden inside the library's default.

One number, two kinds of mistake

Matching a face produces a distance: how far that face sits from a known face, measured across 128 numbers. Zero would be identical. The tolerance is the cut-off — any distance above it is treated as a different person.

Turning the dial moves two mistakes in opposite directions, and no setting makes both vanish:

Which mistake matters more is a question about the use, not about the code. A door that must not open for the wrong person wants a strict setting. A photo library that should find every picture of a friend can afford to be loose.

1. Set the tolerance once

images_folder = "images"
my_tolerance = 0.5  # library default is 0.6

The variable sits with the other settings, above the folder search, so it is easy to find and easy to change. The text after # is a comment: Python ignores it, and it stays in the file to tell the next reader what the number would be without the line. Keeping the default visible like that is cheaper than remembering it.

2. Pass the number to compare_faces

        matches = face_recognition.compare_faces(known_face_encodings, face_encoding, tolerance=my_tolerance)

Everything else in that line is version 7's work. The addition is tolerance=my_tolerance: the name on the left of the equals sign is the setting inside compare_faces and must be spelled exactly that way; the name on the right is the variable set above.

Notice that face_distance on the next line is left alone. It has no tolerance, because it only reports distances; the decision about how large a distance is acceptable belongs to compare_faces. The two lines therefore stay consistent: the distances are measured once, and the same cut-off is applied to them.

What the distances look like

The encoding vectors have a length of about 1, so distances rarely exceed 1.2 in practice. Typical values:

Useful settings therefore sit between roughly 0.4 and 0.7:

The library's advice matches the table: if two different people are being reported as the same person, lower the number.

Tolerance and the shrink work against each other

Version 8 halves the frame before encoding, which costs fine detail and pushes distances up a little. Version 9 then tightens the cut-off. Do both and the effect compounds: a face that used to pass at 0.57 against a 0.6 default may now sit at 0.62 and fail against 0.5.

So the two settings should be chosen together. If you tighten the tolerance and the right person keeps showing Unknown, raise scale_ratio towards 1.0 before loosening the tolerance again. Recovering accuracy by keeping more pixels is usually better than recovering it by accepting weaker evidence.

The whole program

Version 1's download already contains this file. Compared with version 8 there is one new line, and one line gains a setting; every later line number shifts by one.

import face_recognition
import cv2
import os
import glob
import numpy as np
images_folder = "images"
my_tolerance = 0.5 # library default = 0.6
images_path = glob.glob(os.path.join(images_folder, "*.*"))

known_face_encodings=[]
known_face_names=[]
for img_path in images_path:
    img = cv2.imread(img_path)
    rgb_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    basename = os.path.basename(img_path)
    (filename, ext) = os.path.splitext(basename)
    img_encoding = face_recognition.face_encodings(rgb_img)[0]
    known_face_encodings.append(img_encoding)
    known_face_names.append(filename)

cap = cv2.VideoCapture(0)
scale_ratio = 0.5
multiple = int(1 / scale_ratio)
ret, img = cap.read()
width=int(img.shape[1]*scale_ratio)
height=int(img.shape[0]*scale_ratio)
dim=(width,height)

while True:
    ret, img = cap.read()
    resized=cv2.resize(img,dim,interpolation=cv2.INTER_NEAREST)
    rgb_img = cv2.cvtColor(resized,cv2.COLOR_BGR2RGB)
    face_locations = face_recognition.face_locations(rgb_img)
    face_encodings = face_recognition.face_encodings(rgb_img, face_locations)

    face_names = []
    for face_encoding in face_encodings:
        matches = face_recognition.compare_faces(known_face_encodings, face_encoding, tolerance=my_tolerance)
        name = "Unknown"
        face_distances = face_recognition.face_distance(known_face_encodings, face_encoding)
        best_match_index = np.argmin(face_distances)
        if matches[best_match_index]:
            name = known_face_names[best_match_index]
        face_names.append(name)

    for face_loc, name in zip(face_locations, face_names):
        y1, x2, y2, x1 = face_loc[0], face_loc[1], face_loc[2], face_loc[3]
        y1=y1*multiple
        x2=x2*multiple
        y2=y2*multiple
        x1=x1*multiple
        cv2.putText(img, name, (x1, y1 - 10), cv2.FONT_HERSHEY_DUPLEX, 1, (0, 200, 0), 2)
        cv2.rectangle(img, (x1, y1), (x2, y2), (0, 200, 0), 4)

    cv2.imshow("Frame", img)
    key = cv2.waitKey(1)
    if key in (ord('q'), ord('Q')):
        break

cap.release()
cv2.destroyAllWindows()

Add your own photo first

As before, a face is only named if a file with that name is in the images folder, for example Peter.jpg. Without it every face is Unknown, whatever the tolerance.

Running it on your own computer

Use the same download as version 1: Download the zip file, unzip it, and open the folder in Visual Studio Code. This note uses face09.py. The first time in this folder, run all three lines:

uv python pin 3.12.10
uv sync
uv run face09.py

Those three lines are needed only once. The next time you want to run the program again, type just uv run face09.py.

The series in one view

Nine small programs, each one a step that can be explained in a sentence. Put together, they are a working camera that names the people in front of it, and the same pattern — make a number list, compare it, decide, draw the decision — sits underneath far larger systems.