How to recognise faces in live video (Face recognition, version 7)?

Howard Pong Yuen Lam · AI Notes · November 13, 2026 · 6 min read

This is the version the series has been building towards. The face files in the folder are turned into numbers once, at the start. Then every frame from the camera is searched for faces, each face is compared with the folder, and the closest name is drawn above a green box. A face that belongs to nobody in the folder is labelled Unknown rather than guessed.

Nothing here is new machinery. Version 2 built the cast list. Version 3 compared one photograph against it. Version 6 drew boxes and labels. Version 7 puts all three inside the camera loop, and that is the whole step.

The program has two halves

The split is worth noticing before reading the code. Everything before cv2.VideoCapture(0) runs once: read each file in the folder, encode the face, remember the name. Everything inside the loop runs for every frame. The first half is expensive but happens once, the second half is cheap on its own but repeats as long as the camera is open.

1. Build the cast list once

images_path = glob.glob(os.path.join("images", "*.*"))

known_face_encodings = []
known_face_names = []
for img_path in images_path:
    img = cv2.imread(img_path)
    rgb_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    filename, ext = os.path.splitext(os.path.basename(img_path))
    known_face_encodings.append(face_recognition.face_encodings(rgb_img)[0])
    known_face_names.append(filename)

This is version 2, compressed: the folder listing, the BGR to RGB conversion, the [0] that takes the one face in each file, and two lists kept in the same order. The filename is the name that will appear on screen, so images/Howard.JPG becomes Howard.

2. Find and encode every face in the frame

    rgb_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    face_locations = face_recognition.face_locations(rgb_img)
    face_encodings = face_recognition.face_encodings(rgb_img, face_locations)

Three lines, and the middle and last are version 3's work. face_locations holds one box per face found; face_encodings holds the matching 128-number description of each of those faces.

There is no [0] here, and that is deliberate. In the folder each file holds one face, so the first is the only one. In a camera frame there may be zero, one or three faces, and the program has to handle whatever arrives. When two people stand in front of the camera, both boxes are found and both are processed.

The two lists are returned in the same order, so the first location belongs to the first encoding. The program relies on that alignment later, silently, without ever checking it.

3. Compare each face and keep the closest name

    face_names = []
    for face_encoding in face_encodings:
        matches = face_recognition.compare_faces(known_face_encodings, face_encoding)
        name = "Unknown"
        face_distances = face_recognition.face_distance(known_face_encodings, face_encoding)
        best_match_index = np.argmin(face_distances)
        if matches[best_match_index]:
            name = known_face_names[best_match_index]
        face_names.append(name)

For each face in the frame, compare_faces gives one True or False per known person, and face_distance gives the number behind that decision; smaller means more alike. np.argmin picks the position of the smallest distance, so the most similar person in the folder is found.

Being the most similar is not the same as being the same. A stranger is still closest to somebody, even when the distance is large, which is why line 33 checks matches[best_match_index] before accepting the name. Without that check every face would be given a name, and the folder would seem to recognise the whole world.

The starting value "Unknown" covers the remaining case, and append sits outside the if so that one name is added for every face, matched or not. face_names then has the same length as face_locations, and position 0 in one belongs with position 0 in the other.

4. Pair each box with its name

    for face_loc, name in zip(face_locations, face_names):
        y1, x2, y2, x1 = face_loc[0], face_loc[1], face_loc[2], face_loc[3]
        cv2.putText(img, name, (x1, y1 - 10), cv2.FONT_HERSHEY_DUPLEX, 1, (0, 200, 0), 2)
        cv2.rectangle(img, (x1, y1), (x2, y2), (0, 200, 0), 4)

zip walks the two lists together: first box with first name, second with second. It only works because of the care taken above, which is why the ordering was mentioned twice.

The odd line is the second one. face_recognition describes a face by its top, right, bottom and left edges, in that order, while cv2.rectangle wants a top-left corner and a bottom-right corner. The unpacking shuffles the four numbers into the names the drawing calls expect: y1 is the top, x2 the right, y2 the bottom, x1 the left. Name the variables wrongly here and the box appears as a thin line or a large rectangle across the whole picture.

The colour (0, 200, 0) is green in BGR order: no blue, a little under full green, no red. The 200 instead of 255 is a matter of taste; both look green. The thickness of 4 is heavier than version 6 used, so the box is visible against a busy background.

The whole program

Version 1's download already contains this file, so no new installation is needed:

import face_recognition
import cv2
import os
import glob
import numpy as np
images_folder = "images"
images_path = glob.glob(os.path.join(images_folder, "*.*"))

known_face_encodings=[]
known_face_names=[]
for img_path in images_path:
    img = cv2.imread(img_path)
    rgb_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    basename = os.path.basename(img_path)
    (filename, ext) = os.path.splitext(basename)
    img_encoding = face_recognition.face_encodings(rgb_img)[0]
    known_face_encodings.append(img_encoding)
    known_face_names.append(filename)

cap = cv2.VideoCapture(0)
while True:
    ret, img = cap.read()
    rgb_img = cv2.cvtColor(img,cv2.COLOR_BGR2RGB)
    face_locations = face_recognition.face_locations(rgb_img)
    face_encodings = face_recognition.face_encodings(rgb_img, face_locations)

    face_names = []
    for face_encoding in face_encodings:
        matches = face_recognition.compare_faces(known_face_encodings, face_encoding)
        name = "Unknown"
        face_distances = face_recognition.face_distance(known_face_encodings, face_encoding)
        best_match_index = np.argmin(face_distances)
        if matches[best_match_index]:
            name = known_face_names[best_match_index]
        face_names.append(name)

    for face_loc, name in zip(face_locations, face_names):
        y1, x2, y2, x1 = face_loc[0], face_loc[1], face_loc[2], face_loc[3]
        cv2.putText(img, name, (x1, y1 - 10), cv2.FONT_HERSHEY_DUPLEX, 1, (0, 200, 0), 2)
        cv2.rectangle(img, (x1, y1), (x2, y2), (0, 200, 0), 4)

    cv2.imshow("Frame", img)
    key = cv2.waitKey(1)
    if key in (ord('q'), ord('Q')):
        break

cap.release()
cv2.destroyAllWindows()

What to expect on screen

Put your own photo in the folder first

The camera can only name people it has seen before. To be recognised yourself, drop a photograph of your face into the images folder and name the file after the person in it, for example Peter.jpg. The program reads the name from the file name, so that file is what makes the green box say Peter; a file called IMG_4821.jpg would be recognised just as well and labelled IMG_4821. One clear face per file, facing the camera, is enough.

Without such a file the program still runs, but every face it sees is reported as Unknown. Add the photo, restart the program, and the label changes.

Running it on your own computer

Use the same download as version 1: Download the zip file, unzip it, and open the folder in Visual Studio Code. This note uses face07.py. The first time in this folder, run all three lines:

uv python pin 3.12.10
uv sync
uv run face07.py

Those three lines are needed only once. The next time you want to run the program again, type just uv run face07.py.

What comes next

The weakness is speed. Every frame pays the full cost of finding and encoding faces, which is why the picture stutters. Version 8 deals with that cost, without giving up the recognition.