How to recognise faces in live video (Face recognition, version 7)?
This is the version the series has been building towards. The face files in the folder are
turned into numbers once, at the start. Then every frame from the camera is searched for
faces, each face is compared with the folder, and the closest name is drawn above a green box.
A face that belongs to nobody in the folder is labelled Unknown rather than
guessed.
Nothing here is new machinery. Version 2 built the cast list. Version 3 compared one photograph against it. Version 6 drew boxes and labels. Version 7 puts all three inside the camera loop, and that is the whole step.
The program has two halves
The split is worth noticing before reading the code. Everything before
cv2.VideoCapture(0) runs once: read each file in the folder,
encode the face, remember the name. Everything inside the loop runs for every
frame. The first half is expensive but happens once, the second half is cheap on its
own but repeats as long as the camera is open.
1. Build the cast list once
images_path = glob.glob(os.path.join("images", "*.*"))
known_face_encodings = []
known_face_names = []
for img_path in images_path:
img = cv2.imread(img_path)
rgb_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
filename, ext = os.path.splitext(os.path.basename(img_path))
known_face_encodings.append(face_recognition.face_encodings(rgb_img)[0])
known_face_names.append(filename)
This is version 2, compressed: the folder listing, the BGR to RGB conversion, the
[0] that takes the one face in each file, and two lists kept in the same order.
The filename is the name that will appear on screen, so images/Howard.JPG
becomes Howard.
2. Find and encode every face in the frame
rgb_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
face_locations = face_recognition.face_locations(rgb_img)
face_encodings = face_recognition.face_encodings(rgb_img, face_locations)
Three lines, and the middle and last are version 3's work. face_locations holds
one box per face found; face_encodings holds the matching 128-number description
of each of those faces.
There is no [0] here, and that is deliberate. In the folder each file holds one
face, so the first is the only one. In a camera frame there may be zero, one or three faces,
and the program has to handle whatever arrives. When two people stand in front of the camera,
both boxes are found and both are processed.
The two lists are returned in the same order, so the first location belongs to the first encoding. The program relies on that alignment later, silently, without ever checking it.
3. Compare each face and keep the closest name
face_names = []
for face_encoding in face_encodings:
matches = face_recognition.compare_faces(known_face_encodings, face_encoding)
name = "Unknown"
face_distances = face_recognition.face_distance(known_face_encodings, face_encoding)
best_match_index = np.argmin(face_distances)
if matches[best_match_index]:
name = known_face_names[best_match_index]
face_names.append(name)
For each face in the frame, compare_faces gives one True or False per known
person, and face_distance gives the number behind that decision; smaller means
more alike. np.argmin picks the position of the smallest distance, so the most
similar person in the folder is found.
Being the most similar is not the same as being the same. A stranger is still closest to
somebody, even when the distance is large, which is why line 33 checks
matches[best_match_index] before accepting the name. Without that check every
face would be given a name, and the folder would seem to recognise the whole world.
The starting value "Unknown" covers the remaining case, and
append sits outside the if so that one name is added for every face,
matched or not. face_names then has the same length as
face_locations, and position 0 in one belongs with position 0 in the other.
4. Pair each box with its name
for face_loc, name in zip(face_locations, face_names):
y1, x2, y2, x1 = face_loc[0], face_loc[1], face_loc[2], face_loc[3]
cv2.putText(img, name, (x1, y1 - 10), cv2.FONT_HERSHEY_DUPLEX, 1, (0, 200, 0), 2)
cv2.rectangle(img, (x1, y1), (x2, y2), (0, 200, 0), 4)
zip walks the two lists together: first box with first name, second with second.
It only works because of the care taken above, which is why the ordering was mentioned twice.
The odd line is the second one. face_recognition describes a face by its
top, right, bottom and left edges, in that order, while
cv2.rectangle wants a top-left corner and a bottom-right corner. The unpacking
shuffles the four numbers into the names the drawing calls expect: y1 is the top,
x2 the right, y2 the bottom, x1 the left. Name the
variables wrongly here and the box appears as a thin line or a large rectangle across the
whole picture.
The colour (0, 200, 0) is green in BGR order: no blue, a little under full green,
no red. The 200 instead of 255 is a matter of taste; both look green. The thickness of 4 is
heavier than version 6 used, so the box is visible against a busy background.
The whole program
Version 1's download already contains this file, so no new installation is needed:
import face_recognition
import cv2
import os
import glob
import numpy as np
images_folder = "images"
images_path = glob.glob(os.path.join(images_folder, "*.*"))
known_face_encodings=[]
known_face_names=[]
for img_path in images_path:
img = cv2.imread(img_path)
rgb_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
basename = os.path.basename(img_path)
(filename, ext) = os.path.splitext(basename)
img_encoding = face_recognition.face_encodings(rgb_img)[0]
known_face_encodings.append(img_encoding)
known_face_names.append(filename)
cap = cv2.VideoCapture(0)
while True:
ret, img = cap.read()
rgb_img = cv2.cvtColor(img,cv2.COLOR_BGR2RGB)
face_locations = face_recognition.face_locations(rgb_img)
face_encodings = face_recognition.face_encodings(rgb_img, face_locations)
face_names = []
for face_encoding in face_encodings:
matches = face_recognition.compare_faces(known_face_encodings, face_encoding)
name = "Unknown"
face_distances = face_recognition.face_distance(known_face_encodings, face_encoding)
best_match_index = np.argmin(face_distances)
if matches[best_match_index]:
name = known_face_names[best_match_index]
face_names.append(name)
for face_loc, name in zip(face_locations, face_names):
y1, x2, y2, x1 = face_loc[0], face_loc[1], face_loc[2], face_loc[3]
cv2.putText(img, name, (x1, y1 - 10), cv2.FONT_HERSHEY_DUPLEX, 1, (0, 200, 0), 2)
cv2.rectangle(img, (x1, y1), (x2, y2), (0, 200, 0), 4)
cv2.imshow("Frame", img)
key = cv2.waitKey(1)
if key in (ord('q'), ord('Q')):
break
cap.release()
cv2.destroyAllWindows()
What to expect on screen
- Your own face gets your name if a clear photo of you is in
images, andUnknownif it is not. - The picture is slower than in version 4. Finding and encoding faces happens on every frame, and that work is not free, so the window may lag behind your movements.
- Two people can be recognised at the same time, but only if both faces are large enough and reasonably lit.
- Turn away, cover half your face, or stand against a window, and the label can change to
Unknownfor a moment. Recognition reads the picture, not the person. - A photograph in the folder holding two faces will take the first one and quietly ignore
the rest, because of the
[0]in the cast list.
Put your own photo in the folder first
The camera can only name people it has seen before. To be recognised yourself, drop a
photograph of your face into the images folder and name the file after the person
in it, for example Peter.jpg. The program reads the name from the file name, so
that file is what makes the green box say Peter; a file called
IMG_4821.jpg would be recognised just as well and labelled
IMG_4821. One clear face per file, facing the camera, is enough.
Without such a file the program still runs, but every face it sees is reported as
Unknown. Add the photo, restart the program, and the label changes.
Running it on your own computer
Use the same download as version 1:
Download the zip file, unzip it,
and open the folder in Visual Studio Code. This note uses face07.py. The first
time in this folder, run all three lines:
uv python pin 3.12.10
uv sync
uv run face07.py
Those three lines are needed only once. The next time you want to run the program again,
type just uv run face07.py.
What comes next
The weakness is speed. Every frame pays the full cost of finding and encoding faces, which is why the picture stutters. Version 8 deals with that cost, without giving up the recognition.