How to make live face recognition run faster (Face recognition, version 8)?
Version 7 works, and it stutters. Every frame pays the cost of finding and encoding faces at full camera resolution, and that cost is what makes the picture lag behind your movement. Version 8 does the same recognition on a copy shrunk to half width and half height, then draws the boxes back on the full-size picture. The output looks the same. The waiting is shorter.
Half the width is a quarter of the work
A frame 1280 pixels wide and 720 tall holds about 920,000 pixels. Halve both numbers and it holds 640 by 360, which is about 230,000 — one quarter, not one half. Detection scans the picture for patterns, so its cost follows the number of pixels, not the width. This is why a change that sounds modest buys most of the speed.
1. Find out how big the frame is
cap = cv2.VideoCapture(0)
scale_ratio = 0.5
multiple = int(1 / scale_ratio)
ret, img = cap.read()
width = int(img.shape[1] * scale_ratio)
height = int(img.shape[0] * scale_ratio)
dim = (width, height)
The camera has to be read once before the loop, for a single reason: the smaller size cannot be worked out until the frame size is known. That one frame is used and then replaced by the first read inside the loop, so nothing is lost.
img.shape reports height first and width second, so shape[1] is the
width and shape[0] the height. Multiplying by 0.5 gives the smaller
numbers, and int discards the fraction, because a size must be a whole number of
pixels. dim holds the pair ready for cv2.resize.
multiple is the number the boxes will be multiplied by later:
1 / 0.5 is 2. Both ratios are worked out from one variable, so changing
scale_ratio in one place keeps the two in step.
2. Shrink the frame, then search it
ret, img = cap.read()
resized = cv2.resize(img, dim, interpolation=cv2.INTER_NEAREST)
rgb_img = cv2.cvtColor(resized, cv2.COLOR_BGR2RGB)
face_locations = face_recognition.face_locations(rgb_img)
face_encodings = face_recognition.face_encodings(rgb_img, face_locations)
The full-size img stays untouched and is kept for the window. Every recognition
call from version 7 now receives resized instead, so both the search and the
encoding work on a quarter of the pixels.
interpolation=cv2.INTER_NEAREST means each new pixel simply copies the nearest old
one. It is the fastest of the resize methods and the least pretty, which does not matter here:
the small frame is never shown to anyone, only searched. The window still displays the
original.
3. Multiply the boxes back up
for face_loc, name in zip(face_locations, face_names):
y1, x2, y2, x1 = face_loc[0], face_loc[1], face_loc[2], face_loc[3]
y1 = y1 * multiple
x2 = x2 * multiple
y2 = y2 * multiple
x1 = x1 * multiple
This is the step that goes wrong. The faces were found in the small frame, so a box with its top at 100 there belongs at 200 on the full picture. Every one of the four sides has to be doubled before drawing.
Forget it and nothing crashes, which is why it is easy to miss. The box simply appears in the wrong place at the wrong size — usually pressed into the top-left corner, half the expected dimensions. A box that is correct in shape but in the wrong position almost always means this multiplication was skipped or applied to only some of the four numbers.
4. Draw and show at full size
cv2.putText(img, name, (x1, y1 - 10), cv2.FONT_HERSHEY_DUPLEX, 1, (0, 200, 0), 2)
cv2.rectangle(img, (x1, y1), (x2, y2), (0, 200, 0), 4)
cv2.imshow("Frame", img)
The drawing calls are unchanged from version 7, and they still receive img. The
only difference is that the numbers inside x1, y1, x2, y2 have already been
restored to the full-size scale.
The whole program
Version 1's download already contains this file, so no new installation is needed:
import face_recognition
import cv2
import os
import glob
import numpy as np
images_folder = "images"
images_path = glob.glob(os.path.join(images_folder, "*.*"))
known_face_encodings=[]
known_face_names=[]
for img_path in images_path:
img = cv2.imread(img_path)
rgb_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
basename = os.path.basename(img_path)
(filename, ext) = os.path.splitext(basename)
img_encoding = face_recognition.face_encodings(rgb_img)[0]
known_face_encodings.append(img_encoding)
known_face_names.append(filename)
cap = cv2.VideoCapture(0)
scale_ratio = 0.5
multiple = int(1 / scale_ratio)
ret, img = cap.read()
width=int(img.shape[1]*scale_ratio)
height=int(img.shape[0]*scale_ratio)
dim=(width,height)
while True:
ret, img = cap.read()
resized=cv2.resize(img,dim,interpolation=cv2.INTER_NEAREST)
rgb_img = cv2.cvtColor(resized,cv2.COLOR_BGR2RGB)
face_locations = face_recognition.face_locations(rgb_img)
face_encodings = face_recognition.face_encodings(rgb_img, face_locations)
face_names = []
for face_encoding in face_encodings:
matches = face_recognition.compare_faces(known_face_encodings, face_encoding)
name = "Unknown"
face_distances = face_recognition.face_distance(known_face_encodings, face_encoding)
best_match_index = np.argmin(face_distances)
if matches[best_match_index]:
name = known_face_names[best_match_index]
face_names.append(name)
for face_loc, name in zip(face_locations, face_names):
y1, x2, y2, x1 = face_loc[0], face_loc[1], face_loc[2], face_loc[3]
y1=y1*multiple
x2=x2*multiple
y2=y2*multiple
x1=x1*multiple
cv2.putText(img, name, (x1, y1 - 10), cv2.FONT_HERSHEY_DUPLEX, 1, (0, 200, 0), 2)
cv2.rectangle(img, (x1, y1), (x2, y2), (0, 200, 0), 4)
cv2.imshow("Frame", img)
key = cv2.waitKey(1)
if key in (ord('q'), ord('Q')):
break
cap.release()
cv2.destroyAllWindows()
Choosing the ratio
0.5 is a sensible starting point on a laptop. Lower means faster and blinder:
1.0— the version 7 behaviour, no shrinking, best for small or distant faces.0.5— a quarter of the pixels, roughly three to four times smoother on a typical webcam, still finds a face that fills a reasonable part of the frame.0.25— a sixteenth of the pixels, very smooth, but a face has to be close to the camera to survive the shrink.
One requirement: 1 / scale_ratio must come out as a whole number, because
int truncates. 0.5 gives 2 and 0.25 gives 4, both fine.
A ratio such as 0.7 gives 1.42, truncated to 1, and then the boxes are never
scaled up — the same silent misplacement as above. If you want three-quarters, work out
multiple yourself, for example multiple = 1.4286, and keep it a
decimal rather than wrapping it in int.
What to expect
- The window is noticeably smoother than in version 7, and the picture is still full quality, because only the copy was shrunk.
- Boxes and names stay in the right place; if they do not, the multiplication is the place to look.
- A face far from the camera can now go unrecognised, or come back
Unknownmore often than in version 7. The encoding is made from fewer pixels, so fine detail is gone. Move closer, or raise the ratio. - Brightness and colour are untouched;
INTER_NEARESTonly changes how the smaller pixels are sampled.
Add your own photo first
As in version 7, a face is only named if a file with that name sits in the
images folder, for example Peter.jpg for Peter. Without it, every
face is reported as Unknown whatever the ratio.
Running it on your own computer
Use the same download as version 1:
Download the zip file, unzip it,
and open the folder in Visual Studio Code. This note uses face08.py. The first
time in this folder, run all three lines:
uv python pin 3.12.10
uv sync
uv run face08.py
Those three lines are needed only once. The next time you want to run the program again,
type just uv run face08.py.
What comes next
Speed is no longer the problem, but the label still flickers: a name for a few frames, then
Unknown, then the name again. Version 9, the last in the series, deals with that
flicker.