How to use AI to put lipstick on your face in real time?
This note joins the last two together. From How to get live video from your webcam? it takes the camera loop; from How to use AI to put lipstick on your face? it takes the lipstick. The result is 35 lines that put a moving face on screen with cherry red lips.
The only new idea is that the lips are found again in every frame. Find, paint, show; then the next frame arrives and the same three steps repeat. Because each painting is a fraction of a second, your face looks as if it is wearing lipstick continuously.
Why this program is longer than the one for a photo
The lipstick note ran 14 lines. This one runs 35. The lipstick logic itself is unchanged: the same 12 lip points, the same cherry red, the same filling of the lips. What grows is the cost of working live instead of on a file.
- There is a loop. A photo is opened once. A camera keeps delivering frames, so the program needs a loop, a way to show each frame, a way to stop, and a way to release the camera.
- Speed matters. A photo can take a whole second; a frame cannot, or the next one is already late and the video stutters. Hence the quarter-size copy and the points scaled back up.
-
There is no file to draw on. The photo note used
ImageDraw, which fills a shape with a see-through colour in one line. A live frame is a grid of numbers in memory, and OpenCV has no matching one-line fill, so the colour is applied by building a mask and mixing it in. - Two colour orders. OpenCV keeps colours blue-first; the AI library wants red-first.
So the extra length comes from keeping up with a moving face.
How the program works
1. Load three libraries
import cv2
import face_recognition
import numpy as np
cv2 is OpenCV: it reads the camera, resizes each frame, changes
its colour order, and shows it. face_recognition finds the face and its
landmarks, the points that mark the lips. np is
NumPy (Numerical Python), which does arithmetic on a whole grid of pixels in
one line — necessary here, because mixing colour pixel by pixel, 35 times a second, would
otherwise be too slow.
2. Choose the colour and three settings
color_lip = (40, 20, 190)
strength = 120 / 255
small = 0.25
color_lip is cherry red, from the lipstick note. There the red was written as
(190, 20, 40) for red, green, blue. OpenCV stores colours in the opposite order,
blue then green then red, so the same colour is written (40, 20, 190) here.
strength is how much lipstick to show: 120 divided by 255, about 0.47. Each lip
pixel becomes 47% cherry and 53% of its real colour, so your own lips still show through.
small is a speed setting. Finding a face is the slow part of this program, so it
is done on a copy of the frame shrunk to one quarter of the width and height. A
1920 × 1080 frame is searched as 480 × 270, which is 16 times fewer pixels. Without
that trick, each frame would take too long to process and the video would stutter instead of
run smoothly.
3. Open the camera and read one frame
cap = cv2.VideoCapture(0)
while True:
ret, img = cap.read()
if not ret:
break
cv2.VideoCapture(0) connects to the first camera on the computer.
cap.read() returns two things: ret (return), which says whether a
frame arrived, and img (image), the frame itself. If no frame arrives,
ret is False, so not ret is True. That
happens when the camera is unplugged or in use by another program. break then
stops the loop instead of working on an empty frame.
4. Find the face in the small copy
small_img = cv2.resize(img, (0, 0), fx=small, fy=small)
rgb_small_img = cv2.cvtColor(small_img, cv2.COLOR_BGR2RGB)
face_landmarks_list = face_recognition.face_landmarks(rgb_small_img)
cv2.resize makes the quarter-size copy; fx and fy are
the factors for width and height.
cv2.cvtColor flips the colour order from blue-green-red to red-green-blue.
face_recognition was trained on red-first colours, so it expects this order; feed
it the OpenCV order and the face is still found, but the colours it reasons with are wrong.
cvtColor is short for convert colour.
face_landmarks returns a list with one entry per face found, each holding the
points for top_lip, bottom_lip, the eyebrows, the eyes and so on.
5. Paint a white mask where the lips are
mask = np.zeros(img.shape[0:2], dtype=np.uint8)
for face_landmarks in face_landmarks_list:
for lip in ['top_lip', 'bottom_lip']:
points = (np.array(face_landmarks[lip]) / small).astype(np.int32)
cv2.fillPoly(mask, [points], 255)
A mask is a black and white picture the same size as the frame. White marks
where the lipstick goes; black marks everywhere it does not. np.zeros makes it all
black at the start, and img.shape[0:2] gives its size as height and width, the
same as the frame.
The two loops handle every face, and both lips of each face. The points were found on the
quarter-size copy, so they are divided by small to map them back onto the full
frame: a point at 100 becomes 400. astype(np.int32) keeps them as whole pixels,
because OpenCV cannot fill a shape whose corners fall between pixels.
cv2.fillPoly then fills each lip outline with 255, the brightest value. Because
the top and bottom lip points run along both the outer and inner edge of the mouth, the white
area covers the lips without covering the teeth.
6. Soften the edge
mask = cv2.GaussianBlur(mask, (7, 7), 0)
Without this line the mask would jump from white to black at the lip edge, and the lipstick would look cut out with scissors. Blurring mixes each pixel with its neighbours in a 7 × 7 square, so the edge fades instead. The result reads as make-up rather than as a painted shape.
7. Mix the colour into the frame
alpha = (mask / 255 * strength)[:, :, None]
img = (img * (1 - alpha) + np.array(color_lip) * alpha).astype(np.uint8)
What alpha is. alpha is simply the mask turned into a number
between 0 and 1. Dividing by 255 turns pure white (255) into 1 and pure black (0) into 0;
multiplying by strength lowers the white to 0.47. So on the lips alpha is 0.47,
away from them it is 0, and the blurred grey edge lands somewhere in between.
img.shape # (1080, 1920, 3) rows, columns, colour values
alpha.shape # (1080, 1920) rows, columns
alpha[:, :, None].shape # (1080, 1920, 1) rows, columns, 1
Start from the frame. Take a frame 1920 pixels wide and 1080 tall, the usual
size for a webcam at full high definition. NumPy stores it as (1080, 1920, 3): 1080
rows, 1920 pixels in each row, and 3 numbers in each pixel — one for blue, one for green,
one for red.
Now look at alpha. It has the shape (1080, 1920): 1080 rows and
1920 columns, with one number in each pixel and no colour values.
What None adds. The square brackets hold three parts, split by
the two commas: the first : takes every row, the second : takes every
column, and None adds a third layer holding exactly 1 number. The shape becomes
(1080, 1920, 1).
Why that is enough. NumPy has a fixed rule for shapes that do not match: where one side has a 1, it repeats that value until the other side is matched. So the 1 is stretched to 3, and each pixel's single number is used three times — once for blue, once for green, once for red.
The last line. It applies one formula to every pixel of the frame at once: new pixel = old pixel × (1 − alpha) + cherry × alpha. On the lips, alpha is 0.47, so each pixel becomes 53% of your own lips plus 47% cherry — the natural shading of your lips still shows through, which is why the result reads as make-up rather than paint. Away from the lips, alpha is 0, so the old pixel passes through untouched.
8. Show the frame, and stop with q or Q
cv2.imshow("Frame", img)
key = cv2.waitKey(1)
if key in (ord('q'), ord('Q')):
break
cap.release()
cv2.destroyAllWindows()
cv2.imshow shows the finished frame in a window. cv2.waitKey(1)
waits one millisecond for a key, then returns a code for which key was pressed, or
-1 for none. If the code is q or Q, the loop ends and the
camera is released, so other programs can use it and the camera light goes out.
Click inside the video window before typing q; a key press goes to the selected
window, and the terminal in Visual Studio Code is usually the one selected after you press Run.
The two-key way out also works: press Ctrl and C at the same time in
the terminal.
The whole program
The pieces above, put back together. This is the complete file, 35 lines in all:
import cv2
import face_recognition
import numpy as np
# Cherry red from face04_lip.py: (red, green, blue, see-through)
# OpenCV uses blue-green-red order, so the color is written backwards
color_lip = (40, 20, 190)
strength = 120 / 255 # how strong the lipstick is: 0 = none, 1 = solid
small = 0.25 # look for the face in a picture 1/4 the size, so the video is fast
cap = cv2.VideoCapture(0)
while True:
ret, img = cap.read()
if not ret:
break
# face_recognition uses red-green-blue order, OpenCV uses blue-green-red
small_img = cv2.resize(img, (0, 0), fx=small, fy=small)
rgb_small_img = cv2.cvtColor(small_img, cv2.COLOR_BGR2RGB)
face_landmarks_list = face_recognition.face_landmarks(rgb_small_img)
# Mask: white where the lips are, black everywhere else
mask = np.zeros(img.shape[0:2], dtype=np.uint8)
for face_landmarks in face_landmarks_list:
for lip in ['top_lip', 'bottom_lip']:
# The landmarks were found on the small picture, so make them 4 times bigger
points = (np.array(face_landmarks[lip]) / small).astype(np.int32)
cv2.fillPoly(mask, [points], 255)
# Soften the lip edges
mask = cv2.GaussianBlur(mask, (7, 7), 0)
# Mix the lip color into the picture where the mask is white
alpha = (mask / 255 * strength)[:, :, None]
img = (img * (1 - alpha) + np.array(color_lip) * alpha).astype(np.uint8)
cv2.imshow("Frame", img)
key = cv2.waitKey(1)
if key in (ord('q'), ord('Q')):
break
cap.release()
cv2.destroyAllWindows()
Running it on your own computer
Download the zip file and unzip it to
a folder. The zip holds all six programs in this series; this note uses
face06.py. This one does not need a photo in the folder. Open the folder in Visual
Studio Code, then type these three lines in the terminal:
uv python pin 3.12.10
uv sync
uv run face06.py
If you have not set up uv yet, start with How to install Visual Studio Code? and then the uv note for Windows or Mac.
Two practical points. First, the video is smoother on a fast machine; on an older laptop the face is found less often, and the lipstick may lag a moment behind your mouth. Second, the program paints red lips on every face it finds, including someone sitting behind you — which is worth knowing before you record a video for a class or a meeting.
What comes next
Change the colour in one line and you have a different shade. The face is already found in every frame, so any make-up effect is now a matter of what you do with the landmarks.