How to use AI to put lipstick on your face in real time?

Howard Pong Yuen Lam · AI Notes · November 1, 2026 · 7 min read

English 繁體中文 简体中文

This note joins the last two together. From How to get live video from your webcam? it takes the camera loop; from How to use AI to put lipstick on your face? it takes the lipstick. The result is 35 lines that put a moving face on screen with cherry red lips.

The only new idea is that the lips are found again in every frame. Find, paint, show; then the next frame arrives and the same three steps repeat. Because each painting is a fraction of a second, your face looks as if it is wearing lipstick continuously.

Why this program is longer than the one for a photo

The lipstick note ran 14 lines. This one runs 35. The lipstick logic itself is unchanged: the same 12 lip points, the same cherry red, the same filling of the lips. What grows is the cost of working live instead of on a file.

So the extra length comes from keeping up with a moving face.

How the program works

1. Load three libraries

import cv2
import face_recognition
import numpy as np

cv2 is OpenCV: it reads the camera, resizes each frame, changes its colour order, and shows it. face_recognition finds the face and its landmarks, the points that mark the lips. np is NumPy (Numerical Python), which does arithmetic on a whole grid of pixels in one line — necessary here, because mixing colour pixel by pixel, 35 times a second, would otherwise be too slow.

2. Choose the colour and three settings

color_lip = (40, 20, 190)
strength = 120 / 255
small = 0.25

color_lip is cherry red, from the lipstick note. There the red was written as (190, 20, 40) for red, green, blue. OpenCV stores colours in the opposite order, blue then green then red, so the same colour is written (40, 20, 190) here.

strength is how much lipstick to show: 120 divided by 255, about 0.47. Each lip pixel becomes 47% cherry and 53% of its real colour, so your own lips still show through.

small is a speed setting. Finding a face is the slow part of this program, so it is done on a copy of the frame shrunk to one quarter of the width and height. A 1920 × 1080 frame is searched as 480 × 270, which is 16 times fewer pixels. Without that trick, each frame would take too long to process and the video would stutter instead of run smoothly.

3. Open the camera and read one frame

cap = cv2.VideoCapture(0)
while True:
    ret, img = cap.read()
    if not ret:
        break

cv2.VideoCapture(0) connects to the first camera on the computer. cap.read() returns two things: ret (return), which says whether a frame arrived, and img (image), the frame itself. If no frame arrives, ret is False, so not ret is True. That happens when the camera is unplugged or in use by another program. break then stops the loop instead of working on an empty frame.

4. Find the face in the small copy

small_img = cv2.resize(img, (0, 0), fx=small, fy=small)
rgb_small_img = cv2.cvtColor(small_img, cv2.COLOR_BGR2RGB)
face_landmarks_list = face_recognition.face_landmarks(rgb_small_img)

cv2.resize makes the quarter-size copy; fx and fy are the factors for width and height.

cv2.cvtColor flips the colour order from blue-green-red to red-green-blue. face_recognition was trained on red-first colours, so it expects this order; feed it the OpenCV order and the face is still found, but the colours it reasons with are wrong. cvtColor is short for convert colour.

face_landmarks returns a list with one entry per face found, each holding the points for top_lip, bottom_lip, the eyebrows, the eyes and so on.

5. Paint a white mask where the lips are

mask = np.zeros(img.shape[0:2], dtype=np.uint8)
for face_landmarks in face_landmarks_list:
    for lip in ['top_lip', 'bottom_lip']:
        points = (np.array(face_landmarks[lip]) / small).astype(np.int32)
        cv2.fillPoly(mask, [points], 255)

A mask is a black and white picture the same size as the frame. White marks where the lipstick goes; black marks everywhere it does not. np.zeros makes it all black at the start, and img.shape[0:2] gives its size as height and width, the same as the frame.

The two loops handle every face, and both lips of each face. The points were found on the quarter-size copy, so they are divided by small to map them back onto the full frame: a point at 100 becomes 400. astype(np.int32) keeps them as whole pixels, because OpenCV cannot fill a shape whose corners fall between pixels.

cv2.fillPoly then fills each lip outline with 255, the brightest value. Because the top and bottom lip points run along both the outer and inner edge of the mouth, the white area covers the lips without covering the teeth.

6. Soften the edge

mask = cv2.GaussianBlur(mask, (7, 7), 0)

Without this line the mask would jump from white to black at the lip edge, and the lipstick would look cut out with scissors. Blurring mixes each pixel with its neighbours in a 7 × 7 square, so the edge fades instead. The result reads as make-up rather than as a painted shape.

7. Mix the colour into the frame

alpha = (mask / 255 * strength)[:, :, None]
img = (img * (1 - alpha) + np.array(color_lip) * alpha).astype(np.uint8)

What alpha is. alpha is simply the mask turned into a number between 0 and 1. Dividing by 255 turns pure white (255) into 1 and pure black (0) into 0; multiplying by strength lowers the white to 0.47. So on the lips alpha is 0.47, away from them it is 0, and the blurred grey edge lands somewhere in between.

img.shape                      # (1080, 1920, 3)   rows, columns, colour values
alpha.shape                    # (1080, 1920)      rows, columns
alpha[:, :, None].shape        # (1080, 1920, 1)   rows, columns, 1

Start from the frame. Take a frame 1920 pixels wide and 1080 tall, the usual size for a webcam at full high definition. NumPy stores it as (1080, 1920, 3): 1080 rows, 1920 pixels in each row, and 3 numbers in each pixel — one for blue, one for green, one for red.

Now look at alpha. It has the shape (1080, 1920): 1080 rows and 1920 columns, with one number in each pixel and no colour values.

What None adds. The square brackets hold three parts, split by the two commas: the first : takes every row, the second : takes every column, and None adds a third layer holding exactly 1 number. The shape becomes (1080, 1920, 1).

Why that is enough. NumPy has a fixed rule for shapes that do not match: where one side has a 1, it repeats that value until the other side is matched. So the 1 is stretched to 3, and each pixel's single number is used three times — once for blue, once for green, once for red.

The last line. It applies one formula to every pixel of the frame at once: new pixel = old pixel × (1 − alpha) + cherry × alpha. On the lips, alpha is 0.47, so each pixel becomes 53% of your own lips plus 47% cherry — the natural shading of your lips still shows through, which is why the result reads as make-up rather than paint. Away from the lips, alpha is 0, so the old pixel passes through untouched.

8. Show the frame, and stop with q or Q

cv2.imshow("Frame", img)
key = cv2.waitKey(1)
if key in (ord('q'), ord('Q')):
    break
cap.release()
cv2.destroyAllWindows()

cv2.imshow shows the finished frame in a window. cv2.waitKey(1) waits one millisecond for a key, then returns a code for which key was pressed, or -1 for none. If the code is q or Q, the loop ends and the camera is released, so other programs can use it and the camera light goes out.

Click inside the video window before typing q; a key press goes to the selected window, and the terminal in Visual Studio Code is usually the one selected after you press Run. The two-key way out also works: press Ctrl and C at the same time in the terminal.

The whole program

The pieces above, put back together. This is the complete file, 35 lines in all:

import cv2
import face_recognition
import numpy as np
# Cherry red from face04_lip.py: (red, green, blue, see-through)
# OpenCV uses blue-green-red order, so the color is written backwards
color_lip = (40, 20, 190)
strength = 120 / 255          # how strong the lipstick is: 0 = none, 1 = solid
small = 0.25                  # look for the face in a picture 1/4 the size, so the video is fast
cap = cv2.VideoCapture(0)
while True:
    ret, img = cap.read()
    if not ret:
        break
    # face_recognition uses red-green-blue order, OpenCV uses blue-green-red
    small_img = cv2.resize(img, (0, 0), fx=small, fy=small)
    rgb_small_img = cv2.cvtColor(small_img, cv2.COLOR_BGR2RGB)
    face_landmarks_list = face_recognition.face_landmarks(rgb_small_img)
    # Mask: white where the lips are, black everywhere else
    mask = np.zeros(img.shape[0:2], dtype=np.uint8)
    for face_landmarks in face_landmarks_list:
        for lip in ['top_lip', 'bottom_lip']:
            # The landmarks were found on the small picture, so make them 4 times bigger
            points = (np.array(face_landmarks[lip]) / small).astype(np.int32)
            cv2.fillPoly(mask, [points], 255)
    # Soften the lip edges
    mask = cv2.GaussianBlur(mask, (7, 7), 0)
    # Mix the lip color into the picture where the mask is white
    alpha = (mask / 255 * strength)[:, :, None]
    img = (img * (1 - alpha) + np.array(color_lip) * alpha).astype(np.uint8)
    cv2.imshow("Frame", img)
    key = cv2.waitKey(1)
    if key in (ord('q'), ord('Q')):
        break
cap.release()
cv2.destroyAllWindows()

Running it on your own computer

Download the zip file and unzip it to a folder. The zip holds all six programs in this series; this note uses face06.py. This one does not need a photo in the folder. Open the folder in Visual Studio Code, then type these three lines in the terminal:

uv python pin 3.12.10
uv sync
uv run face06.py

If you have not set up uv yet, start with How to install Visual Studio Code? and then the uv note for Windows or Mac.

Two practical points. First, the video is smoother on a fast machine; on an older laptop the face is found less often, and the lipstick may lag a moment behind your mouth. Second, the program paints red lips on every face it finds, including someone sitting behind you — which is worth knowing before you record a video for a class or a meeting.

What comes next

Change the colour in one line and you have a different shade. The face is already found in every frame, so any make-up effect is now a matter of what you do with the landmarks.