Face Recognition with OpenCV

This chapter describes how to use OpenCV to compare feature databases for face recognition. This approach is less efficient than MediaPipe, but it allows detection of other objects by replacing the feature database file.

Preparation

Since the product runs the main program automatically at startup by default, which occupies the camera resource, you cannot use this tutorial under that condition. You need to terminate the main program or disable its auto-start, then restart the robot.

Note that the robot's main program uses multi‑threading and is configured to run at boot via crontab, so a conventional sudo killall python usually does not work. Therefore we describe here how to disable the auto-start of the main program.

If you have already disabled the auto-start of the robot's main program, you do not need to execute the Terminate the Main Program section below.

Terminate the Main Program

1. Click the "+" icon next to the current page tab to open a new Launcher tab.

2. Click "Terminal" under "Other" to open a terminal window.

3. In the terminal window, type bash and press Enter.

4. You can now control the robot using the Bash shell.

5. Enter the command: crontab -e

6. If asked which editor to use, type 1 and press Enter to select nano.

7. After opening the crontab configuration file, you should see the following two lines:

@reboot ~/ugv_pt_rpi/ugv-env/bin/python ~/ugv_pt_rpi/app.py >> ~/ugv.log 2>&1
@reboot /bin/bash ~/ugv_pt_rpi/start_jupyter.sh >> ~/jupyter_log.log 2>&1

8. Add a # at the very beginning of the line that starts with ……app.py >> …… to comment it out.

# @reboot ~/ugv_pt_rpi/ugv-env/bin/python ~/ugv_pt_rpi/app.py >> ~/ugv.log 2>&1
@reboot /bin/bash ~/ugv_pt_rpi/start_jupyter.sh >> ~/jupyter_log.log 2>&1

9. In the terminal page, press Ctrl+X to exit. It will ask Save modified buffer? Type Y and press Enter to save the changes.

10. Reboot the device. Note that this process will temporarily close the current Jupyter Lab session. If you did not comment out the line ……start_jupyter.sh >> …… in the previous step, you will still be able to use Jupyter Lab normally after the robot restarts (JupyterLab and the robot main program app.py run independently). You may need to refresh the page.

11. One important point: because the lower computer continuously communicates with the upper computer via the serial port, a voltage glitch on the serial line during the upper computer reboot may prevent it from booting correctly. For example, on a Raspberry Pi as the upper computer, after a reboot the Pi may shut down but not restart – the red LED stays on while the green LED does not light. In that case, you can turn off the robot power switch and then turn it on again; the robot will then restart normally.

12. Enter the reboot command: sudo reboot

13. Wait for the device to restart (during reboot the green LED on the Raspberry Pi will blink; when the blinking slows down or stops, it indicates that startup has succeeded), refresh the page, and continue with the remaining parts of this tutorial.

Example

The following code block can be executed directly:

1. Select the code block below.

2. Press Shift+Enter to run the code block.

3. Watch the real-time video window.

4. Press STOP to close the real-time video and release the camera resource.

If you cannot see the camera's real-time video when running

Click "Kernel" → "Shut down all kernels"
Close this chapter's tab and reopen it
Press STOP to release the camera resource, then re‑run the code block
Reboot the device

Features of this chapter

The face feature database file is located in the same directory as this `.ipynb` file. You can change the detection target by modifying faceCascade – replace the current haarcascade_frontalface_default.xml with another feature file.

When the code block is running normally, point the robot's camera at a face and observe that the detected face is automatically outlined in the video.

import cv2  # import OpenCV for image processing
from picamera2 import Picamera2  # library to access Raspberry Pi Camera
import numpy as np  # library for mathematical operations
from IPython.display import display, Image  # display images in Jupyter Notebook
import ipywidgets as widgets  # create interactive widgets such as buttons
import threading  # create new threads for asynchronous execution

# Load the Haar cascade classifier for face detection
faceCascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')

# Create a "Stop" button that users can click to stop the video stream
# ================
stopButton = widgets.ToggleButton(
    value=False,
    description='Stop',
    disabled=False,
    button_style='danger', # 'success', 'info', 'warning', 'danger' or ''
    tooltip='Description',
    icon='square' # (FontAwesome names without the `fa-` prefix)
)


# Define the display function, which processes video frames and performs face detection
# ================
def view(button):
    # If you are using a CSI camera, uncomment the picam2 code and comment out the camera code
    # Because newer versions of OpenCV (4.9.0.80) no longer support CSI cameras, you need to use picamera2 to capture camera frames
    
    # picam2 = Picamera2()  # create a Picamera2 instance
    # picam2.configure(picam2.create_video_configuration(main={"format": 'XRGB8888', "size": (640, 480)}))  # configure camera parameters
    # picam2.start()  # start the camera

    camera = cv2.VideoCapture(-1) # create a camera instance
    # set resolution
    camera.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
    camera.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)
    
    display_handle = display(None, display_id=True)  # create a display handle for updating the displayed image
    i = 0
    
    avg = None
    
    while True:
        # frame = picam2.capture_array()
        _, frame = camera.read() # capture a frame from the camera
        # frame = cv2.flip(frame, 1) # if your camera reverses your image

        img = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)  # convert from RGB to BGR because OpenCV uses BGR by default
        gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)  # convert to grayscale (face detection is usually done on grayscale images)

        # Perform face detection using the cascade classifier
        faces = faceCascade.detectMultiScale(
                gray,     
                scaleFactor=1.2,
                minNeighbors=5,     
                minSize=(20, 20)
            )

        if len(faces):
            for (x,y,w,h) in faces: # iterate over all detected faces
                cv2.rectangle(frame,(x,y),(x+w,y+h),(64,128,255),1) # draw a rectangle around each detected face
        
        _, frame = cv2.imencode('.jpeg', frame) # encode the frame to JPEG format
        display_handle.update(Image(data=frame.tobytes()))
        if stopButton.value == True:
            # picam2.close() # if yes, close the camera
            cv2.release() # if yes, release the camera
            display_handle.update(None)

            
# Display the "Stop" button and start the display function thread
# ================
display(stopButton)
thread = threading.Thread(target=view, args=(stopButton,))
thread.start()

Navigation menu

17 OpenCV Face Recognition

Contents