19 Color Recognition Based on OpenCV
Color Recognition with OpenCV
In this chapter, we will add some functions related to modifying frames, such as blurring, color space conversion, erosion, and dilation, to the relevant OpenCV functionality.
Preparation
Since the product runs the main program automatically at startup by default, which occupies the camera resource, you cannot use this tutorial under that condition. You need to terminate the main program or disable its auto-start, then restart the robot.
Note that the robot's main program uses multi‑threading and is configured to run at boot via crontab, so a conventional sudo killall python usually does not work. Therefore we describe here how to disable the auto-start of the main program.
If you have already disabled the auto-start of the robot's main program, you do not need to execute the Terminate the Main Program section below.
Terminate the Main Program
1. Click the "+" icon next to the current page tab to open a new Launcher tab.
2. Click "Terminal" under "Other" to open a terminal window.
3. In the terminal window, type bash and press Enter.
4. You can now control the robot using the Bash shell.
5. Enter the command: crontab -e
6. If asked which editor to use, type 1 and press Enter to select nano.
7. After opening the crontab configuration file, you should see the following two lines:
@reboot ~/ugv_pt_rpi/ugv-env/bin/python ~/ugv_pt_rpi/app.py >> ~/ugv.log 2>&1 @reboot /bin/bash ~/ugv_pt_rpi/start_jupyter.sh >> ~/jupyter_log.log 2>&1
8. Add a # at the very beginning of the line that starts with ……app.py >> …… to comment it out.
# @reboot ~/ugv_pt_rpi/ugv-env/bin/python ~/ugv_pt_rpi/app.py >> ~/ugv.log 2>&1 @reboot /bin/bash ~/ugv_pt_rpi/start_jupyter.sh >> ~/jupyter_log.log 2>&1
9. In the terminal page, press Ctrl+X to exit. It will ask Save modified buffer? Type Y and press Enter to save the changes.
10. Reboot the device. Note that this process will temporarily close the current Jupyter Lab session. If you did not comment out the line ……start_jupyter.sh >> …… in the previous step, you will still be able to use Jupyter Lab normally after the robot restarts (JupyterLab and the robot main program app.py run independently). You may need to refresh the page.
11. One important point: because the lower computer continuously communicates with the upper computer via the serial port, a voltage glitch on the serial line during the upper computer reboot may prevent it from booting correctly. For example, on a Raspberry Pi as the upper computer, after a reboot the Pi may shut down but not restart – the red LED stays on while the green LED does not light. In that case, you can turn off the robot power switch and then turn it on again; the robot will then restart normally.
12. Enter the reboot command: sudo reboot
13. Wait for the device to restart (during reboot the green LED on the Raspberry Pi will blink; when the blinking slows down or stops, it indicates that startup has succeeded), refresh the page, and continue with the remaining parts of this tutorial.
Example
The following code block can be executed directly:
1. Select the code block below.
2. Press Shift+Enter to run the code block.
3. Watch the real-time video window.
4. Press STOP to close the real-time video and release the camera resource.
If you cannot see the camera's real-time video when running
- Click "Kernel" → "Shut down all kernels"
- Close this chapter's tab and reopen it
- Press STOP to release the camera resource, then re‑run the code block
- Reboot the device
Operation
In the example, we detect a blue ball by default. Make sure there are no blue objects in the background that could interfere with color recognition. You can also change the detection color (HSV color space) through further development.
import cv2
import imutils, math
from picamera2 import Picamera2 # library to access Raspberry Pi Camera
import numpy as np # library for mathematical operations
from IPython.display import display, Image # display images in Jupyter Notebook
import ipywidgets as widgets # create interactive widgets such as buttons
import threading # create new threads for asynchronous execution
# Create a "Stop" button that users can click to stop the video stream
# ================
stopButton = widgets.ToggleButton(
value=False,
description='Stop',
disabled=False,
button_style='danger', # 'success', 'info', 'warning', 'danger' or ''
tooltip='Description',
icon='square' # (FontAwesome names without the `fa-` prefix)
)
# Define the display function, which processes video frames and recognizes objects of a specific color
def view(button):
# If you are using a CSI camera, uncomment the picam2 code and comment out the camera code
# Because newer versions of OpenCV (4.9.0.80) no longer support CSI cameras, you need to use picamera2 to capture camera frames
# picam2 = Picamera2() # create a Picamera2 instance
# Configure camera parameters: set video format and size
# picam2.configure(picam2.create_video_configuration(main={"format": 'XRGB8888', "size": (640, 480)}))
# picam2.start() # start the camera
camera = cv2.VideoCapture(-1) # create a camera instance
# set resolution
camera.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
camera.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)
display_handle = display(None, display_id=True) # create a display handle for updating the displayed image
i = 0
# Define the color range to detect
color_upper = np.array([120, 255, 220])
color_lower = np.array([90, 120, 90])
min_radius = 12 # define the minimum radius of the object to detect
while True:
# img = picam2.capture_array() # capture a frame from the camera
_, img = camera.read() # capture a frame from the camera
# frame = cv2.flip(frame, 1) # if your camera reverses your image
blurred = cv2.GaussianBlur(img, (11, 11), 0) # apply Gaussian blur to the image to remove noise
hsv = cv2.cvtColor(blurred, cv2.COLOR_BGR2HSV) # convert the image from BGR to HSV color space
mask = cv2.inRange(hsv, color_lower, color_upper) # create a mask to keep only objects within the specific color range
mask = cv2.erode(mask, None, iterations=5) # apply erosion to the mask to remove small white spots
mask = cv2.dilate(mask, None, iterations=5) # apply dilation to the mask to highlight object regions
# Find contours in the mask
cnts = cv2.findContours(mask.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = imutils.grab_contours(cnts) # extract contours
center = None # initialize the center of the object
if len(cnts) > 0:
# find the largest contour in the mask, then use
# it to compute the minimum enclosing circle and
# centroid
c = max(cnts, key=cv2.contourArea) # find the largest contour
((x, y), radius) = cv2.minEnclosingCircle(c) # compute the minimum enclosing circle of the contour
M = cv2.moments(c) # compute the moments of the contour
center = (int(M["m10"] / M["m00"]), int(M["m01"] / M["m00"])) # compute the center of the contour from the moments
if radius > min_radius: # if the radius of the minimum enclosing circle is larger than the preset minimum radius, draw the circle and center point
cv2.circle(img, (int(x), int(y)), int(radius), (128, 255, 255), 1) # draw the minimum enclosing circle
_, frame = cv2.imencode('.jpeg', img) # encode the frame to JPEG format
display_handle.update(Image(data=frame.tobytes())) # update the displayed image
if stopButton.value == True: # check if the "Stop" button has been pressed
# picam2.close() # if yes, close the camera
cv2.release() # if yes, release the camera
display_handle.update(None) # clear the displayed content
# Display the "Stop" button and start the display function thread
# ================
display(stopButton)
thread = threading.Thread(target=view, args=(stopButton,))
thread.start()