JetRacer ROS AI Kit Advanced Tutorial III: ROS OpenCV

From Waveshare Wiki
Revision as of 08:09, 7 November 2022 by Eng52 (talk | contribs) (Created page with "==Introduction== *Since it is inconvenient for the robot to display images, the following OpenCV programs are run on the virtual machine side. The robot starts the camera sect...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Introduction

  • Since it is inconvenient for the robot to display images, the following OpenCV programs are run on the virtual machine side. The robot starts the camera section to collect images and compress them for transmission, and the virtual machine receives the compressed images and processes them.
  • Run the following command on the robot side to start the camera node.
  roslaunch jetbot_pro csi_camera.launch #Start the camera node

ROS OpenCV01.png

  • [Note: Before running the OpenCV program, you must ensure that the robot camera node starts normally, otherwise the program will not respond or make an error.]

Step 1: Convert ROS image topic to OpenCV image using CV Brige

  • Convert the ros image topic to opencv image through cv brige, convert the image to grayscale, and then publish the ROS image topic.
  roslaunch jetbot_pro cv_video.launch

ROS OpenCV02.png

Step 2: Image Calibration

  • The camera image is calibrated through the camera_info camera information topic data, the camera needs to be calibrated before running this program.
  roslaunch jetbot_pro calibration.launch #Camera calibration

ROS OpenCV03.png

  • Camera image shape has been fixed.

Step 3: Contour detection

  • Find the outline of the original image and display it, and finally add a colorful dynamic effect to the outline.
  roslaunch jetbot_pro contours.launch

ROS OpenCV04.png

Step 4: Color Recognization

  • After the program runs, it will detect the blue in the image and filter the blue image to display it separately.
  roslaunch jetbot_pro color_tracking.launch

ROS OpenCV05.png

  • The default color checked by the program is blue. You can also set the recognized color by selecting the color in the image with the mouse.
  • [Note: Choose bright colors for easy identification; the range selected by the box must be a single color, and the selection of multiple colors will cause the HSV range to be too wide to be recognized normally.]

ROS OpenCV06.png

  • After selecting the color, you can also fine-tune the color by adjusting the HSV range through dynamic parameters. Run the following command to start the dynamic parameter adjustment interface, and select color_tracking.
  • First, please adjust the range of hue H to determine the color type, and then adjust the range of saturation S and lightness V to make the recognition effect better.
 rosrun rqt_reconfigure rqt_reconfigure   

ROS OpenCV07.png

Step 5: Motion Detection

  • Enter the following command to start the motion detection node.
  roslaunch jetbot_pro motion_detect.launch
  • After starting the program, the moving object will be detected in the image and framed, and the image on the right shows the afterimage of the moving object.

ROS OpenCV08.png

Step 6: Object Tracking

  • nter the following command to start the object tracking node.
 roslaunch jetbot_pro object_tracking.launch
  • After starting the program, drag the speed in the image window to select the object to be tracked. After the tracking object is created, the tracking algorithm type will be displayed in the upper left corner. After moving the object, the blue frame will also move with the object.

ROS OpenCV09.png

  • Note: It is easier to track objects with complex images, and it is easy to lose tracked objects if you move too fast.
  • The program supports eight tracking algorithms such as ['BOOSTING', 'MIL', 'KCF', 'TLD', 'MEDIANFLOW', 'GOTURN', 'MOSSE', 'CSRT']. The default MOSSE algorithm is used, and the speed of this algorithm is Fast, but less accurate. The accuracy of choosing KCF will be higher, but this algorithm requires higher network transmission speed, and when the network transmission speed is slow, it is easy to cause image delay and freeze.
  • Enter the following command to use the KCF tracking algorithm.
 roslaunch jetbot_pro object_tracking.launch tracker_type:="KCF"

Step 7: AR Vision

  • Enter the following command to start the AR vision node.
 roslaunch jetbot_pro pose.launch
  • After the program runs, it will detect the chessboard paper and display a cube on the chessboard paper, and the chessboard paper moves the cube with it.

ROS OpenCV10.png

Step 8: Face Detection

  • Enter the following command to start the face detection node.
 roslaunch jetbot_pro face_detect.launch
  • After the program runs, the face will be detected in the image, and when the face is detected, the position of the face will be displayed with a green frame.

ROS OpenCV11.png

Step 9: Visual Transmission Line

  • Note: The camera node and the robot chassis node need to be activated for visual inspection. If the camera node has been activated, it does not need to be activated repeatedly.
roslaunch jetbot_pro jetbot.launch #Start the chassis node
roslaunch jetbot_pro csi_camera.launch #Start the camera node
  • Enter the following command to start the visual line patrol node.
 roslaunch jetbot_pro line_follow.launch
  • After the program starts, drag the mouse to select the line to select the color, the left side is the recognized image, and the right side is the line patrol effect.
  • + means to identify the center position of the image, and then adjust the angular velocity of the robot according to the slope of the blue line in the figure to achieve the effect of line tracking.

ROS OpenCV12.png

  • The program also starts the dynamic parameter debugging interface. If line_follow is not displayed, you can click Refresh to update it.
  • You can fine-tune the color by adjusting the HSV range with dynamic parameters. First adjust the range of hue H to determine the color type, and then adjust the range of saturation S and lightness V to make the recognition effect better.

ROS OpenCV13.png

  • Among which:
    • linear is to set the linear speed of the trolley
    • kp, kd are motor speed PID parameters
    • test is the robot switch
  • After selecting the color, you can check the test to start the robot and start selecting the trace line. When the line is recognized, the robot moves along the line, and the robot stops when it rushes out of the line or does not recognize the line.
  • The value of the initial startup linear is set to a smaller value, and the speed is slower to prevent it from rushing out of the line.
  • Adjust the Kp and Kd values to achieve the replacement line tracking effect, first adjust the proportional parameter Kp, and then adjust the integral parameter kd. [You can start the motor waveform display to observe the response results after changing Kp and Kd.]