Automatic Obstacle Avoiding

From Waveshare Wiki
Jump to: navigation, search

JetBot User Guide

Collision-avoidance Document

  • The collision-avoidance demo of the NVIDIA JetBot open source project is used here. The autonomous obstacle avoidance function of JetBot is divided into three steps: data collection, model training, and autonomous obstacle avoidance.

Data Collection

  • Open the "data_collection.ipynb" file under the directory of Notebooks/collision_avoidance/.

Automatic obstacle avoiding01.png

  • Create a camera connection, display the camera screen on the web page, and be careful not to modify the resolution settings of the camera.

Automatic obstacle avoiding02.png

  • Create a dataset directory to store the images we collect next. Create two folders in the dataset directory, blocked and free. The blocked folder will be used to store the picture of the obstacle avoidance scene, and the free file will be used to store the picture of the unblocked scene. A try/except structure is used here to avoid file overwriting. If these two folders already exist in the path, the program will report an error that the folder already exists, and will not overwrite the old directory.

Automatic obstacle avoiding03.png

  • Create two buttons and two display boxes. The buttons are used to add obstacle avoidance scene pictures and smooth scene pictures. The display boxes respectively display the number of currently stored pictures. If you are running the script for the first time, the number of pictures should be 0. Don't press the button hard, the button here is just for demonstration and has no actual effect.

Automatic obstacle avoiding04.png

  • Write the button associated function. Among them, save_snapshot(directory), this function is used to save the picture, it will be called by the key-triggered program, the function saves the current camera image to the corresponding path; the save_free() function, corresponding to the add free green button above, the function Call the save image function in , save the current camera image to the free folder, and update the number of free images; the save_blocked() function saves the current image to the blocked folder and updates the number of images.

Automatic obstacle avoiding05.png

  • Now, the obstacle avoidance scene and the clear scene graph can be collected. When the car needs to turn when encountering an obstacle, you can press the add blocked red button, and press the add free green button in the unblocked scene; collect at least 100 obstacle avoidance maps and unblocked pictures for each.

Automatic obstacle avoiding06.png

  • Pack and compress the collected images.

Automatic obstacle avoiding07.png

Model Trainning

  • Open Notebooks/collision_avoidance/train_model.ipynb.
  • Import the PyTorch library. PyTorch is a common deep learning library. If you are interested in it, you can learn about it on Baidu or Google.

Automatic Obstacle Avoiding21.png

  • Decompress the packet. Note: If you are doing model training on the same car as me, please skip this step, because your data is already saved in the current directory, if you run the program below, it will keep stuck; If you are training on a different car, you need to copy the data package you collected and packaged before to the current directory, and then run the program below to decompress it.

Automatic Obstacle Avoiding22.png

  • Use the ImgeFolder class to convert our data packets for later training.

Automatic Obstacle Avoiding23.png

  • Divide the data packets into two groups, namely the training group and the test group. The test group is used to verify the accuracy of the model.

Automatic Obstacle Avoiding24.png

  • Create two instances for later shuffling data and generating pictures.

Automatic Obstacle Avoiding25.png

  • Define a neural network. In transfer learning, we can use pre-trained models for new trainning tasks, so that we can directly use some of the learned functions in the pre-trained models, and then train new tasks, which will save a lot of effort. To sum up, it is like teaching a middle school student high school knowledge, which is easier and faster than teaching a child who has no primary school knowledge.

Automatic Obstacle Avoiding26.png

  • Use the downloaded model to judge the results: avoid obstacles and go straight. That is, two labels are required.
  • Convert the model to run on the GPU.

Automatic Obstacle Avoiding27.png

  • Model training. 30 sets of training are performed on the model, and a model file of best_model.pth will be generated after the training. Wait for the training to finish.

Automatic Obstacle Avoiding28.png

Autonomous Obstacle Avoidance

  • After obtaining the model through the above training, we start the obstacle avoidance test.
  • Open the live_demo.ipynb file in the Notebooks/collision_avoidance path. Initialize the PyTorch model.

Automatic Obstacle Avoiding30.png

  • Load the previously trained model file.

Automatic Obstacle Avoiding31.png

  • Load the model into CPU memory and prepare to pass it into the GPU device for calculation.

Automatic Obstacle Avoiding32.png

  • Process camera image data for later calculations.

Automatic Obstacle Avoiding33.png

  • Display the camera image on the web page, and use a slider to display the obstacle avoidance value of the current image.

Automatic Obstacle Avoiding34.png

  • Call the Robot library to prepare the movement of the car.

Automatic Obstacle Avoiding35.png

  • According to the obstacle avoidance value, operate the car to go straight or turn (obstacle avoidance). If the speed of the car is too fast when avoiding obstacles, you can reduce the parameter value in robot.forward() to reduce the forward speed of the car, and reduce the parameter value in robot.left() to reduce the turning range of the car.

Automatic Obstacle Avoiding36.png

  • The camera data is updated in real time for easy observation.

Automatic Obstacle Avoiding37.png

  • If your car is not running properly, or you want to stop the car, you can continue to run the following program.

Automatic Obstacle Avoiding38.png