JetBot AI Kit

From Waveshare Wiki
Jump to: navigation, search
JetBot AI Kit
JetBot AI Kit

AI Robot Kit base on Jetson Nano Developer Kit
{{{name2}}}

{{{name3}}}

{{{name4}}}

{{{name5}}}

Instruction

This is a AI Robot kit based on Jetson Nano Developer Kit. Supports facial recognition, object tracking, auto line following or collision advancing and so on.

User Guides

1. Install Image

【Note】 The software part of this guide mostly based on NVIDIA jetbot wiki , you can also directly refer to it

Step 1. Write JetBot image to SD card
JetBot AI Kit Manual 1.jpg
  • After writting, eject the SD card
Step 2. Start up Jetson Nano Developer Kit
  • Insert SD card to SD card slot of Jetson nano (slot is under Jetson Nano board)
  • Connect HDMI display (if you don't have HDMI or DP display and want to buy one, recommend our HDMI display), keyboard and Mouse to Jetson Nano Developer Kit

【Note】You had better test the Jetson Nano Developer Kit before you assemble JetBot

Step 3. Connect Jetbot to WIFI

All the examples use WIFI, we need to connect JetBot to WIFI firstly.

  • Start Jetson nano Developer Kit, default user name and password of Jetbot are both jetbot
  • Click Network icon on top-right of Desktop and connect WIFI
  • Power off. Then assemble Jetbot
  • Start Jetson nano again. After booting, Ubuntu will auto-connect WIFI, IP address is also displayed on OLED
Step 4. Access JetBot via Web
  • After networking. You can remove peripherals and power adapter.
  • Turn Power switch of Jetbot into On
  • After booting, IP address of OLED can be displayed on OLED
  • Navigate to http://<jetbot_ip_address>:8888 from your desktop's web browser
Step 5. Install latest software (optional)

The JetBot GitHub repository may contain software that is newer than that pre-installed on the SD card image. To install the latest software:

  • Access Jetbot by going to http://<jetbot_ip_address>:8888
  • Launch a new terminal. Default user name and password are both jetbot
JetBot AI Kit Manual 2.jpg
  • Get and install the latest JetBot repository from GitHub.The repository provided here is modified by Waveshare, supports displaying current voltage of batteries. If you want to install original codes, please following NVIDIA jetbot GitHub.
git clone https://github.com/waveshare/jetbot
cd jetbot
sudo python3 setup.py install
Step 6. Configure power mode

To ensure that the Jetson Nano doesn't draw more current than the battery pack can supply, place the Jetson Nano in 5W mode by calling the following command

  • You need to launch a new Terminal and enter following commands to select 5W power mode
sudo nvpmodel -m1
  • Check if mode is correct
sudo nvpmodel -q

【Note】m1: 5W power mode, m2: 10W power model

2. Basic motion

  • Access jetbot by going to http://<jetbot_ip_address>:8888, navigate to ~/Notebooks/basic_motion/
  • Open basic_motion.ipynb file and following the notebook

【Note】You can click icon ▶ to run codes, or select Run -> Run Select Cells. Make sure the JetBot has engough space to run.

3. Teleoperations

  • Access JetBot by going to https://<jetbot_ip_address>:8888, navigate to ~/Notebooks/teleoperation/
  • Open teleoperation.ipynb file and following notebook
  • Connect USB adapter to PC
  • Go to https://html5gamepad.com, check the INDEX of Gamepad
JetBot AI Kit Manual 7.jpg
  • Modify the index. Run and test Gamepad
JetBot AI Kit Manual 8.jpg
  • Modify axes values if require, here we use axes[0] and axes[1]
JetBot AI Kit Manual 9.jpg
  • For more details, please refer to notebook

4. Collision_avoidance

In this example we'll collect an image classification dataset that will be used to help keep JetBot safe! We'll teach JetBot to detect two scenarios free and blocked. We'll use this AI classifier to prevent JetBot from entering dangerous territory.

Step 1. Collect data on JetBot
  • Access JetBot by going to https://<jetbot_ip_address>:8888, navigate to ~/Notebooks/collision_avoidance/
  • Open data_collection.ipynb file and following notebook
  • This model was trained on a limited dataset using the IMX219-160 Camera with wide angle attachment.
  • You need to put the JetBot to different spaces for collecting data as more as possible
JetBot AI Kit Manual 13.jpg
Step 2. Train neural network
  • Access JetBot by going to https://<jetbot_ip_address>:8888, navigate to ~/Notebooks/collision_avoidance/
  • Open and follow the tain_model.ipynb notebook
Step 3. Run live demo on JetBot
  • Access JetBot by going to https://<jetbot_ip_address>:8888, navigate to ~/Notebooks/collision_avoidance/
  • Open and following the live_demo.ipynb notebook

5. Object following

Here we use coco dataset

  • Access JetBot by going to https://<jetbot_ip_address>:8888, navigate to ~/Notebooks/object_following/
  • Before running example, you should upload the pre-trained ssd_mobilenet_v2_coco.engine model to current directory, and the model used at the last chapter is required as well.
  • Open and follow the live_demo.ipynb notebook
JetBot AI Kit Manual 23.jpg

6. Line tracking

This chapter we will use data collect, link tracking and auto detecting to realize Robot auto line-tracking

Step 1. Collect data by JetBot
  • Access JetBot by going to https://<jetbot_ip_address>:8888, navigate to ~/Notebooks/road_following/
  • Open data-collection.ipynb file
JetBot AI Kit Manual 24.jpg
  • Running the codes and a video is played, you can follow it
JetBot AI Kit Manual 25.jpg
  • On the image captured by camera, there are a green point and a blue line. The point and line is the expected road which Robot run
  • The content below is similar to [#3. Teleoperation], modify the index and axes values

【Note】The axes buttons used here should be analog buttons, which support

JetBot AI Kit Manual 26.jpg
  • Modify button value for capturing. (you can also keep default setting)
JetBot AI Kit Manual 27.jpg
  • Collecting data. Set JetBot to different place of the lines, use Gamepad to move green point to the black line. Blue line is the way Jetbot expected to run in. You can press capture button to capture picture. You should collect pictures as soon as possible, count shows amount of the pictures captured.

【Note】If Gamepad is inconvenient for you, you can set the position of green point by dragging steering and throttle sliders.

  • Save pictures
JetBot AI Kit Manual 28.jpg
Step 2. Training model
  • Access Jetbo by going to https://<jetbot_ip_address>:8888, navigate to ~/Notebooks/road_following/
  • Open train_model.ipynb file
  • If you use the data collect above, you needn't to unzip files next
  • If you use external data, you need to modify the name road_following.zip to corresponding file name and run the cell
JetBot AI Kit Manual 29.jpg
JetBot AI Kit Manual 30.jpg
  • Download Model
JetBot AI Kit Manual 31.jpg
  • Train model, it will genrate best_steerin_mdel_xy.pth file
Step 3. Road following
  • Access Jetbo by going to https://<jetbot_ip_address>:8888, navigate to ~/Notebooks/road_following/
  • Open live_demo.ipynb file
  • Load model and open camera for living video.
  • You can drag the sliders to modify parameters
JetBot AI Kit Manual 32.jpg
  • x, y are forecast values. Speed is VSL of jetbot, steering is steering speed of jetbot
JetBot AI Kit Manual 33.jpg
  • Move Jetbot by change the speed gain
【Note】 You cannot set the speed gain too high, otherwise, JetBot may run fast and go off the rail. You can also set the steering smaller to make motion of jetbot much more smooth.
JetBot AI Kit Manual 34.jpg

7. ROS

Resources

FAQ

 Answer:
Three 18650 batteries are used and voltage of every battery is 3.7V. Generally, voltage of per battery is 4.2V when full charging.

BTW, Power of JetBot is down when the voltage of whole system is similar to 9V (it is not accurate), we recommend you to charge batteries if the voltage displayed is lower than 10V.

{{{4}}}

{{{5}}}


Supports

Support

If you require technical support, please go to the Support page and open a ticket.