Raspberry Pi AI Kit

Hailo-h10

Overview

Introduction

It is equipped with a Hailo-10H AI accelerator and 8GB of onboard RAM and brings generative AI capabilities to the Raspberry Pi 5.

Features

Hailo-10H AI accelerator, delivering 40 TOPS (INT4) inference performance
Computer vision model performance comparable to the Raspberry Pi AI HAT+ (26 TOPS)
Efficiently runs generative AI models leveraging 8GB of onboard RAM
Deep integration into the Raspberry Pi camera software ecosystem
Compliant with the Raspberry Pi HAT+ standard
Includes a heatsink, 16mm stacking header, standoffs, and screws for mounting on a Raspberry Pi 5 with the official Raspberry Pi cooler installed
Operating temperature: 0°C ~ 50°C (ambient temperature)
Comes with mounting hardware kit
Stacked GPIO headers

Hardware Connection

The connection is shown in the figure below:

Working with Raspberry Pi

Update System

Only Raspberry Pi OS Trixie is supported.

sudo apt update
sudo apt full-upgrade -y
sudo rpi-eeprom-update -a
sudo reboot

Identify Device

1. Enable the PCIe interface

Simply connect the hardware. The latest system includes hardware detection and will automatically enable PCIe when the hardware is connected
If it is not enabled, add "dtparam=pciex1" to /boot/firmware/config.txt

2. Enable PCIe Gen3, then add following to /boot/firmware/config.txt (Gen3 mode must be enabled):

dtparam=pciex1_gen=3

3. Restart PI5 after modification, and it will then identify the device. (You can choose not to restart immediately, instead waiting to restart after installing the library)

4. Install required dependencies

sudo apt install dkms
sudo apt install hailo-h10-all

Test

Test Running LLM

Please refer to the link

Download Software Package

Download the file and copy it to the Raspberry Pi: Hailo Model Zoo GenAI Debian Package version 5.1.1

sudo dpkg -i hailo_gen_ai_model_zoo_5.1.1_arm64.deb

Start Hailo Ollama Server and Run LLM

1. Start the local hailo-ollama server:

hailo-ollama

2. In a new terminal window, get the list of available LLMs:

curl --silent http://localhost:8000/hailo/v1/list


As shown, it returns 5 models:
deepseek_r1_distill_qwen:1.5b
llama3.2:3b
qwen2.5-coder:1.5b
qwen2.5-instruct:1.5b
qwen2:1.5b

3. Download a model

curl --silent http://localhost:8000/api/pull \
     -H 'Content-Type: application/json' \
     -d '{ "model": "examplemodel:tag", "stream" : true }'

#Replace examplemodel:tag with a model from the LLM list, e.g., using the llama3.2:3b model
curl --silent http://localhost:8000/api/pull \l \
     -H 'Content-Type: application/json' \
     -d '{ "model": "llama3.2:3b", "stream": true }'
# This will download the model. Wait for the download to complete.

4. Execute
Send a query to the LLM via a POST request.

curl --silent http://localhost:8000/api/chat \
     -H 'Content-Type: application/json' \
     -d '{"model": "examplemodel:tag", "messages": [{"role": "user", "content": "Translate to French: The cat is on the table."}]}'

#Replace examplemodel:tag with a model from the LLM list, e.g., using the llama3.2:3b model
curl --silent http://localhost:8000/api/chat \
     -H 'Content-Type: application/json' \
     -d '{"model": "llama3.2:3b", "messages": [{"role": "user", "content": "Who are you."}]}'
# Wait for a reply (be patient, it may be slow).

Install Docker (Optional)

It is used for Open WebUI and can be skipped.
1. Remove any existing Docker packages:

sudo apt remove $(dpkg --get-selections docker.io docker-compose docker-doc podman-docker containerd runc | cut -f1)

2. Install the Docker apt repository:

# Add Docker's GPG key:
sudo apt update
sudo apt install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
# Add the repository to APT sources:
sudo tee /etc/apt/sources.list.d/docker.sources <<EOF
Types: deb
URIs: https://download.docker.com/linux/debian
Suites: $(. /etc/os-release && echo "$VERSION_CODENAME")
Components: stable
Signed-By: /etc/apt/keyrings/docker.asc
EOF

sudo apt update

3. Check if the docker.sources file was created correctly:

cat /etc/apt/sources.list.d/docker.sources

The output should list the following:

4. Install and run the Docker service:

sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo systemctl start docker

5. Create a docker group:

sudo groupadd docker

6. Add your user to the docker group:

sudo usermod -aG docker $USER

7. Log out and back in or run:

newgrp docker

8. Test Docker:

docker run hello-world

9. Troubleshooting:

1) If you encounter "Unable to find image 'hello-world:latest' locally":

#Check if the service started properly:
sudo systemctl status docker --no-pager -l
#If you see download failure messages like "failed to do request" or "context canceled", try enabling a VPN or check your network connection.
#After adjusting network settings, restart the service:
sudo systemctl restart docker

2) If "sudo systemctl status docker --no-pager -l" shows normal status, but "docker run hello-world" still prompts "Unable to find image 'hello-world:latest' locally"

#This indicates the file wasn't downloaded.
#Try pulling the official image again:
docker pull hello-world
#If a network timeout occurs, configure Docker's systemd proxy.
#After configuring the proxy, retry pulling the official image.

Use Open WebUI (Optional)

After installing Docker in the previous step, you can use Docker to deploy and run the Open WebUI container.

1. Install Open WebUI:

docker pull ghcr.io/open-webui/open-webui:main

2. Ensure the hailo-ollama service is running. Then, start the Open WebUI container and connect it to the hailo-ollama backend server:

docker run -d -e OLLAMA_BASE_URL=http://127.0.0.1:8000 -v open-webui:/app/backend/data --name open-webui --network=host --restart always ghcr.io/open-webui/open-webui:main

3. Monitor the container startup. Container initialization may take up to a minute. To view progress and logs, run the following command and wait for the logs to indicate the server is running and ready to accept connections.

docker logs open-webui -f

4. Access Open WebUI in a web browser by entering: http://127.0.0.1:8080. This will open a chat interface where you can select a model and start interacting with the LLM model.
Click here for more information

Real-Time Vision AI Demo

rpicam-apps uses the Hailo AI neural network accelerator to run camera demos

1. Check the camera

rpicam-hello --list-cameras
Please make sure that the camera is working properly

2. Clone the rpicam-apps repository

sudo apt update && sudo apt install rpicam-apps

3. Test

Object detection

# Original model
rpicam-hello -t 0 --post-process-file /usr/share/rpi-camera-assets/hailo_classifier.json --lores-width 640 --lores-height 640
# Yolov5 model
rpicam-hello -t 0 --post-process-file /usr/share/rpi-camera-assets/hailo_yolov5_segmentation.json --lores-width 640 --lores-height 640
# Yolov8 model
rpicam-hello -t 0 --post-process-file /usr/share/rpi-camera-assets/hailo_yolov8_inference.json --lores-width 640 --lores-height 640

Pose detection

# Yolov8 pose estimation model
rpicam-hello -t 0 --post-process-file /usr/share/rpi-camera-assets/hailo_yolov8_pose.json --lores-width 640 --lores-height 640

For more information, please visit GitHub, Hailo official website

Support

Technical Support

If you need technical support or have any feedback/review, please click the Submit Now button to submit a ticket, Our support team will check and reply to you within 1 to 2 working days. Please be patient as we make every effort to help you to resolve the issue.
Working Time: 9 AM - 6 PM GMT+8 (Monday to Friday)

Submit Now

Navigation menu

Raspberry Pi AI HAT+ 2

Contents