12 Implement Low Latency Video Transmission with Flask

From Waveshare Wiki
Jump to: navigation, search

Low‑Latency Video Streaming Using Flask

This chapter introduces how to use Flask to build a web application that displays real‑time video from the robot's camera. Because web applications are cross‑platform, users can view the live camera feed on mobile phones, PCs, tablets, etc., via a browser, enabling wireless video transmission.

What is Flask?

Flask is a lightweight web application framework for quickly building web applications using Python.

  • Lightweight: Flask is a lightweight framework. Its core library is relatively small, but it is flexible and extensible, allowing developers to add the extensions and libraries they need.
  • Simple and easy to use: Flask is simple to design and easy to get started with. Its API is clear and concise, and its documentation is detailed, allowing developers to quickly start building web applications.
  • Routing system: Flask uses decorators to define URL routes, mapping requests to corresponding handler functions. This makes creating different pages and handling different requests intuitive and simple.
  • Template engine: Flask integrates the Jinja2 template engine, making it easier to build dynamic content within an application. The template engine allows you to embed dynamically generated content in HTML.
  • Integrated development server: Flask comes with a simple integrated development server that facilitates development and debugging. However, in production environments, it is recommended to use more powerful web servers such as Gunicorn or uWSGI.
  • Plugins and extensions: Flask supports many plugins and extensions to add additional functionality, such as database integration, authentication, form handling, etc.
  • RESTful support: Flask provides good support for RESTful‑style APIs, making it easy to build and design RESTful APIs.
  • WSGI compliant: Flask is based on WSGI (Web Server Gateway Interface), so it can run on many WSGI‑compliant web servers.
  • Active community: Flask has a large and active community, which means you can easily find extensive documentation, tutorials, third‑party extensions, and get support.

Preparation

Because the product automatically runs the main program at boot by default, and the main program uses the camera resource, you cannot use this tutorial while it is running. You need to stop the main program or disable its auto‑start, then restart the robot.

Note: The main program uses multi‑threading and is configured to run automatically at boot via crontab, so the usual sudo killall python method typically does not work. Therefore, we describe how to disable the automatic startup of the main program.

If you have already disabled the automatic startup of the main program, you can skip the Stop the Main Program section below.

Stop the Main Program

1. Click the "+" next to the current page tab to open a new tab named "Launcher".

2. Click "Terminal" under "Other" to open a terminal window.

3. In the terminal window, type bash and press Enter.

4. You can now use the Bash shell to control the robot.

5. Enter the command: crontab -e

6. If asked which editor to use, type 1 and press Enter to select nano.

7. After opening the crontab configuration file, you will see the following two lines: 

@reboot ~/ugv_pt_rpi/ugv-env/bin/python ~/ugv_pt_rpi/app.py >> ~/ugv.log 2>&1
@reboot /bin/bash ~/ugv_pt_rpi/start_jupyter.sh >> ~/jupyter_log.log 2>&1

8. Add a # at the very beginning of the line containing `……app.py >> ……` to comment it out. 

# @reboot ~/ugv_pt_rpi/ugv-env/bin/python ~/ugv_pt_rpi/app.py >> ~/ugv.log 2>&1
@reboot /bin/bash ~/ugv_pt_rpi/start_jupyter.sh >> ~/jupyter_log.log 2>&1

9. In the terminal page, press Ctrl+X to exit. It will ask Save modified buffer? – type Y and press Enter to save the changes.

10. Restart the device. Note that this will temporarily close the current Jupyter Lab. If you did not comment out the line containing ……start_jupyter.sh >>…… in the previous step, Jupyter Lab will still work normally after the robot reboots (Jupyter Lab and the main program app.py run independently). You may need to refresh the page.

11. One thing to note: because the lower computer continuously communicates with the upper computer via the serial port, the upper computer may not boot properly due to continuous level changes on the serial port during the reboot. For example, with a Raspberry Pi as the upper computer, after rebooting, the Raspberry Pi may not turn back on – the red LED stays on and the green LED does not light. In that case, you can turn off the robot's power switch and then turn it back on; the robot should restart normally.

12. Enter the reboot command: sudo reboot

13. Wait for the device to restart (during the reboot, the green LED on the Raspberry Pi will blink; when the blinking slows down or stops, the system has started successfully). Refresh the page and continue with the rest of this tutorial.

Web Application Example

Note: Do not run the following code block inside Jupyter Lab

Because a Flask application will conflict with Jupyter Lab over port usage, the following code cannot be run inside Jupyter Lab. This program is stored in the folder named 12 inside tutorial_cn or tutorial_en. Inside the 12 folder, there is also a folder named template containing web resources. Here is how to run the example.

1. Open a terminal as described above. Note the folder path on the left; the new terminal opens with the same path as the current file location. You need to navigate to the tutorial_cn or tutorial_en folder. After opening the terminal, type cd 12 to navigate into the 12 folder.

2. Start the Flask web application server with the following command: python flask_camera.py

3. Then, on a device within the same local network (or on the same device, open a new browser tab), open a browser and enter the Raspberry Pi's IP address followed by :5000 (for example, if the Raspberry Pi's IP is 192.168.10.104, open 192.168.10.104:5000). Note that the colon : must be an English colon.

4. Press Ctrl+C in the terminal to stop the program.

Flask Program Explanation

from flask import Flask, render_template, Response  # Import Flask class, render_template for HTML templates, Response for response objects
from picamera2 import Picamera2  # Import Picamera2 class for camera access and control
import time  # Import time module for time‑related tasks
import cv2  # Import OpenCV for image processing

app = Flask(__name__)  # Create a Flask application instance

def gen_frames():  # Define a generator function to yield camera frames one by one
    picam2 = Picamera2()  # Create a Picamera2 instance

    # Configure camera parameters: set video format and size
    picam2.configure(picam2.create_video_configuration(main={"format": 'XRGB8888', "size": (640, 480)}))

    picam2.start()  # Start the camera
    while True:
        frame = picam2.capture_array()  # Capture one frame from the camera
        frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
        ret, buffer = cv2.imencode('.jpg', frame)  # Encode the captured frame as JPEG

        frame = buffer.tobytes()  # Convert the JPEG image to a byte stream

        # Use yield to return the image byte stream, forming a continuous video stream
        yield (b'--frame\r\n'
               b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n')

@app.route('/')  # Define the root route
def index():
    return render_template('index.html')  # Return the index.html page

@app.route('/video_feed')  # Define the video stream route
def video_feed():
    # Return a response object containing the video stream, content type is multipart/x-mixed-replace
    return Response(gen_frames(), mimetype='multipart/x-mixed-replace; boundary=frame')

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000, debug=True)  # Start the Flask application, listen on all network interfaces on port 5000, enable debug mode

Explanation of Key Parts of the Code

gen_frames(): This generator function continuously captures frames from the camera, encodes them as JPEG, and yields the frame bytes as part of a multipart response. The generated frames are streamed to the client in real time.

@app.route('/'): This decorator associates the index() function with the root URL (/). When a user visits the root URL, it renders the HTML template named '12_index.html'.

@app.route('/video_feed'): This decorator associates the video_feed() function with the /video_feed URL. This route is used for real‑time video streaming; frames are sent as a multipart response.

app.run(host='0.0.0.0', port=5000, debug=True): This line starts the Flask development server, listening on all available network interfaces (0.0.0.0) on port 5000. The debug=True option enables debug mode for the server.

Explanation of the Web Page Part

Comment:
<!doctype html>: Declares the HTML document type.
<html lang="en">: The root element of the HTML document, specifying the page language as English.
<head>: Contains meta‑information about the document, such as character set and page title.
<!-- Required meta tags -->: HTML comment reminding that these are required meta tags.
<meta charset="utf-8">: Specifies that the document uses the UTF‑8 character set.
<title>Live Video Based on Flask</title>: Sets the page title.
<body>: Contains the visible part of the document.
<!-- The image tag below is dynamically updated with the video feed from Flask -->: HTML comment explaining that the image tag below is dynamically updated with the video stream from Flask.
<img src="{{ url_for('video_feed') }}">: An image tag that uses the video_feed route defined in Flask to obtain the live video stream.
Retrieved from "https://www.waveshare.com/w/index.php?title=12_Implement_Low_Latency_Video_Transmission_with_Flask&oldid=110512"