Controlling Camera Capture with Buttons

This chapter adds on‑screen buttons to control camera photo capture and video recording. Similar to previous tutorials, images are saved by default in the static folder, and videos are saved by default in the videos folder.

Preparation

Since the product runs the main program automatically at startup by default, which occupies the camera resource, you cannot use this tutorial under that condition. You need to terminate the main program or disable its auto-start, then restart the robot.

Note that the robot's main program uses multi‑threading and is configured to run at boot via crontab, so a conventional sudo killall python usually does not work. Therefore we describe here how to disable the auto-start of the main program.

If you have already disabled the auto-start of the robot's main program, you do not need to execute the Terminate the Main Program section below.

Terminate the Main Program

1. Click the "+" icon next to the current page tab to open a new Launcher tab.

2. Click "Terminal" under "Other" to open a terminal window.

3. In the terminal window, type bash and press Enter.

4. You can now control the robot using the Bash shell.

5. Enter the command: crontab -e

6. If asked which editor to use, type 1 and press Enter to select nano.

7. After opening the crontab configuration file, you should see the following two lines:

@reboot ~/ugv_pt_rpi/ugv-env/bin/python ~/ugv_pt_rpi/app.py >> ~/ugv.log 2>&1
@reboot /bin/bash ~/ugv_pt_rpi/start_jupyter.sh >> ~/jupyter_log.log 2>&1

8. Add a # at the very beginning of the line that starts with ……app.py >> …… to comment it out.

# @reboot ~/ugv_pt_rpi/ugv-env/bin/python ~/ugv_pt_rpi/app.py >> ~/ugv.log 2>&1
@reboot /bin/bash ~/ugv_pt_rpi/start_jupyter.sh >> ~/jupyter_log.log 2>&1

9. In the terminal page, press Ctrl+X to exit. It will ask Save modified buffer? Type Y and press Enter to save the changes.

10. Reboot the device. Note that this process will temporarily close the current Jupyter Lab session. If you did not comment out the line ……start_jupyter.sh >> …… in the previous step, you will still be able to use Jupyter Lab normally after the robot restarts (JupyterLab and the robot main program app.py run independently). You may need to refresh the page.

11. One important point: because the lower computer continuously communicates with the upper computer via the serial port, a voltage glitch on the serial line during the upper computer reboot may prevent it from booting correctly. For example, on a Raspberry Pi as the upper computer, after a reboot the Pi may shut down but not restart – the red LED stays on while the green LED does not light. In that case, you can turn off the robot power switch and then turn it on again; the robot will then restart normally.

12. Enter the reboot command: sudo reboot

13. Wait for the device to restart (during reboot the green LED on the Raspberry Pi will blink; when the blinking slows down or stops, it indicates that startup has succeeded), refresh the page, and continue with the remaining parts of this tutorial.

Example

The following code block can be executed directly:

1. Select the code block below.

2. Press Shift+Enter to run the code block.

3. Watch the real-time video window.

4. Press STOP to close the real-time video and release the camera resource.

If you cannot see the camera's real-time video when running

Click "Kernel" → "Shut down all kernels"
Close this chapter's tab and reopen it
Press STOP to release the camera resource, then re‑run the code block
Reboot the device

Notes

If you are using a USB camera, uncomment the line frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB).

Operation

When the code block is running, click the PHOTO button to take a picture.

import cv2 # import OpenCV for image processing
from picamera2 import Picamera2 # library to access Raspberry Pi Camera
import numpy as np # library for mathematical operations
from IPython.display import display, Image # display images in Jupyter Notebook
import ipywidgets as widgets # create interactive widgets such as buttons
import threading # create new threads for asynchronous execution

import os, time # file and directory operations and time‑related functions

time_intervel = 3 # set the interval for timed photo capture (seconds)

photo_path = '/home/ws/ugv_pt_rpi/static/' # set the directory path to store photos and videos

# Create a "Stop" button that users can click to stop video capture and photo taking
# ================
stopButton = widgets.ToggleButton(
    value=False,
    description='Stop',
    disabled=False,
    button_style='danger', # set button style: 'success', 'info', 'warning', 'danger' or ''
    tooltip='Description',
    icon='square' # set button icon (FontAwesome names without the `fa-` prefix)
)

# Create a "Photo" button that users can click to instantly take a photo
# ================
photoButton = widgets.ToggleButton(
    value=False,
    description='Photo',
    disabled=False,
    button_style='danger', # 'success', 'info', 'warning', 'danger' or ''
    tooltip='Description',
    icon='square' # (FontAwesome names without the `fa-` prefix)
)

# Set the interval for continuous photo capture (seconds)
time_interval = 3
photo_num_count = 0 # initialize photo counter
capture_lock = threading.Lock()
last_photo_time = time.time() # record the time of the last photo

def photo_button_clicked(change, frame):
    global photo_num_count
    if change['new']: # when the "Photo" button is clicked
        photo_num_count = photo_num_count + 1 # increment photo counter
        photo_filename = f'{photo_path}photo_{photo_num_count}.jpg'  # set the save path and filename for the photo
        cv2.imwrite(photo_filename, frame) # save the photo
        print(f'{photo_num_count} photos saved. new photo: {photo_filename}') # print photo save information
        photoButton.value = False # reset the "Photo" button state

# Define the display function, which captures and displays video frames and responds to photo requests
# ================
def view(stop_button, photo_button):
    # If you are using a CSI camera, uncomment the picam2 code and comment out the camera code
    # Because newer versions of OpenCV (4.9.0.80) no longer support CSI cameras, you need to use picamera2 to capture camera frames
    
    # picam2 = Picamera2() # create a Picamera2 instance
    # picam2.configure(picam2.create_video_configuration(main={"format": 'XRGB8888', "size": (640, 480)})) # configure camera parameters
    # picam2.start()

    camera = cv2.VideoCapture(-1) # create a camera instance
    # set resolution
    camera.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
    camera.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)
    
    display_handle = display(None, display_id=True) # create a display handle for updating the displayed image
    i = 0
    while True:
        # frame = picam2.capture_array() # capture a frame from the camera
        _, frame = camera.read() # capture a frame from the camera

        photoButton.observe(lambda change: photo_button_clicked(change, frame), names='value') # listen for clicks on the "Photo" button
            
        _, frame = cv2.imencode('.jpeg', frame) # encode the frame to JPEG format
        display_handle.update(Image(data=frame.tobytes()))
        if stopButton.value == True:
            # picam2.close() # if yes, close the camera
            cv2.release() # if yes, release the camera
            display_handle.update(None)

            
# Display the "Stop" and "Photo" buttons, and start a new thread to execute the display function
# ================
display(stopButton)
display(photoButton)
thread = threading.Thread(target=view, args=(stopButton, photoButton,))
thread.start()

Note that because this example uses JupyterLab widgets, which have some stability issues, when you press the "Photo" button you might save multiple photos. You can browse the ugv_pt_rpi/static/ folder in the left panel of JupyterLab to view the captured photos.

Navigation menu

16 Take Photos by Pressing Buttons

Contents