Template: RPi Camera Libcamera Guide

From Waveshare Wiki
Jump to: navigation, search

About the Libcamera

Raspberry Pi is transitioning from a legacy camera software stack based on proprietary Broadcom GPU code to an open-source stack based on libcamera. Raspberry Pi OS images from Bullseye onwards will contain only the libcamera-based stack. If you use the newest Bullseye Raspberry Pi OS, the libcamera is pre-installed. If you want to use the legacy camera stack (Raspicam), please move to the Raspicam guide.

Installation (optional)

Open a terminal and run the following command:

sudo apt install libcamera-apps

Camera configuration

If you use the official V1 camera (ov5647), V2 camera (IMX219), and the HQ camera (IMX477), you can just plug in the camera and play.
If you use third-party cameras, the support camera size is as below:

Camera Sensor Supported Board Configuration
OV5647 A full range of Raspberry Pi boards Default setting or
dtoverlay=ov5647
IMX219 Compute Module series only default setting or
dtoverlay=imx219
IMX290 and IMX327 A full range of Raspberry Pi Boards dtoverlay=imx290,clock-frequency=74250000 or
dtoverlay=imx290,clock-frequency=37125000
IMX378 A full range of Raspberry Pi boards dtoverlay=imx378
IMX477 Compute Module series only default setting or
dtoverlay=imx477
OV9281 A full range of Raspberry Pi boards dtoverlay=ov9281

To override the automatic camera detection, Bullseye users will also need to delete the entry or change it to value 0.

camera_auto_detect=1

Using libcamera

The libcamera software stack provides six applications/commands for using cameras.

Command Description
libcamera-hello "Hello world" application for previewing camera
libcamera-jpeg A simple still image capture application
libcamera-still Still image capture application which feature more funtsion then libcamera-jpeg
libcamera-vid A video capture application
libcamera-raw A raw Bayer frames recording application
libcamera-detect -

libcamera-hello

This is a "hello world" application for the camera, it starts the camera and displays a preview window.

example

libcamera-hello -t 0

This command should dispay a preview window for about 5 seconds. The -t <duration> option lets the user select how long the window is displayed. if you set the duration value to 0, it runs the preview indefinitely.
The preview can be halted either by clicking the window’s close button or using Ctrl-C in the terminal.

The Tuning File

Raspberry Pi’s libcamera implementation includes a tuning file for each different type of camera module. This is a file that describes or "tunes" the parameters that will be passed to the algorithms and hardware to produce the best image quality. libcamera is only able to determine automatically the image sensor being used, not the module as a whole - even though the whole module affects the "tuning".
For example, the NOIR (no IR-filter) versions of sensors require different AWB settings to the standard versions, so the IMX219 NOIR should be run using.

libcamera-hello --tuning-file /usr/share/libcamera/ipa/raspberrypi/imx219_noir.json

Uses can copy an existing tuning file and alter it according to their own preferences.
The --tuning-file parameter applies identically across all the libcamera applications.

Preview Window

Most of the libcamera applications display a preview image in a window, users can use --info-text parameter to display certain helpful image information on the window title bar using "% directives".
For examples:

libcamera-hello --info-text "focus %focus"

For more information about the option and parameters, please refer to the Command Options part below.

libcamera-jpeg

libcamera-jpeg is a simple still image capture application. It deliberately avoids some of the additional features of libcamera-still which attempts to emulate raspistill more fully. As such the code is significantly easier to understand, and in practice still provides many of the same features.

To capture a full resolution JPEG image use

libcamera-jpeg -o test.jpg

This command will display a preview for about 5s and then acpture a full resolution JPEG image to the file test.jpg.
The -t <duration> option can be used to alter the length of time the preview shows, and the --width and --height options will change the resolution of the captured still image. For example.

libcamera-jpeg -o test.jpg -t 2000 --width 640 --height 480

Exposure Control

All the libcamera applications allow the user to run the camera with fixed shutter speed and gain. for example:

libcamera-jpeg -o test.jpg -t 2000 --shutter 20000 --gain 1.5

This comamnd would capture an image with an exposure of 20ms and a gain of 1.5x. The the gain will be applied as analogue gain within the sensor up until it reaches the maximum analogue gain permitted by the kernel sensor driver, after which the remainder will be applied as digital gain.
Note: Digital gain is applied by the ISP, note by the sensor. The digital gain will always be very close to 1.p unless:

  1. The total gain requested (either by the --gain option, or by the exposure profile in the camera tuning) exceeds that which can be applied as analogue gain within the sensor. Only the extra gain required will be applied as digital gain.
  2. One of the colour gains is less than 1 (note that colour gains are applied as digital gain too). In this case the advertised digital gain will settle to 1 / min(red_gain, blue_gain). This actually means that one of the colour channels - just not the green one - is having unity digital gain applied to it.
  3. The AEC/AGC is changing. When the AEC/AGC is moving the digital gain will typically vary to some extent to try and smooth out any fluctuations, but it will quickly settle back to its "normal" value.

Raspberry Pi’s AEC/AGC algorithm allows applications to specify exposure compensation, that is, the ability to make images darker or brighter by a given number of stops. For example:

libcamera-jpeg --ev -0.5 -o darker.jpg
libcamera-jpeg --ev 0 -o normal.jpg
libcamera-jpeg --ev 0.5 -o brighter.jpg

libcamera-still

libcamera-still is very similar to libcamera-jpeg but supports more of the legacy raspistill options

example

libcamera-still -o test.jpg

Encoders

libcamera-still allows files to be saved in a number of different formats. It supports both png and bmp encoding. It also allows files to be saved as a binary dump of RGB or YUV pixels with no encoding or file format at all. In these latter cases, the application reading the files will have to understand the pixel arrangement for itself.

libcamera-still -e png -o test.png
libcamera-still -e bmp -o test.bmp
libcamera-still -e rgb -o test.data
libcamera-still -e yuv420 -o test.data

Note that the format in which the image is saved depends on the -e option and is not selected automatically based on the output file name.

Raw Image Capture

Raw images are the images produced directly by the image sensor before any processing is applied to them either by the ISP or any of the CPU cores. For colour image sensors these are usually Bayer format images. Note that raw images are quite different from the processed but unencoded RGB or YUV images that we saw earlier.

libcamera-still -r -o test.jpg

These DNG files contain metadata pertaining to the image capture, including black levels, white balance information and the colour matrix used by the ISP to produce the JPEG. This makes these DNG files much more convenient for later "by hand" raw conversion with some of the aforementioned tools. Using exiftool shows all the metadata encoded into the DNG file:

File Name                       : test.dng
Directory                       : .
File Size                       : 24 MB
File Modification Date/Time     : 2021:08:17 16:36:18+01:00
File Access Date/Time           : 2021:08:17 16:36:18+01:00
File Inode Change Date/Time     : 2021:08:17 16:36:18+01:00
File Permissions                : rw-r--r--
File Type                       : DNG
File Type Extension             : dng
MIME Type                       : image/x-adobe-dng
Exif Byte Order                 : Little-endian (Intel, II)
Make                            : Raspberry Pi
Camera Model Name               : /base/soc/i2c0mux/[email protected]/[email protected]
Orientation                     : Horizontal (normal)
Software                        : libcamera-still
Subfile Type                    : Full-resolution Image
Image Width                     : 4056
Image Height                    : 3040
Bits Per Sample                 : 16
Compression                     : Uncompressed
Photometric Interpretation      : Color Filter Array
Samples Per Pixel               : 1
Planar Configuration            : Chunky
CFA Repeat Pattern Dim          : 2 2
CFA Pattern 2                   : 2 1 1 0
Black Level Repeat Dim          : 2 2
Black Level                     : 256 256 256 256
White Level                     : 4095
DNG Version                     : 1.1.0.0
DNG Backward Version            : 1.0.0.0
Unique Camera Model             : /base/soc/i2c0mux/[email protected]/[email protected]
Color Matrix 1                  : 0.8545269369 -0.2382823821 -0.09044229197 -0.1890484985 1.063961506 0.1062747385 -0.01334283455 0.1440163847 0.2593136724
As Shot Neutral                 : 0.4754476844 1 0.413686484
Calibration Illuminant 1        : D65
Strip Offsets                   : 0
Strip Byte Counts               : 0
Exposure Time                   : 1/20
ISO                             : 400
CFA Pattern                     : [Blue,Green][Green,Red]
Image Size                      : 4056x3040
Megapixels                      : 12.3
Shutter Speed                   : 1/20

Very long exposures

To capture very long exposure images, we need to be careful to disable the AEC/AGC and AWB because these algorithms will otherwise force the user to wait for a number of frames while they converge. The way to disable them is to supply explicit values. Additionally, the entire preview phase of the capture can be skipped with the --immediate option.
Capature image and set the exposure time to 100s.

libcamera-still -o long_exposure.jpg --shutter 100000000 --gain 1 --awbgains 1,1 --immediate

The maximum exposure times of the three official Raspberry Pi cameras are given in the table below.

Module Max exposure (seconds)
OV5647 6
IMX219 10
IMX417 230

libcamera-vid

libcamera-vid is the video capture application. By default, it uses the Raspberry Pi’s hardware H.264 encoder. It will display a preview window and write the encoded bitstream to the specified output.
For example, write a 10s video to file:

libcamera-vid -t 10000 -o test.h264

You can use vlc to play the file:

vlc test.h264

Note that this is an unpackaged video bitstream, it is not wrapped in any kind of container format (such as an mp4 file). The --save-pts option can be used to output frame timestamps so that the bitstream can subsequently be converted into an appropriate format

libcamera-vid -o test.h264 --save-pts timestamps.txt

For outputing mkv file:

mkvmerge -o test.mkv --timecodes 0:timestamps.txt test.h264

Encoders

There is support for motion JPEG, and also for uncompressed and unformatted YUV420

libcamera-vid -t 10000 --codec mjpeg -o test.mjpeg
libcamera-vid -t 10000 --codec yuv420 -o test.data

The --codec parameter determines the output format, not the extension of the output file.
The --segment parameter breaks output files up into chunks of the segment size (given in milliseconds). This is quite handy for breaking a motion JPEG stream up into individual JPEG files by specifying very short (1 millisecond) segments

libcamera-vid -t 10000 --codec mjpeg --segment 1 -o test%05d.jpeg

libcamera-raw

libcamera-raw is like a video recording application except that it records raw Bayer frames directly from the sensor. It does not show a preview window. For a 2 second raw clip use

libcamera-raw -t 2000 -o test.raw

The raw frames are dumped with no formatting information at all, one directly after another. The application prints the pixel format and image dimensions to the terminal window so that the user can know how to interpret the pixel data.
By default the raw frames are saved in a single (potentially very large) file. As we saw previously, the --segement option can be used conveniently to direct each to a separate file.

libcamera-raw -t 2000 --segment 1 -o test%05d.raw

In good conditions (using a fast SSD) libcamera-raw can get close to writing 12MP HQ camera frames (18MB of data each) to disk at 10 frames per second. It writes the raw frames with no formatting in order to achieve these speeds; it has no capability to save them as DNG files (like libcamera-still). If you want to be sure not to drop frames you could reduce the framerate slightly using the --framerate option, for example.

libcamera-raw -t 5000 --width 4056 --height 3040 -o test.raw --framerate 8

Common Command Line Options

Option Option (abbreviate) Description
--help -h Print help information for the application
--version Print out a software version number
--list-cameras List the cameras available for use
--camera Selects which camera to use <index>
--config -c Read options from the given file <filename>
--timeout -t Delay before application stops automatically <milliseconds>
--preview -p Preview window settings <x,y,w,h>
--fullscreen -f Fullscreen preview mode
--qt-preview Use Qt-based preview window
--nopreview Set window title bar text <string>
--width Capture image width <width>
--height Capture image height <height>
--viewfinder-width Capture image width <width>
--viewfinder-height Capture image height <height>
--rawfull Force sensor to capture in full resolution mode
--mode Specify sensor mode, given as <width>:<height>:<bit-depth>:<packing>
--viewfinder-mode Specify sensor mode, given as <width>:<height>:<bit-depth>:<packing>
--lores-width Low resolution image width <width>
--lores-height Low resolution image height <height>
--hflip
--vflip
--rotation
Read out with horizontal mirror
Read out with vertical flip
Use hflip and vflip to create the given rotation <angle>
--roi Select a crop (region of interest) from the camera <x,y,w,h>
--sharpness Set image sharpness <number>
--contrast Set image contrast <number>
--brightness Set image brightness <number>
--saturation Set image colour saturation <number>
--ev Set EV compensation <number>
--shutter Set the exposure time in microseconds <number>
--gain
--analoggain
Sets the combined analogue and digital gains <number>
Synonym for --gain
--metering Set the metering mode <string>
--exposure Set the exposure profile <string>
--awb Set the AWB mode <string>
--awbgains Set fixed colour gains <number,number>
--denoise Set the denoising mode <string>
--tuning-file Specify the camera tuning to use <string>
--output -o Output file name <string>
--wrap Wrap output file counter at <number>
--flush Flush output files immediately
Parameter for --info-text option
Directive Substitution
%frame The sequence number of the frame
%fps The instantaneous frame rate
%exp The shutter speed used to capture the image, in microseconds
%ag The analogue gain applied to the image in the sensor
%dg The digital gain applied to the image by the ISP
%rg The gain applied to the red component of each pixel
%bg The gain applied to the blue component of each pixel
%focus The focus metric for the image, where a larger value implies a sharper image
Parameter for --awb option
Mode name Colour temperature
auto 2500K to 8000K
incandescent 2500K to 3000K
tungsten 3000K to 3500K
fluorescent 4000K to 4700K
indoor 3000K to 5000K
daylight 5500K to 5400K
cloudy 7000K to 8500K

Still Command Line Options

Option Option (abbreviate) Description
--quality -q JPEG quality <number>
--exif -x Add extra EXIF tags <string>
--timelapse Time interval between timelapse captures <milliseconds>
--framestart The starting value for the frame counter <number>
--datetime Use date format for the output file names
--timestamp Use system timestamps for the output file names
--restart Set the JPEG restart interval <number>
--keypress -k Capture image when Enter pressed
--signal -s Capture image when SIGUSR1 received
--thumb Set thumbnail parameters <w:h:q> or none
--encoding -e Set the still image codec <string>
  • jpg - JPEG (the default)
  • png - PNG format
  • bmp - BMP format
  • rgb - binary dump of uncompressed RGB pixels
  • yuv420 - binary dump of uncompressed YUV420 pixels.
--raw -r Save the raw file
--latest Make a symbolic link to the latest file saved <string>

Video Command Line Options

Option Option (abbreviate) Description
--quality -q JPEG quality <number>
--bitrate -b H.264 bitrate <number>
--intra -g Intra-frame period (H.264 only) <number>
--profile H.264 profile <string>
--level H.264 level <string>
--codec Encoder to be used <string>
  • h264 - use H.264 encoder (the default)
  • mjpeg - use MJPEG encoder
  • yuv420 - output uncompressed YUV420 frames
--keypress -k Toggle between recording and pausing
--signal -s Toggle between recording and pausing when SIGUSR1 received
--initial Start the application in the recording or paused state <string>
--split Split multiple recordings into separate files
--segment Write the video recording into multiple segments <number>
--circular Write the video recording into a circular buffer of the given <size>
--inline Write sequence header in every I frame (H.264 only)
--listen Wait for an incoming TCP connection
--frames Record exactly this many frames <number>

For more information about the libcamera, please refer to Official Document