IMX219-83 Stereo Camera

From Waveshare Wiki
Jump to: navigation, search
IMX219-83 Stereo Camera
IMX219-83 Stereo Camera

IMX219 sensor, FOV 83, compatible with jetson nano
{{{name2}}}

{{{name3}}}

{{{name4}}}

{{{name5}}}

Onboard Interfaces
CSI

Introduction

IMX219 Camera, 800 megapixels, and 83 FOV. Compatible with Jetson nano Developer Kit (B01)

More

Specification

  • 8 Megapixels
  • Sensor: Sony IMX219
  • Resolution: 3280 × 2464 (per camera)
  • Lens specifications:
    • CMOS size: 1/4inch
    • Focal Length: 2.6mm
    • Angle of View: 83/73/50 degree (diagonal/horizontal/vertical)
    • Distortion: <1%
    • Baseline Length: 60mm
  • ICM20948:
    • Accelerometer:
      • Resolution: 16-bit
      • Measuring Range (configurable): ±2, ±4, ±8, ±16g
      • Operating Current: 68.9uA
    • Gyroscope:
      • Resolution: 16-bit
      • Measuring Range (configurable): ±250, ±500, ±1000, ±2000°/sec
      • Operating Current: 1.23mA
    • Magnetometer:
      • Resolution: 16-bit
      • Measuring Range: ±4900μT
      • Operating Current: 90uA
  • Dimension: 24mm × 85mm

Selection Guide

Product Pixels Sensor Dual Len Infrared Angle of View (Diagonal)(degree) Afterture (F) Focal Length (MM) PCBA Suit for
RPi Camera 8 Mega IMX219 × × 62.2 2.0 3.04 Raspberry Pi, Jetson Nano
RPi NoIR Camera V2 62.2 2.0 3.04 Raspberry Pi, Jetson Nano
IMX219-77 Camera × 77 2.0 2.96 Jetson Nano, CM3/3+
IMX219-77IR 77 2.0 2.96 Jetson Nano, CM3/3+
IMX219-120 Camera × 120 2.2 1.88 Jetson Nano, CM3/3+
IMX219-160 Camera × 160 2.35 3.15 Jetson Nano, CM3/3+
IMX219-160IR Camera 160 2.35 3.15 Jetson Nano, CM3/3+
IMX219-160 IR-CUT Camera 160 2.7 3.62 Jetson Nano, CM3/3+
IMX219-170 Camera × 170 2.0 0.87 Jetson Nano, CM3/3+
IMX219-200 Camera × 200 2.0 3.15 Jetson Nano, CM3/3+
IMX219-D160 × 160 2.35 3.15 × Raspberry Pi, Jetson Nano
IMX219-83 Stereo Camera × 83 NA 2.6 Jetson Nano, CM3/3+

Hardware connection

  • Connect the camera to the CSI interfaces of Jetson Nano. Set the metal side of FFC into Heat-sink
  • Connect an HDMI LCD to Jetson Nano
  • Connect the I2C interface (only the SDA and SCL pins are required) of the Camera to I2C interface of the Jetson Nano Developer Kit (the Pin3, and Pin5)

Software setting

  • Power on Jetson Nano and open the Terminal (Ctrl+ALT+T)
  • Check the video devices with command:
ls /dev/video*
Check if both video0 and video1 are detected
  • Test video0
DISPLAY=:0.0 gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=3280, height=2464, format=(string)NV12, framerate=(fraction)20/1' ! nvoverlaysink -e
  • Test video1
 DISPLAY=:0.0 gst-launch-1.0 nvarguscamerasrc sensor-id=1 ! 'video/x-raw(memory:NVMM), width=3280, height=2464, format=(string)NV12, framerate=(fraction)20/1' ! nvoverlaysink -e
  • Test the ICM20948
wget https://www.waveshare.com/w/upload/a/a4/D219-9dof.tar.gz
tar zxvf D219-9dof.tar.gz
cd D219-9dof/07-icm20948-demo
make
./ICM20948-Demo
  • If you find that the image captured is red. You can try to download .isp file and installed:
wget https://www.waveshare.com/w/upload/e/eb/Camera_overrides.tar.gz
tar zxvf Camera_overrides.tar.gz 
sudo cp camera_overrides.isp /var/nvidia/nvcam/settings/
sudo chmod 664 /var/nvidia/nvcam/settings/camera_overrides.isp
sudo chown root:root /var/nvidia/nvcam/settings/camera_overrides.isp

Selection Guide

Product Pixels Sensor Dual Len Infrared Angle of View (Diagonal)(degree) Afterture (F) Focal Length (MM) PCBA Suit for
RPi Camera 8 Mega IMX219 × × 62.2 2.0 3.04 Raspberry Pi, Jetson Nano
RPi NoIR Camera V2 62.2 2.0 3.04 Raspberry Pi, Jetson Nano
IMX219-77 Camera × 77 2.0 2.96 Jetson Nano, CM3/3+
IMX219-77IR 77 2.0 2.96 Jetson Nano, CM3/3+
IMX219-120 Camera × 120 2.2 1.88 Jetson Nano, CM3/3+
IMX219-160 Camera × 160 2.35 3.15 Jetson Nano, CM3/3+
IMX219-160IR Camera 160 2.35 3.15 Jetson Nano, CM3/3+
IMX219-170 Camera × 170 2.0 0.87 Jetson Nano, CM3/3+
IMX219-200 Camera × 200 2.0 3.15 Jetson Nano, CM3/3+
IMX219-D160 × 160 2.35 3.15 × Raspberry Pi, Jetson Nano
IMX219-83 Stereo Camera × 83 NA 2.6 Jetson Nano, CM3/3+

Test with Compute Module

The IMX219 series can be used with Compute Module series of Raspberry Pi with carriers. And the usage is the same as other Raspberry Pi cameras
Because of the encryption, the IMX219 series cameras cannot be used with a Raspberry Pi board like the Raspberry Pi 4B, please note it.

About the Libcamera

Raspberry Pi is transitioning from a legacy camera software stack based on proprietary Broadcom GPU code to an open-source stack based on libcamera. Raspberry Pi OS images from Bullseye onwards will contain only the libcamera-based stack. If you use the newest Bullseye Raspberry Pi OS, the libcamera is pre-installed. If you want to use the legacy camera stack (Raspicam), please move to the Raspicam guide.

Installation (optional)

Open a terminal and run the following command:

sudo apt install libcamera-apps

Camera configuration

If you use the official V1 camera (ov5647), V2 camera (IMX219), and the HQ camera (IMX477), you can just plug in the camera and play.
If you use third-party cameras, the support camera size is as below:

Camera Sensor Supported Board Configuration
OV5647 A full range of Raspberry Pi boards Default setting or
dtoverlay=ov5647
IMX219 Compute Module series only default setting or
dtoverlay=imx219
IMX290 and IMX327 A full range of Raspberry Pi Boards dtoverlay=imx290,clock-frequency=74250000 or
dtoverlay=imx290,clock-frequency=37125000
IMX378 A full range of Raspberry Pi boards dtoverlay=imx378
IMX477 Compute Module series only default setting or
dtoverlay=imx477
OV9281 A full range of Raspberry Pi boards dtoverlay=ov9281

To override the automatic camera detection, Bullseye users will also need to delete the entry or change it to value 0.

camera_auto_detect=1

Using libcamera

The libcamera software stack provides six applications/commands for using cameras.

Command Description
libcamera-hello "Hello world" application for previewing camera
libcamera-jpeg A simple still image capture application
libcamera-still Still image capture application which feature more funtsion then libcamera-jpeg
libcamera-vid A video capture application
libcamera-raw A raw Bayer frames recording application
libcamera-detect -

libcamera-hello

This is a "hello world" application for the camera, it starts the camera and displays a preview window.

example

libcamera-hello -t 0

This command should dispay a preview window for about 5 seconds. The -t <duration> option lets the user select how long the window is displayed. if you set the duration value to 0, it runs the preview indefinitely.
The preview can be halted either by clicking the window’s close button or using Ctrl-C in the terminal.

The Tuning File

Raspberry Pi’s libcamera implementation includes a tuning file for each different type of camera module. This is a file that describes or "tunes" the parameters that will be passed to the algorithms and hardware to produce the best image quality. libcamera is only able to determine automatically the image sensor being used, not the module as a whole - even though the whole module affects the "tuning".
For example, the NOIR (no IR-filter) versions of sensors require different AWB settings to the standard versions, so the IMX219 NOIR should be run using.

libcamera-hello --tuning-file /usr/share/libcamera/ipa/raspberrypi/imx219_noir.json

Uses can copy an existing tuning file and alter it according to their own preferences.
The --tuning-file parameter applies identically across all the libcamera applications.

Preview Window

Most of the libcamera applications display a preview image in a window, users can use --info-text parameter to display certain helpful image information on the window title bar using "% directives".
For examples:

libcamera-hello --info-text "focus %focus"

For more information about the option and parameters, please refer to the Command Options part below.

libcamera-jpeg

libcamera-jpeg is a simple still image capture application. It deliberately avoids some of the additional features of libcamera-still which attempts to emulate raspistill more fully. As such the code is significantly easier to understand, and in practice still provides many of the same features.

To capture a full resolution JPEG image use

libcamera-jpeg -o test.jpg

This command will display a preview for about 5s and then acpture a full resolution JPEG image to the file test.jpg.
The -t <duration> option can be used to alter the length of time the preview shows, and the --width and --height options will change the resolution of the captured still image. For example.

libcamera-jpeg -o test.jpg -t 2000 --width 640 --height 480

Exposure Control

All the libcamera applications allow the user to run the camera with fixed shutter speed and gain. for example:

libcamera-jpeg -o test.jpg -t 2000 --shutter 20000 --gain 1.5

This comamnd would capture an image with an exposure of 20ms and a gain of 1.5x. The the gain will be applied as analogue gain within the sensor up until it reaches the maximum analogue gain permitted by the kernel sensor driver, after which the remainder will be applied as digital gain.
Note: Digital gain is applied by the ISP, note by the sensor. The digital gain will always be very close to 1.p unless:

  1. The total gain requested (either by the --gain option, or by the exposure profile in the camera tuning) exceeds that which can be applied as analogue gain within the sensor. Only the extra gain required will be applied as digital gain.
  2. One of the colour gains is less than 1 (note that colour gains are applied as digital gain too). In this case the advertised digital gain will settle to 1 / min(red_gain, blue_gain). This actually means that one of the colour channels - just not the green one - is having unity digital gain applied to it.
  3. The AEC/AGC is changing. When the AEC/AGC is moving the digital gain will typically vary to some extent to try and smooth out any fluctuations, but it will quickly settle back to its "normal" value.

Raspberry Pi’s AEC/AGC algorithm allows applications to specify exposure compensation, that is, the ability to make images darker or brighter by a given number of stops. For example:

libcamera-jpeg --ev -0.5 -o darker.jpg
libcamera-jpeg --ev 0 -o normal.jpg
libcamera-jpeg --ev 0.5 -o brighter.jpg

libcamera-still

libcamera-still is very similar to libcamera-jpeg but supports more of the legacy raspistill options

example

libcamera-still -o test.jpg

Encoders

libcamera-still allows files to be saved in a number of different formats. It supports both png and bmp encoding. It also allows files to be saved as a binary dump of RGB or YUV pixels with no encoding or file format at all. In these latter cases, the application reading the files will have to understand the pixel arrangement for itself.

libcamera-still -e png -o test.png
libcamera-still -e bmp -o test.bmp
libcamera-still -e rgb -o test.data
libcamera-still -e yuv420 -o test.data

Note that the format in which the image is saved depends on the -e option and is not selected automatically based on the output file name.

Raw Image Capture

Raw images are the images produced directly by the image sensor before any processing is applied to them either by the ISP or any of the CPU cores. For colour image sensors these are usually Bayer format images. Note that raw images are quite different from the processed but unencoded RGB or YUV images that we saw earlier.

libcamera-still -r -o test.jpg

These DNG files contain metadata pertaining to the image capture, including black levels, white balance information and the colour matrix used by the ISP to produce the JPEG. This makes these DNG files much more convenient for later "by hand" raw conversion with some of the aforementioned tools. Using exiftool shows all the metadata encoded into the DNG file:

File Name                       : test.dng
Directory                       : .
File Size                       : 24 MB
File Modification Date/Time     : 2021:08:17 16:36:18+01:00
File Access Date/Time           : 2021:08:17 16:36:18+01:00
File Inode Change Date/Time     : 2021:08:17 16:36:18+01:00
File Permissions                : rw-r--r--
File Type                       : DNG
File Type Extension             : dng
MIME Type                       : image/x-adobe-dng
Exif Byte Order                 : Little-endian (Intel, II)
Make                            : Raspberry Pi
Camera Model Name               : /base/soc/i2c0mux/[email protected]/[email protected]
Orientation                     : Horizontal (normal)
Software                        : libcamera-still
Subfile Type                    : Full-resolution Image
Image Width                     : 4056
Image Height                    : 3040
Bits Per Sample                 : 16
Compression                     : Uncompressed
Photometric Interpretation      : Color Filter Array
Samples Per Pixel               : 1
Planar Configuration            : Chunky
CFA Repeat Pattern Dim          : 2 2
CFA Pattern 2                   : 2 1 1 0
Black Level Repeat Dim          : 2 2
Black Level                     : 256 256 256 256
White Level                     : 4095
DNG Version                     : 1.1.0.0
DNG Backward Version            : 1.0.0.0
Unique Camera Model             : /base/soc/i2c0mux/[email protected]/[email protected]
Color Matrix 1                  : 0.8545269369 -0.2382823821 -0.09044229197 -0.1890484985 1.063961506 0.1062747385 -0.01334283455 0.1440163847 0.2593136724
As Shot Neutral                 : 0.4754476844 1 0.413686484
Calibration Illuminant 1        : D65
Strip Offsets                   : 0
Strip Byte Counts               : 0
Exposure Time                   : 1/20
ISO                             : 400
CFA Pattern                     : [Blue,Green][Green,Red]
Image Size                      : 4056x3040
Megapixels                      : 12.3
Shutter Speed                   : 1/20

Very long exposures

To capture very long exposure images, we need to be careful to disable the AEC/AGC and AWB because these algorithms will otherwise force the user to wait for a number of frames while they converge. The way to disable them is to supply explicit values. Additionally, the entire preview phase of the capture can be skipped with the --immediate option.
Capature image and set the exposure time to 100s.

libcamera-still -o long_exposure.jpg --shutter 100000000 --gain 1 --awbgains 1,1 --immediate

The maximum exposure times of the three official Raspberry Pi cameras are given in the table below.

Module Max exposure (seconds)
OV5647 6
IMX219 10
IMX417 230

libcamera-vid

libcamera-vid is the video capture application. By default, it uses the Raspberry Pi’s hardware H.264 encoder. It will display a preview window and write the encoded bitstream to the specified output.
For example, write a 10s video to file:

libcamera-vid -t 10000 -o test.h264

You can use vlc to play the file:

vlc test.h264

Note that this is an unpackaged video bitstream, it is not wrapped in any kind of container format (such as an mp4 file). The --save-pts option can be used to output frame timestamps so that the bitstream can subsequently be converted into an appropriate format

libcamera-vid -o test.h264 --save-pts timestamps.txt

For outputing mkv file:

mkvmerge -o test.mkv --timecodes 0:timestamps.txt test.h264

Encoders

There is support for motion JPEG, and also for uncompressed and unformatted YUV420

libcamera-vid -t 10000 --codec mjpeg -o test.mjpeg
libcamera-vid -t 10000 --codec yuv420 -o test.data

The --codec parameter determines the output format, not the extension of the output file.
The --segment parameter breaks output files up into chunks of the segment size (given in milliseconds). This is quite handy for breaking a motion JPEG stream up into individual JPEG files by specifying very short (1 millisecond) segments

libcamera-vid -t 10000 --codec mjpeg --segment 1 -o test%05d.jpeg

libcamera-raw

libcamera-raw is like a video recording application except that it records raw Bayer frames directly from the sensor. It does not show a preview window. For a 2 second raw clip use

libcamera-raw -t 2000 -o test.raw

The raw frames are dumped with no formatting information at all, one directly after another. The application prints the pixel format and image dimensions to the terminal window so that the user can know how to interpret the pixel data.
By default the raw frames are saved in a single (potentially very large) file. As we saw previously, the --segement option can be used conveniently to direct each to a separate file.

libcamera-raw -t 2000 --segment 1 -o test%05d.raw

In good conditions (using a fast SSD) libcamera-raw can get close to writing 12MP HQ camera frames (18MB of data each) to disk at 10 frames per second. It writes the raw frames with no formatting in order to achieve these speeds; it has no capability to save them as DNG files (like libcamera-still). If you want to be sure not to drop frames you could reduce the framerate slightly using the --framerate option, for example.

libcamera-raw -t 5000 --width 4056 --height 3040 -o test.raw --framerate 8

Common Command Line Options

Option Option (abbreviate) Description
--help -h Print help information for the application
--version Print out a software version number
--list-cameras List the cameras available for use
--camera Selects which camera to use <index>
--config -c Read options from the given file <filename>
--timeout -t Delay before application stops automatically <milliseconds>
--preview -p Preview window settings <x,y,w,h>
--fullscreen -f Fullscreen preview mode
--qt-preview Use Qt-based preview window
--nopreview Set window title bar text <string>
--width Capture image width <width>
--height Capture image height <height>
--viewfinder-width Capture image width <width>
--viewfinder-height Capture image height <height>
--rawfull Force sensor to capture in full resolution mode
--mode Specify sensor mode, given as <width>:<height>:<bit-depth>:<packing>
--viewfinder-mode Specify sensor mode, given as <width>:<height>:<bit-depth>:<packing>
--lores-width Low resolution image width <width>
--lores-height Low resolution image height <height>
--hflip
--vflip
--rotation
Read out with horizontal mirror
Read out with vertical flip
Use hflip and vflip to create the given rotation <angle>
--roi Select a crop (region of interest) from the camera <x,y,w,h>
--sharpness Set image sharpness <number>
--contrast Set image contrast <number>
--brightness Set image brightness <number>
--saturation Set image colour saturation <number>
--ev Set EV compensation <number>
--shutter Set the exposure time in microseconds <number>
--gain
--analoggain
Sets the combined analogue and digital gains <number>
Synonym for --gain
--metering Set the metering mode <string>
--exposure Set the exposure profile <string>
--awb Set the AWB mode <string>
--awbgains Set fixed colour gains <number,number>
--denoise Set the denoising mode <string>
--tuning-file Specify the camera tuning to use <string>
--output -o Output file name <string>
--wrap Wrap output file counter at <number>
--flush Flush output files immediately
Parameter for --info-text option
Directive Substitution
%frame The sequence number of the frame
%fps The instantaneous frame rate
%exp The shutter speed used to capture the image, in microseconds
%ag The analogue gain applied to the image in the sensor
%dg The digital gain applied to the image by the ISP
%rg The gain applied to the red component of each pixel
%bg The gain applied to the blue component of each pixel
%focus The focus metric for the image, where a larger value implies a sharper image
Parameter for --awb option
Mode name Colour temperature
auto 2500K to 8000K
incandescent 2500K to 3000K
tungsten 3000K to 3500K
fluorescent 4000K to 4700K
indoor 3000K to 5000K
daylight 5500K to 5400K
cloudy 7000K to 8500K

Still Command Line Options

Option Option (abbreviate) Description
--quality -q JPEG quality <number>
--exif -x Add extra EXIF tags <string>
--timelapse Time interval between timelapse captures <milliseconds>
--framestart The starting value for the frame counter <number>
--datetime Use date format for the output file names
--timestamp Use system timestamps for the output file names
--restart Set the JPEG restart interval <number>
--keypress -k Capture image when Enter pressed
--signal -s Capture image when SIGUSR1 received
--thumb Set thumbnail parameters <w:h:q> or none
--encoding -e Set the still image codec <string>
  • jpg - JPEG (the default)
  • png - PNG format
  • bmp - BMP format
  • rgb - binary dump of uncompressed RGB pixels
  • yuv420 - binary dump of uncompressed YUV420 pixels.
--raw -r Save the raw file
--latest Make a symbolic link to the latest file saved <string>

Video Command Line Options

Option Option (abbreviate) Description
--quality -q JPEG quality <number>
--bitrate -b H.264 bitrate <number>
--intra -g Intra-frame period (H.264 only) <number>
--profile H.264 profile <string>
--level H.264 level <string>
--codec Encoder to be used <string>
  • h264 - use H.264 encoder (the default)
  • mjpeg - use MJPEG encoder
  • yuv420 - output uncompressed YUV420 frames
--keypress -k Toggle between recording and pausing
--signal -s Toggle between recording and pausing when SIGUSR1 received
--initial Start the application in the recording or paused state <string>
--split Split multiple recordings into separate files
--segment Write the video recording into multiple segments <number>
--circular Write the video recording into a circular buffer of the given <size>
--inline Write sequence header in every I frame (H.264 only)
--listen Wait for an incoming TCP connection
--frames Record exactly this many frames <number>

For more information about the libcamera, please refer to Official Document

FAQ

 Answer:
The cheese tool could only work with the USB camera. The IMX219 series are CSI camera. Please use the provided test command.
{{{4}}}

{{{5}}}


 Answer:

The CSI camera could work with the Gstreamer pipeline, please check the calling method.

{{{4}}}

{{{5}}}


Support

If you require technical support, please go to the Support page and open a tickets.