Template: IMX462 2MP Starlight Camera Jetson Nano

From Waveshare Wiki
Revision as of 08:39, 28 September 2022 by Waveshare-eng11 (talk | contribs) (→‎Driver Installation)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Working with Jetson Nano

Version Statement

The IMX462 camera is used on the NVIDIA platform to install the relevant driver and load the new device tree file. Currently supports Jetson Nano, Jetson Xavier NX, Jetson TX2 NX, and Jetson AGX Xavier.
Note: AGX Xavier and other interfaces are packaged differently, and you need to purchase or make an adapter board support them.
Currently supported Jetapck versions:

  • Jetpack4.2.2, L4T version r32.2.1
  • Jetpack4.3, L4T version r32.3.1
  • Jetpack4.4, L4T version r32.4.3
  • Jetpack4.4.1, L4T version r32.4.4
  • Jetpack4.5, L4T version r32.5
  • Jetpack4.5.1, L4T version r32.5.1
  • Jetpack4.6, L4T version r32.6.1
  • Jetpack4.6.1, L4T version r32.7.1
  • Jetpack4.6.2, L4T version r32.7.2
  • Jetpack5.0.1DP, L4T version r34.1.1

Check JetPack Version

  • If you are not sure whether your software version is supported, you can use the following command to check.
cat /etc/nv_tegra_release
  • If it shows R32 (release), REVISION: 4.3 means the current version is R32.4.3.

Hardware Connection

  • Please arrange the camera cable, with the gold finger contact surface facing the side of the core board, and connect it to the jetson development board.

IMX462 2MP Starlight Camera Jetson Nano.jpg

  • Jetpack4.x Version

To use the IMX462 camera, the Image and DTB parts of the Jetson system need to be updated. In the Image part, the camera driver is added, and the dtb part indicates the camera model used. In general, you only need to use the compiled Image and dtb, and you don't need to compile it according to the code if it is not necessary.

  • JetPack 5.0 or later (for Xavier NX and AGX Xavier and higher performance modules)

The driver is no longer compiled into Image, but loaded into the system as an independent module. dtb is still compiled as a whole to facilitate replacement. dtb can also be dynamically loaded as an overlay.

BSP Package

  • Download path: https://github.com/veyeimaging/nvidia_jetson_veye_bsp
  • BSP package:
    • Precompiled dtb for different platforms
    • driver source code
    • dts source code
    • i2c communication toolset
  • The correspondence between DTB package files and boards:
    • Nano B01
    tegra210-p3448-0000-p3449-0000-b00.dtb
    • AGX XAVIER
    tegra194-p2888-0001-p2822-0000.dtb
    • XAVIER NX
    tegra194-p3668-all-p3509-0000.dtb
    (Jetpack5.0.1) tegra194-p3668-0000-p3509-0000.dtb
    • AGX Orin
    tegra234-p3701-0000-p3737-0000.dtb

Driver Installation

Upgrade directly on the jetson board:

wget https://github.com/veyeimaging/nvidia_jetson_veye_bsp/releases/latest/download/nvidia_jetson_veye_bsp.tar.gz
tar -xzvf nvidia_jetson_veye_bsp.tar.gz

Upgrade Image (for JetPack4.x)

  • Back up the original Image
cp /boot/Image /boot/Image.backup
cp /boot/Image.sig /boot/Image.sig.backup
  • (Jetson Nano) In the downloaded bsp package, find the corresponding Image compressed package, decompress it, and then execute the command.
sudo cp <path to your Image dir>/Image /boot/Image -f

Note here to replace <path to your Image dir> with the path of the Image file you actually unzipped.

  • (NX) If you are using Xavier NX, you need to use the signed version, that is, the package with the word signed.
sudo cp <path to your Image dir>/Image /boot/Image -f
sudo cp <path to your Image dir>/Image.sig /boot/Image.sig -f

Module Installation (For JetPack5.x)

  • Check Linux version
uname -r
  • Find the corresponding driver version directory in the nvidia_jetson_veye_bsp/ko directory, and run the command:
sudo cp nvidia_jetson_veye_bsp/ko/$(uname -r)/* /lib/modules/$(uname -r)/kernel/drivers/media/i2c/ -f
sudo depmod

Upgrade dtb

  • In the bsp package directory, find the dtb file corresponding to the camera (VEYE-MIPI-CAM2M) model and L4T version number. Copy:
sudo mkdir /boot/veyecam/
sudo cp <path to your dtb dir>/<DTB file name> /boot/veyecam -f
  • Backup the extlinux.conf file.
cp /boot/extlinux/extlinux.conf /boot/extlinux/extlinux.conf.back
  • Edit the /boot/extlinux/extlinux.conf file and add the following line at the bottom:
FDT /boot/veyecam/<DTB file name>

Note: <DTB file name> should be replaced with the dtb name corresponding to the motherboard, to ensure that the file indicated by the FDT line does exist correctly.

  • Reboot the system:
sudo reboot

Camera Test

Check if the camera is connected properly

  • After the system upgrade is successful, you can enter the following command in the Jetson nano terminal to check whether the camera is connected normally.
dmesg | grep veye

There are prompts such as IMX462. Also view ls /dev/ there are videox device nodes.

Test Camera (Gstreamer)

  • Video preview 1080P
export DISPLAY=:0
gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)NV12" ! nvoverlaysink sync=false
gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvoverlaysink sync=false
  • Video preview 1080p (xvimagesink)
export DISPLAY=:0
gst-launch-1.0 -e v4l2src io-mode=4 device=/dev/video0 do-timestamp=true ! 'video/x-raw, width=1920, height=1080, framerate=30/1, format=UYVY' ! xvimagesink sync=false
  • Preview two cameras at the same time
WIDTH=960
HEIGHT=540

CAPS="video/x-raw(memory:NVMM),format=(string)UYVY, width=1920, height=1080"

gst-launch-1.0 nvcompositor name=comp sink_0::xpos=0 sink_0::ypos=0 sink_0::width=$WIDTH sink_0::height=$HEIGHT sink_1::xpos=$WIDTH sink_1::ypos=0 sink_1::width=$WIDTH sink_1::height=$HEIGHT ! nvoverlaysink nvv4l2camerasrc device=/dev/video0 ! $CAPS ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420"! comp. nvv4l2camerasrc device=/dev/video1 ! $CAPS ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420"! comp
  • Calling Gstreamer with Opencv
gst-launch-1.0 nvv4l2camerasrc ! video/x-raw(memory:NVMM), format=(string)UYVY, width=(int)1920, height=(int)1080 ! nvvidconv ! video/x-raw(memory:NVMM), format=(string)I420 ! nvvidconv ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink
  • Video recording 1080p
gst-launch-1.0 nvv4l2camerasrc num-buffers=300 ! "video/x-raw(memory:NVMM),format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)NV12" ! nvv4l2h264enc control-rate=1 bitrate=10000000 ! h264parse ! qtmux ! filesink location=filename.mp4 -e
  • Play video
gst-launch-1.0 filesrc location=filename.mp4 ! qtdemux ! queue ! h264parse ! nvv4l2decoder ! nvoverlaysink -e
  • Snap a picture
gst-launch-1.0 v4l2src num-buffers=1 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvjpegenc ! filesink location=jpgname.jpg

Test camera on Jetpack5.x system

  • Video preview 1080p
gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,format=(string)UYVY,width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)NV12" ! nv3dsink -e
  • Video recording 1080p
gst-launch-1.0 v4l2src num-buffers=300 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)NV12" ! nvv4l2h264enc control-rate=1 bitrate=10000000 ! h264parse ! qtmux ! filesink location=filename.mp4 -e
  • Snap image
gst-launch-1.0 v4l2src num-buffers=1 ! "video/x-raw,format=(string)UYVY, width=(int)1920, height=(int)1080" ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string)I420" ! nvjpegenc ! filesink location=jpgname.jpg

Test Camera(v4l2-ctl)

  • Install v4l2-utils
sudo apt-get install v4l-utils
  • Check the data formats supported by the camera.
v4l2-ctl --list-formats-ext
  • Frame rate statistics
v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=UYVY--stream-mmap --stream-count=-1 --stream-to=/dev/null
  • Save image to file
v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=UYVY --stream-mmap --stream-count=1 --stream-to=uyvy-1920x1080.yuv

Test Camera (yavta)

  • Install yavta
git clone https://github.com/veyeimaging/yavta.git
cd yavta
sudo make
  • Save image to file
./yavta -c1 -FUYVY-1920x1080.yuv --skip 0 -f UYVY -s 1920x1080 /dev/video0

Opencv Demo