Cannot Record 2x USB Frame-Synced MJPEG Cameras Simultaneously for Stereo

Issue Overview

Users are experiencing difficulties when attempting to record two USB 2.0 (UVC) MJPEG cameras simultaneously on the Nvidia Jetson Orin Nano Dev board. The primary symptoms include:

• Synchronization Issues: The recorded videos are not synchronized, with noticeable frame drops and inconsistent playback quality.
• Performance Degradation: Unlike similar setups on desktop systems, where the recording works flawlessly, the Orin Nano struggles to maintain a stable frame rate and produces larger file sizes.
• Context: The issue arises during attempts to create a stereo camera setup for robotics applications, using GStreamer commands to composite the video streams from both cameras.
• Hardware/Software Specifications:
  • Cameras: 2x USB 2.0 MJPEG, 1600×1200 @ 60FPS
  • Jetson Orin Nano: Kernel version 5.15.122-tegra
  • GStreamer commands attempted include both standard and Nvidia-accelerated pipelines.
• Frequency: The issue is consistently reproducible when using the specified setup, leading to an unusable video output.
• Impact: The inability to record synchronized video affects the functionality of the robotics application, hindering development and testing.

Possible Causes

Several potential causes for the issue have been identified:

• Hardware Limitations: The Jetson Orin Nano has lower CPU capabilities compared to desktop systems, which may lead to performance bottlenecks when processing multiple high-resolution video streams.
• Driver Issues: Incompatibilities or bugs in the GStreamer plugins or drivers could result in poor performance and synchronization issues.
• Configuration Errors: Incorrect GStreamer pipeline configurations might prevent optimal performance and synchronization.
• Environmental Factors: Power supply issues or thermal throttling could impact performance during intensive tasks like video processing.
• User Errors: Misconfigurations in the GStreamer commands or incorrect camera settings could lead to unsatisfactory results.

Troubleshooting Steps, Solutions & Fixes

To address the issue, users can follow these troubleshooting steps and potential solutions:

1. Test Individual Cameras:

   • Run commands for each camera separately to confirm they can achieve the target frame rate:

     gst-launch-1.0 v4l2src device=/dev/video0 ! image/jpeg,width=1600,framerate=60/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! nv3dsink sync=0
     gst-launch-1.0 v4l2src device=/dev/video2 ! image/jpeg,width=1600,framerate=60/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! nv3dsink sync=0
     

2. Use Hardware Decoding:

   • Ensure that hardware decoding is utilized in your GStreamer pipelines to enhance performance:

     gst-launch-1.0 nvcompositor name=mix sink_0::xpos=0 sink_0::ypos=0 sink_1::xpos=1600 sink_1::ypos=0 ! nvvidconv ! "video/x-raw(memory:NVMM), width=3200, height=1200, framerate=60/1, format=RGBA" ! avimux ! filesink location=output.avi \
     v4l2src device=/dev/video0 ! image/jpeg,width=1600,framerate=60/1 ! nvv4l2decoder mjpeg=1 ! mix.sink_0 \
     v4l2src device=/dev/video2 ! image/jpeg,width=1600,framerate=60/1 ! nvv4l2decoder mjpeg=1 ! mix.sink_1
     

3. Optimize GStreamer Pipeline:

   • Experiment with different configurations in your GStreamer pipeline to find a balance between quality and performance.

4. Check System Resources:

   • Monitor CPU and memory usage during video processing tasks to identify potential bottlenecks.

5. Update Drivers and Firmware:

   • Ensure that all drivers and firmware are up-to-date. Check for any Nvidia-specific updates that might address performance issues.

6. Consider Alternative Hardware:

   • If performance issues persist, consider using a more powerful model like the Jetson Orin NX that may better handle dual camera inputs.

7. Review Documentation and Community Posts:

   • Consult Nvidia’s documentation and community forums for any updates or similar issues reported by other users.

8. Report Findings:

   • If a solution is found that resolves the issue effectively, share this information back on the forum for others facing similar challenges.

By following these steps and utilizing community resources, users can better diagnose and potentially resolve synchronization issues when recording from multiple USB cameras on the Jetson Orin Nano Dev board.