Quad IMX390 Camera Support Issues on Nvidia Jetson Orin Nano

Issue Overview

Users of the Nvidia Jetson Orin Nano Dev board have reported issues related to the support of multiple IMX390 cameras. The specific symptoms include:

• Frame Jamming: When attempting to use four IMX390 cameras simultaneously, users experience frame jamming, leading to a lack of performance.
• Performance Degradation: While a single camera operates normally, adding a second camera results in dropped frames, and performance further deteriorates with three or four cameras.
• Configuration Confusion: Users are unsure about the correct configuration settings for pixel clock rates and whether the current hardware supports the required bandwidth for multiple cameras.

This issue primarily occurs during the setup and operation phases when users attempt to stream video from multiple cameras using commands like nvgstcapture and gst-launch-1.0. The hardware in question is the Jetson Orin Nano with 4GB RAM running R35.1. The frequency of these issues appears to be consistent among users attempting similar configurations, significantly impacting their ability to utilize the device for multi-camera applications effectively.

Possible Causes

Several potential causes have been identified for the issues experienced:

• Hardware Limitations: The nvcsi_max_rate of 214M may not support the bandwidth required for four IMX390 cameras, which collectively need approximately 222.75M.

• Software Bugs or Conflicts: Users have noted that the nvgstcapture tool seems to be limited by single-core performance, leading to CPU bottlenecks when multiple streams are attempted.

• Configuration Errors: There may be misunderstandings regarding pixel clock configurations and how they relate to the number of lanes used in data transmission.

• Driver Issues: Potential bugs in the drivers or firmware could be affecting performance, especially when handling multiple video streams.

• Environmental Factors: Power supply issues or overheating could lead to throttling, impacting performance when multiple devices are in use.

• User Misconfigurations: Incorrect settings in commands or system parameters may lead to suboptimal performance.

Troubleshooting Steps, Solutions & Fixes

To address these issues, users can follow these troubleshooting steps and solutions:

1. Verify Configuration Settings:

   • Ensure that pixel clock settings are correctly configured. For example:

     pix_clk_hz = 148500000 / 2
     

2. Test Individual Cameras:

   • Start by testing each camera individually using:

     nvgstcapture --sensor-id=0
     

   • Confirm that each camera operates at 30fps without issues.

3. Monitor CPU Usage:

   • Use tools like top or htop to monitor CPU usage when running multiple streams. High CPU usage on a single core may indicate a bottleneck.

4. Adjust Clock Rates:

   • Boost clock rates to maximize performance by executing:

     sudo jetson_clocks
     echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
     echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
     echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
     echo 1 > /sys/kernel/debug/bpmp/debug/clk/emc/mrq_rate_locked
     

5. Use Alternative Tools:

   • If nvgstcapture shows poor performance, consider using v4l2 with OpenCV for better efficiency:

     v4l2-ctl --list-devices
     

   • This method reportedly reduces CPU/GPU usage as it bypasses certain processing pipelines.

6. Test Different Configurations:

   • Experiment with different numbers of cameras (e.g., start with two) and monitor performance changes.

7. Check for Driver Updates:

   • Ensure that all drivers and firmware are up-to-date to avoid known bugs affecting performance.

8. Review Environmental Conditions:

   • Ensure adequate power supply and cooling solutions are in place to prevent thermal throttling during operation.

9. Investigate Kernel Settings:

   • If issues persist, consider whether kernel dynamic debugging settings might be affecting performance.

10. Document Findings:

    • Keep track of configurations that yield better results for future reference and community sharing.

By following these steps, users can systematically diagnose and potentially resolve the issues related to quad IMX390 camera support on the Nvidia Jetson Orin Nano Dev board.