Nvidia Jetson Orin Nano Dev Board: Troubleshooting Low Frame Rate with CSI Raw Sensor

Issue Overview

Users are experiencing lower than expected frame rates when using a CSI raw sensor (IMX459) with the Nvidia Jetson Orin Nano Dev Board. The sensor is capable of 1314×12@3000fps, but users are only achieving around 600fps with the v4l2 driver. Increasing the sensor output to 1000fps allows v4l2 to successfully capture at that frame rate. However, when attempting to capture at the sensor’s maximum of 3000fps, the frame rate is limited to approximately 1500fps even after boosting clocks on the Orin Nano.

Possible Causes

There are several potential factors contributing to the frame rate limitation:

• Sensor Limitations: The specific frame rate capabilities of the IMX459 sensor are not well-documented, so it’s unclear if it can consistently output 3000fps in this configuration.

• v4l2 Driver: The v4l2 driver settings and capabilities can impact the achievable frame rate. Misconfiguration or driver limitations may restrict the frame rate.

• Jetson Orin Nano Hardware Constraints:

  • The Orin Nano has a maximum of 4 CSI lanes, each capable of 1.5 Gbps, providing a theoretical maximum bandwidth of 6 Gbps. The high data rate required for 3000fps may exceed this limit.
  • Memory speed and performance on the Orin Nano may create a bottleneck when attempting to capture and process such high frame rates.

• Clocking and Power Settings: The various clock frequencies for the ISP, CSI, and VI components, as well as the EMC clock, need to be optimized and potentially boosted to support higher frame rates.

Troubleshooting Steps, Solutions & Fixes

1. Verify Sensor Configuration: Double-check the IMX459 sensor settings and capabilities to ensure it is properly configured for 3000fps output over 4 CSI lanes.

2. Optimize v4l2 Settings: Experiment with different v4l2 configuration options to find the optimal settings for high frame rate capture. This may include buffer sizes, queue sizes, and other parameters.

3. Boost Clocks: Maximize the clock frequencies for the relevant components:

   sudo su
   echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
   echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked 
   echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
   echo 1 > /sys/kernel/debug/bpmp/debug/clk/emc/mrq_rate_locked
   cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
   cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee  /sys/kernel/debug/bpmp/debug/clk/isp/rate
   cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
   cat /sys/kernel/debug/bpmp/debug/clk/emc/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/emc/rate
   

4. Investigate CHANSEL_NOMATCH Errors: The repeated CHANSEL_NOMATCH errors in the CSI trace logs, even at lower frame rates, suggest a potential mismatch between the sensor output and the CSI input. Further investigation into the cause of these errors is recommended.

5. Test on AGX Orin: For comparison, connect the same sensor setup to an AGX Orin dev kit if available, which has higher IO and memory bandwidth than the Orin Nano. If frame rates are still limited, it suggests a more fundamental sensor or driver limitation.

6. Evaluate Argus: Attempt to use the Argus API to capture from the sensor, although this may introduce additional overhead compared to v4l2.

7. Monitor Memory Usage: Collect data on memory usage and throughput during high frame rate capture to assess if the memory subsystem is a bottleneck at 3000fps.

In summary, pushing the Orin Nano to capture from a 3000fps sensor is reaching the limits of the hardware capabilities. Achieving a stable 1500fps after clock boosting is a notable improvement, but further gains may be difficult without more fundamental changes to the hardware, drivers, or sensor configuration.