Different FPS When Using V4L2 and Gst-launch with Nvidia Jetson Orin Nano Dev Board

Issue Overview

Users of the Nvidia Jetson Orin Nano Dev Board have reported discrepancies in frame rates when capturing video using different methods: V4L2 (Video for Linux 2) and GStreamer with the nvarguscamera plugin. The primary symptoms include:

• V4L2 Command: Users can achieve frame rates close to the expected maximum (e.g., 238 fps for sensor mode 4).

• GStreamer Command: When using gst-launch-1.0 with nvarguscamerasrc, users are only able to achieve frame rates significantly lower than expected (e.g., below 110 fps).

The issue occurs during video capture, specifically when trying to utilize high frame rate sensor modes (e.g., 240 fps). The board is running JetPack version 5.1.1, and users have attempted to upgrade to 5.1.2 with no improvement in results.

The impact of this problem is significant, as it affects the usability of high-frame-rate sensors in applications requiring real-time video processing. Users have expressed frustration due to the limitations encountered when using the GStreamer pipeline compared to V4L2.

Possible Causes

Several potential causes for the observed discrepancies in frame rates have been identified:

• Hardware Limitations: The Orin Nano may not fully support high frame rates when utilizing certain plugins or configurations.

• Software Bugs or Conflicts: There may be underlying issues within the GStreamer nvarguscamera plugin that prevent it from achieving optimal performance.

• Configuration Errors: Incorrect settings in the GStreamer pipeline could lead to suboptimal performance.

• Driver Issues: The drivers associated with the camera sensor and the Orin Nano might not be optimized for high frame rates.

• Clock Rate Settings: The clock rates for various components (VI, CSI, ISP) may not be set high enough to support maximum performance.

• Environmental Factors: Power supply issues or thermal throttling could also affect performance.

Troubleshooting Steps, Solutions & Fixes

To address the issue of varying frame rates, users can follow these troubleshooting steps and solutions:

1. Check Current Frame Rates:

   • Use the command:

     v4l2-ctl --list-formats-ext -d /dev/video1
     

   • This will provide details about supported formats and frame rates.

2. Boost Clock Rates:

   • Execute the following commands to increase clock rates:

     sudo su
     echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
     echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
     echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
     echo 1 > /sys/kernel/debug/bpmp/debug/clk/emc/mrq_rate_locked
     

   • Check maximum clock rates:

     cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
     

3. Run GStreamer Pipeline with Adjustments:

   • Test different configurations in your GStreamer pipeline:

     gst-launch-1.0 nvarguscamerasrc sensor-id=1 ! "video/x-raw(memory:NVMM), width=(int)2028, height=(int)1112, format=(string)NV12, framerate=(fraction)240/1" ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420' ! queue ! fpsdisplaysink text-overlay=false video-sink=fakesink --verbose
     

   • This modification has been reported to yield higher frame rates (over 170 fps).

4. Use Clock.sh Script:

   • Execute a provided script (clock.sh) that boosts additional clocks, including VIC, which has shown to improve performance significantly.
   • After running the script, re-test your GStreamer command.

5. Test Different Sensor Modes:

   • Experiment with other sensor modes such as 2028x1520@130fps to determine if they yield better results with nvarguscamerasrc.

6. Monitor Performance Metrics:

   • Use tools like nvidia-smi or other monitoring utilities to check for any bottlenecks in GPU or CPU usage during video capture.

7. Consult Documentation and Updates:

   • Ensure that you are using the latest drivers and firmware updates for both the Jetson board and camera sensors.
   • Review Nvidia’s documentation for any known issues or updates related to high-frame-rate video capture.

8. Contact Support:

   • If issues persist, consider reaching out to Nvidia support or community forums for assistance, especially if you suspect hardware limitations.

By following these steps, users can effectively troubleshoot and potentially resolve discrepancies in frame rates between V4L2 and GStreamer on the Nvidia Jetson Orin Nano Dev Board.