The encoding capability of Orin Nano at 1080p30 may impact the real-time capture of a 1080p60 USB camera

Issue Overview

Users are experiencing limitations with the Nvidia Jetson Orin Nano Developer Kit regarding its video encoding capabilities. Specifically, when connecting a USB camera that supports 1080p60 frame rates, users have reported that the captured video is constrained to a maximum of 1080p30 due to the Orin Nano’s encoding limitations. This issue arises during real-time image capture using OpenCV’s cv2.VideoCapture() function. The lack of a hardware encoder in the Orin Nano necessitates the use of a software encoder, which caps the performance and results in lower frame rates. Users have noted that even though their USB cameras can generate frames at 1080p60, the encoded output cannot exceed 1080p30. This limitation significantly impacts applications requiring high frame rates and real-time processing, especially in scenarios where multiple video feeds are involved.

Possible Causes

1. Hardware Limitations: The Orin Nano lacks a dedicated hardware encoder, relying instead on software encoding, which restricts performance.
2. Software Encoding Constraints: The software encoder’s capability limits the maximum output frame rate to 1080p30, regardless of the input frame rate from connected cameras.
3. Configuration Errors: Incorrect settings in OpenCV or GStreamer pipelines may exacerbate performance issues.
4. Driver Issues: Outdated or incompatible drivers could hinder optimal performance, affecting video capture and encoding.
5. Environmental Factors: Factors such as power supply inconsistencies or overheating could impact performance.
6. User Misconfigurations: Improper setup or configuration of OpenCV parameters may lead to unexpected behavior during video capture.

Troubleshooting Steps, Solutions & Fixes

1. Diagnosing the Problem:

   • Verify camera capabilities by checking specifications to ensure it supports 1080p60.
   • Test the camera with different software configurations to determine if the issue is consistent across setups.

2. Gathering System Information:

   • Use terminal commands to check connected devices:

     v4l2-ctl --list-devices
     

   • Inspect current OpenCV settings:

     import cv2
     print(cv2.getBuildInformation())
     

3. Isolating the Issue:

   • Connect only one USB camera and test video capture to see if performance improves.
   • Change settings in cv2.VideoCapture() to ensure proper configuration:

     capture.set(cv2.CAP_PROP_FRAME_WIDTH, 1920)
     capture.set(cv2.CAP_PROP_FRAME_HEIGHT, 1080)
     

4. Potential Fixes:

   • If higher frame rates are crucial, consider upgrading to a more capable device such as the Nvidia Xavier NX or Orin NX, which include hardware encoders for better performance.
   • Implement GStreamer pipelines for more efficient handling of video streams:

     gst-launch-1.0 v4l2src ! videoconvert ! x264enc ! matroskamux ! filesink location=output.mkv
     

5. Documentation and Updates:

   • Ensure all software and drivers are up-to-date by checking Nvidia’s official documentation for JetPack updates.
   • Refer to Nvidia’s developer forums for additional troubleshooting tips and community support.

6. Best Practices:

   • Regularly monitor system temperatures and power supply stability to prevent thermal throttling.
   • When multiple cameras are used, ensure that their combined bandwidth does not exceed USB bus limits.

7. Unresolved Aspects:

   • Further investigation may be needed into optimizing software encoders or exploring alternative libraries that may offer better performance on the Orin Nano.

By following these steps and solutions, users can better understand and potentially mitigate the limitations imposed by the Nvidia Jetson Orin Nano’s encoding capabilities when working with high-frame-rate video applications.