Delay in processing as the resolution increases

Issue Overview

Users are experiencing significant delays in processing when increasing the resolution of real-time visuals streamed from a stereoscopic camera module using the Nvidia Jetson Orin Nano. The problem manifests when users attempt to upscale the resolution from a lower setting (540×320) to higher resolutions such as 720p, Full HD, and 2K.

The symptoms include:

• Noticeable lag in frame rendering as the resolution increases.
• A decrease in the processing capability of the Jetson Orin Nano, particularly at higher resolutions.
• Users have reported that while lower resolutions (e.g., 540×320) function well at a shuttering frequency of 48Hz, higher resolutions lead to delays.

The issue occurs primarily during the development phase when utilizing OpenCV and Python for streaming applications. Users have noted that this delay impacts their ability to develop effective 3D technology for real-time visual streaming.

Possible Causes

Several potential causes for this issue have been identified:

• Hardware Limitations: The Jetson Orin Nano may not have sufficient processing power or memory bandwidth to handle higher resolutions effectively.

• Software Bugs or Conflicts: There may be bugs in the software stack being used (e.g., OpenCV, JetPack) that affect performance at higher resolutions.

• Configuration Errors: Incorrect settings or configurations in the camera or software could lead to suboptimal performance.

• Driver Issues: Missing or outdated drivers for camera interfaces (e.g., Argus stack for Bayer cameras) may hinder performance.

• Environmental Factors: Power supply issues or thermal throttling could impact processing speeds.

• User Errors: Misconfigurations in code or setup might lead to inefficient processing routines.

Troubleshooting Steps, Solutions & Fixes

To address the delays experienced when increasing resolution, users can follow these troubleshooting steps and potential solutions:

1. Check Camera Type and Configuration:

   • Verify if you are using a Bayer camera (like the Waveshare IMX219-83) or a YUV camera.
   • For Bayer cameras, ensure you are utilizing the Argus stack for hardware ISP engine support.

2. Update Software and Drivers:

   • Ensure you are using the latest version of JetPack. As of now, users reported using JetPack 5.1.2.
   • Check for updates on relevant libraries and drivers:

     sudo apt update
     sudo apt upgrade
     

3. Verify Library Availability:

   • Check if critical libraries such as libargus.so and libnv-eglstream.so are present:

     ls /usr/lib/aarch64-linux-gnu/libargus.so
     ls /usr/lib/aarch64-linux-gnu/tegra-egl/libnv-eglstream.so
     

   • If these libraries are missing, reinstall JetPack or relevant packages.

4. Optimize Code:

   • Review your code for efficiency. Ensure that you are not performing unnecessary computations during frame capture.
   • Consider using multi-threading to handle frame processing separately from frame capture.

5. Test with Different Resolutions:

   • Experiment with intermediate resolutions (e.g., 640×480) to identify at which point performance begins to degrade.
   • This can help isolate whether the issue is strictly related to resolution or other factors.

6. Monitor System Performance:

   • Use system monitoring tools (like htop or nvidia-smi) to observe CPU and GPU usage during operation.
   • Look for bottlenecks that could indicate where delays are occurring.

7. Consider Hardware Upgrades:

   • If consistent delays occur even after optimization, consider whether a more powerful board is necessary for your application needs.

8. Community Support and Documentation:

   • Engage with community forums dedicated to Nvidia Jetson platforms for additional insights and shared experiences.
   • Refer to Nvidia’s official documentation for specific guidance on optimizing performance with your hardware setup.

By following these steps, users can systematically diagnose and potentially resolve the issues related to processing delays at higher resolutions on the Nvidia Jetson Orin Nano.