Performance Issue with Stereoscopic Camera on Jetson Orin Nano: Delays in Alternate Frame Display at Higher Resolutions

Issue Overview

The user is experiencing performance issues with a stereoscopic camera module interfaced with the Nvidia Jetson Orin Nano development board. The problem manifests as delays between frame renders when displaying alternate frames from the stereoscopic camera at higher resolutions. Specifically:

• The setup works smoothly at 540×320 resolution
• Performance degrades (delays occur) when increasing resolution to 720p, Full HD, or 2K
• The issue affects an active 3D shutter demo where frames from both cameras are interlaced and displayed alternately
• Pinned array memory is being used for faster GPU-CPU data transfer

Possible Causes

1. Insufficient Memory Bandwidth: Higher resolutions require more data to be processed and transferred, potentially exceeding the Orin Nano’s memory bandwidth capabilities.

2. CPU/GPU Performance Limitations: The increased computational load from higher resolutions may be pushing the Orin Nano’s processing capabilities to their limits.

3. Display Rendering Bottleneck: The issue could be related to the display rendering process rather than data acquisition or processing.

4. Suboptimal Clock Speeds: Default clock speeds for various components (VI, CSI, ISP, EMC) might not be optimized for high-resolution stereoscopic processing.

5. Software Inefficiencies: The code handling the frame alternation and display might not be optimized for higher resolutions.

6. Hardware Limitations: The Orin Nano might lack specific hardware acceleration features needed for efficient high-resolution stereoscopic processing.

Troubleshooting Steps, Solutions & Fixes

1. Boost Component Clock Speeds:
   Try increasing the clock speeds of relevant components to improve performance:

   sudo su
   echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
   echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
   echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
   echo 1 > /sys/kernel/debug/bpmp/debug/clk/emc/mrq_rate_locked
   cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
   cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/isp/rate
   cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
   cat /sys/kernel/debug/bpmp/debug/clk/emc/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/emc/rate
   

   Note: This solution did not resolve the issue for the user.

2. Isolate Display Rendering:
   To determine if the issue is related to display rendering, try encoding the video stream locally without displaying it. This will help isolate whether the problem is in the capture/processing pipeline or the display output.

3. Software Encoding:
   Since the Orin Nano lacks a hardware NVENC engine, implement software encoding. Refer to the Orin Nano Developer Guide for instructions on Software Encode.

4. Optimize Memory Usage:

   • Analyze your code for any memory leaks or inefficient memory usage
   • Consider using more efficient data structures or algorithms for handling high-resolution frames
   • Investigate if using shared memory between CPU and GPU could improve performance

5. Profile the Application:
   Use Nvidia’s profiling tools to identify bottlenecks in your application:

   • Nsight Systems for system-wide performance analysis
   • Nsight Compute for CUDA kernel analysis

6. Reduce Resolution or Frame Rate:
   If the Orin Nano is truly at its limit, consider:

   • Slightly reducing the resolution to find a sweet spot between quality and performance
   • Lowering the frame rate to give more processing time between frames

7. Optimize GPU Usage:

   • Ensure that GPU operations are properly synchronized and optimized
   • Consider using CUDA streams to overlap computation and data transfer

8. Investigate Alternative Display Methods:

   • Explore using NVIDIA’s hardware overlay system for more efficient display rendering
   • Consider using OpenGL ES instead of other rendering methods, if not already in use

9. Check Thermal Throttling:
   Ensure that the Orin Nano is not thermally throttling under load:

   • Monitor temperatures during operation
   • Improve cooling if necessary

10. Update Software and Drivers:
    Ensure you are using the latest JetPack SDK and have all the latest drivers installed for your Orin Nano.

If these steps do not resolve the issue, it may be necessary to reassess whether the Jetson Orin Nano has the necessary capabilities for your specific high-resolution stereoscopic camera application. Consider exploring more powerful Jetson modules or optimizing your application further to work within the Orin Nano’s constraints.