ArgusCamera Application Feed Stuck After a While

Issue Overview

Users are experiencing issues with the ArgusCamera application on the Nvidia Jetson Orin Nano Dev board, where the camera feed becomes stuck after a period of time. The problem occurs specifically when streaming multiple cameras in multi-session mode. Users have reported that even after restarting the ArgusCamera application, the feed does not resume unless the nv_argus_daemon is also restarted.

Symptoms and Context

• The camera feed freezes, requiring a restart of the daemon for restoration.
• The issue manifests when using three single-lane IMX296 cameras at a resolution of 1456×1088 pixels and a frame rate of 60 fps.
• Users have verified that the issue does not occur when using one or two cameras, indicating that it is related to the simultaneous use of three cameras.
• The problem appears to be intermittent, as it occurs after some time of operation but does not have a consistent trigger.

Impact on User Experience

This issue significantly affects the functionality of applications relying on real-time camera feeds, as users must manually restart the daemon to restore functionality. This can disrupt workflows and reduce efficiency, particularly in applications requiring continuous monitoring or processing from multiple camera sources.

Possible Causes

• Hardware Limitations: The Jetson Orin Nano may be reaching its limits when handling multiple high-resolution camera streams simultaneously, leading to performance bottlenecks.

• Software Bugs: There may be bugs in the ArgusCamera application or nv_argus_daemon that cause them to become unresponsive under certain conditions.

• Configuration Errors: Improper configuration settings for camera streams could lead to instability when multiple cameras are in use.

• Driver Issues: Outdated or incompatible drivers for the cameras or the Jetson platform could cause communication failures.

• Environmental Factors: Insufficient power supply or overheating could affect system stability during intensive operations.

• User Errors: Misconfigurations in camera settings or stream parameters might contribute to the problem.

Troubleshooting Steps, Solutions & Fixes

1. Isolate the Issue:

   • Test each camera individually to confirm they operate correctly on their own.
   • Test combinations of two cameras to see if the issue persists before adding a third camera.

2. Boost Clock Rates:

   • Increase clock rates for various components to enhance performance. Use the following commands:

     sudo su
     echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
     echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
     echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
     echo 1 > /sys/kernel/debug/bpmp/debug/clk/emc/mrq_rate_locked
     cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
     cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/isp/rate
     cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
     cat /sys/kernel/debug/bpmp/debug/clk/emc/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/emc/rate
     

3. Restart Daemon and Application:

   • If the feed gets stuck, restart both the nv_argus_daemon and ArgusCamera application to restore functionality.

4. Check Logs for Errors:

   • Investigate log files for any error messages that may indicate why the nv_argus_daemon becomes unresponsive. This can help pinpoint software bugs or configuration issues.

5. Update Drivers and Firmware:

   • Ensure that all drivers for the cameras and Jetson platform are up-to-date. Check Nvidia’s official documentation for any available firmware upgrades.

6. Monitor System Resources:

   • Use monitoring tools to check CPU and memory usage while streaming from multiple cameras. This can reveal whether resource limitations are causing the issue.

7. Best Practices:

   • Limit simultaneous streams if possible or reduce resolution/frame rate settings to lessen system load.
   • Regularly check for software updates from Nvidia that may address known issues with the ArgusCamera application.

Recommended Approach

Many users have found success by boosting clock rates as outlined above, which is recommended as a first step in troubleshooting this issue.

Unresolved Aspects

Further investigation may be needed into potential software bugs within the nv_argus_daemon or compatibility issues with specific camera models when used in multi-camera configurations.