GPU/HW acceleration with GStreamer nvvidconv

Issue Overview

Users are experiencing performance issues when attempting to utilize GPU hardware acceleration with the Nvidia Jetson Orin Nano Dev board while using GStreamer. The primary symptoms include:

• Frame Rate Limitations: Users report being able to achieve frame rates of 60 fps with a basic pipeline but only up to 70 fps when incorporating Nvidia plugins for format conversion.

• Dropped Frames: When attempting to increase the frame rate beyond 70 fps (e.g., 80 fps), users encounter dropped frames, leading to warnings about buffer drops and potential timestamping issues.

• Pipeline Complexity: Users are questioning the necessity of multiple nvvidconv instances in their GStreamer pipelines, as well as looking for more optimal configurations.

The issue occurs during the execution of GStreamer pipelines, specifically when using the nvvidconv plugin for video format conversion from YUY2 to RGB. The hardware being used includes the Jetson Orin Nano (4GB) running Jetpack 5.1.1 and GStreamer version 1.16.3.

The dropped frames and performance limitations impact user experience, particularly for applications requiring high frame rates and smooth video playback.

Possible Causes

Several potential causes for the observed issues have been identified:

• Hardware Limitations: The Jetson Orin Nano may have inherent limitations in processing power or memory bandwidth that restricts the achievable frame rates.

• Software Bottlenecks: The use of videotestsrc, which is a CPU-based implementation, may introduce performance bottlenecks, especially at high framerates.

• Configuration Errors: Incorrect pipeline configurations or unnecessary complexity (e.g., multiple nvvidconv instances) can lead to inefficiencies in processing.

• Driver Issues: Outdated or incompatible drivers may hinder optimal performance of GPU acceleration features.

• Environmental Factors: Insufficient power supply or thermal throttling could affect performance under load.

• User Errors: Misconfigurations in the GStreamer pipeline or misunderstanding of the capabilities of the hardware could lead to suboptimal setups.

Troubleshooting Steps, Solutions & Fixes

To address the performance issues and optimize GStreamer pipelines on the Nvidia Jetson Orin Nano, follow these troubleshooting steps and solutions:

1. Boost CPU Clocks:

   • Run the following command to maximize CPU performance:

     sudo jetson_clocks
     

2. Simplify Pipeline for Testing:

   • Test with a reduced resolution to minimize CPU load:

     gst-launch-1.0 videotestsrc ! video/x-raw,width=200,height=200,format=YUY2,framerate=80/1 ! nvvidconv ! "video/x-raw(memory:NVMM),format=RGBA,width=800,height=800" ! nvvidconv ! videoconvert ! video/x-raw,format=RGB ! queue ! fpsdisplaysink text-overlay=0 video-sink=fakesink -v
     

3. Optimize Plugin Sequence:

   • Use only one instance of nvvidconv where possible. For example:

     gst-launch-1.0 videotestsrc ! video/x-raw,width=800,height=800,format=YUY2,framerate=80/1 ! nvvidconv compute-hw=GPU ! "video/x-raw(memory:NVMM),format=RGBA" ! videoconvert ! video/x-raw,format=RGB ! queue ! ccm800x800cv ! queue ! fpsdisplaysink text-overlay=0
     

4. Disable Unnecessary Features:

   • Disable features like text overlay on fpsdisplaysink during testing to reduce CPU load:

     fpsdisplaysink text-overlay=0
     

5. Check Display Capabilities:

   • Ensure that your display can handle high frame rates (e.g., 80 fps). If necessary, test with a different display or use a fakesink for initial testing.

6. Monitor System Resources:

   • Use tools like htop or nvidia-smi to monitor CPU and GPU usage during pipeline execution to identify bottlenecks.

7. Consider Alternative Sources:

   • If using videotestsrc, consider switching to a hardware source (like a camera) that can provide data at higher rates without CPU overhead.

8. Review Documentation and Updates:

   • Regularly check for updates to Jetpack and GStreamer that may include performance improvements or bug fixes.

9. Experiment with Different Configurations:

   • Adjust parameters in your GStreamer pipeline based on feedback from community discussions or documentation regarding optimal configurations for specific tasks.

10. Seek Community Support:

    • If issues persist after trying these solutions, consider reaching out in forums dedicated to Jetson development for additional insights and support from experienced users.

By following these steps, users should be able to diagnose and resolve issues related to GPU acceleration with GStreamer on the Nvidia Jetson Orin Nano Dev board effectively.