Hardware Acceleration Issues with FFmpeg on Jetson Orin Nano for USB Camera Streaming

Issue Overview

Users are experiencing high CPU usage when using FFmpeg for encoding and streaming video from a USB camera on the Nvidia Jetson Orin Nano development board. The specific problem occurs when attempting to push an RTMP stream to a server, resulting in excessive CPU utilization (up to 400% or all 4 cores fully running) for a 1920×1080 resolution at 20 FPS in H.264 format. This issue is particularly problematic as it prevents users from running other programs concurrently due to the lack of available CPU resources.

The context of this problem stems from users upgrading from a Jetson Nano, where they previously used hardware acceleration (h264_nvmpi encoding) successfully with CSI cameras. However, after upgrading to the Jetson Orin Nano and switching to USB cameras, the hardware acceleration method no longer works efficiently.

Possible Causes

1. Lack of Hardware Encoders: The Jetson Orin Nano does not have dedicated hardware encoders, unlike its predecessor, the Jetson Nano.

2. Incompatible Hardware Acceleration: The previously used h264_nvmpi encoding was specifically designed for the Jetson Nano and may not be compatible or optimized for the Jetson Orin Nano architecture.

3. USB Camera Overhead: The switch from CSI to USB cameras might introduce additional processing overhead, contributing to increased CPU usage.

4. Software Encoder Limitations: Without hardware encoders, the system relies on software encoding, which is inherently more CPU-intensive, especially for high-resolution, high-framerate video streams.

5. Outdated or Incompatible Software: The FFmpeg version or configuration might not be optimized for the Jetson Orin Nano’s capabilities.

Troubleshooting Steps, Solutions & Fixes

1. Confirm Hardware Limitations:

   • Understand that the Jetson Orin Nano lacks hardware encoders, which explains the high CPU usage when encoding video streams.

2. Optimize FFmpeg Command:

   • Experiment with different FFmpeg encoding settings to find a balance between quality and CPU usage. For example:

     ffmpeg -f v4l2 -framerate 20 -video_size 1280x720 -i /dev/video2 -c:v libx264 -preset ultrafast -tune zerolatency -b:v 1M -f flv rtmp://HOST_IP/live/livestream
     

   • This command reduces resolution, uses a faster preset, and lowers bitrate to potentially reduce CPU load.

3. Consider GStreamer Pipeline:

   • GStreamer might offer better performance on Jetson platforms. Try adapting your previous GStreamer command for USB cameras:

     gst-launch-1.0 v4l2src device=/dev/video2 ! \
     video/x-raw,width=1280,height=720,framerate=30/1 ! \
     videoconvert ! x264enc tune=zerolatency ! \
     flvmux ! rtmpsink location="rtmp://HOST_IP/live/livestream"
     

4. Reduce Resolution or Framerate:

   • Lower the video resolution (e.g., from 1920×1080 to 1280×720) or reduce the framerate (e.g., from 20 FPS to 15 FPS) to decrease encoding workload.

5. Explore GPU-Accelerated Solutions:

   • While the Orin Nano lacks hardware encoders, it has a powerful GPU. Investigate NVIDIA’s Video Codec SDK or CUDA-accelerated encoding libraries that might leverage the GPU for encoding tasks.

6. Update JetPack and FFmpeg:

   • Ensure you’re using the latest JetPack (version 6.0 GA or newer) and the most recent FFmpeg package compatible with Jetson Orin Nano.

7. Consider Hardware Alternatives:

   • If high-quality, low-latency encoding is crucial for your application, consider using a Jetson module with hardware encoding capabilities, such as the Jetson AGX Orin.

8. Optimize System Resources:

   • Close unnecessary background processes and services to free up CPU resources for encoding tasks.

9. Investigate Alternative Streaming Methods:

   • Consider using NVIDIA’s DeepStream SDK, which is optimized for Jetson platforms and might offer better performance for video analytics and streaming tasks.

10. Consult NVIDIA Developer Forums:

    • Post your specific use case and requirements on the NVIDIA Developer Forums for Jetson to get tailored advice from NVIDIA engineers or experienced community members.

Remember that without hardware encoders, achieving low CPU usage for high-quality video encoding on the Jetson Orin Nano may be challenging. You may need to compromise on either video quality or CPU usage based on your specific application requirements.