CUDA Driver and Runtime Issues on Jetson Orin Nano DevKit

Issue Overview

Users of the Jetson Orin Nano DevKit 8GB are experiencing difficulties with CUDA components when building custom images using Ridgerun’s instructions. The main issues revolve around the inclusion and functionality of CUDA drivers and runtime libraries in different image configurations. Users are unsure about which packages are necessary for CUDA functionality and how they affect the image size and performance of CUDA applications.

Possible Causes

1. Incomplete CUDA package installation: The core-image-full-cmdline doesn’t include CUDA components by default, leading to potential missing dependencies.

2. Confusion between CUDA driver and runtime: Users may not understand the distinction between the CUDA driver (integrated into the OS) and the CUDA runtime (user-space library).

3. Inconsistent package combinations: Different combinations of CUDA-related packages (libstdc++, cuda-driver, tegra-libraries-cuda, cuda-libraries) yield varying results, causing confusion.

4. Image type limitations: Minimal or command-line images may lack necessary components for full CUDA functionality.

5. Incorrect package installation location: CUDA components may be installed in unexpected locations, leading to runtime errors.

Troubleshooting Steps, Solutions & Fixes

1. Understand CUDA components:

   • CUDA driver: Integrated into the L4T OS (r35 or r36 for Orin)
   • CUDA runtime: User-space library that needs to be installed separately

2. For full images with internet access:
   Install CUDA runtime using apt:

   sudo apt install cuda
   

3. For custom minimal images:
   Add necessary CUDA components in the $YOCTO_DIR/build/conf/local.conf file:

   • libstdc++
   • cuda-driver
   • tegra-libraries-cuda
   • cuda-libraries

4. Verify CUDA installation locations:

   • Check for CUDA runtime in /usr/local/cuda/
   • Look for libcuda.so.1.1 in /usr/lib/
   • Ensure libcudart.so is present in /usr/local/cuda/lib64/

5. Test CUDA functionality:
   Compile and run a simple CUDA "Hello World" program to verify the installation.

6. Optimize image size:

   • If space is a concern, try using only cuda-libraries or tegra-libraries-cuda
   • Note that cuda-libraries may increase image size significantly

7. For minimal or command-line images:
   Be aware that these images may lack components like sudo, dpkg, GUI, and internet connectivity. You may need to add these manually if required.

8. Distinguish between driver and runtime:

   • libcuda.so is the driver library
   • libcudart.so in /usr/local/cuda/lib64/ is the user-space runtime library

9. If issues persist:

   • Double-check L4T (Linux for Tegra) version compatibility with your Jetson Orin Nano
   • Ensure you’re using the correct JetPack version for your device
   • Consult NVIDIA’s official documentation for Jetson Orin Nano DevKit

10. Consider using NVIDIA SDK Manager:
    If building custom images proves challenging, use NVIDIA SDK Manager for a more streamlined setup process that includes necessary CUDA components.

Remember to always refer to the latest NVIDIA Jetson documentation for the most up-to-date information on CUDA setup and configuration for your specific device and use case.