Cannot install PyTorch 2.x with CUDA support on Jetson Orin Nano

Issue Overview

Users are experiencing difficulties installing PyTorch 2.x with CUDA support on the Nvidia Jetson Orin Nano development board. The main symptoms include:

• Inability to import PyTorch after installation
• Error message: "ImportError: libcudnn.so.8: cannot open shared object file: No such file or directory"
• CUDA support not available even after successful installation

This issue occurs during the setup process, specifically when trying to install and use PyTorch 2.x. The problem affects the functionality of deep learning applications that require GPU acceleration on the Jetson Orin Nano.

Possible Causes

1. Incompatible cuDNN version: The prebuilt PyTorch package is built with a specific cuDNN version, and using a different version can cause conflicts.

2. Incorrect PyTorch wheel: Installing the wrong PyTorch wheel for the specific JetPack version can lead to compatibility issues.

3. Environment variables: Incorrect or missing environment variables for CUDA and cuDNN paths can prevent PyTorch from finding the necessary libraries.

4. Incomplete CUDA installation: Missing CUDA components or an incomplete installation can result in PyTorch failing to detect CUDA support.

5. Python version mismatch: Using a Python version different from the one the PyTorch wheel was built for can cause import errors.

Troubleshooting Steps, Solutions & Fixes

1. Verify JetPack and CUDA versions:
   Run the following commands to check your current setup:

   sudo apt show nvidia-jetpack
   nvcc --version
   

2. Install the correct cuDNN version:
   For JetPack 5.1.2, use cuDNN 8.6.0:

   sudo apt-get install libcudnn8 libcudnn8-dev libcudnn8-samples
   

   If you have a different cuDNN version installed, remove it first:

   sudo apt-get purge cudnn-local-tegra-repo-ubuntu2004-<version>
   

3. Install the appropriate PyTorch wheel:
   Download and install the correct PyTorch wheel for your JetPack version:

   wget https://developer.download.nvidia.cn/compute/redist/jp/v512/pytorch/torch-2.1.0a0+41361538.nv23.06-cp38-cp38-linux_aarch64.whl
   pip3 install ./torch-2.1.0a0+41361538.nv23.06-cp38-cp38-linux_aarch64.whl
   

4. Set environment variables:
   Add the following lines to your ~/.bashrc file:

   export PATH=/usr/local/cuda-11.4/bin:$PATH
   export LD_LIBRARY_PATH=/usr/local/cuda-11.4/lib64:$LD_LIBRARY_PATH
   

   Then, run source ~/.bashrc to apply the changes.

5. Verify PyTorch installation and CUDA support:
   Run the following Python commands to check if PyTorch is installed correctly and CUDA is available:

   import torch
   print(torch.__version__)
   print(torch.cuda.is_available())
   print(torch.backends.cudnn.version())
   

6. Reinstall CUDA toolkit (if necessary):
   If CUDA support is still not available, consider reinstalling the CUDA toolkit:

   sudo apt-get install --reinstall cuda-toolkit-11-4
   

7. Use a virtual environment (optional):
   If you encounter issues with system-wide Python packages, create a virtual environment:

   python3 -m venv pytorch_env
   source pytorch_env/bin/activate
   

   Then, install PyTorch within this environment.

8. Check for conflicting packages:
   Ensure that you don’t have conflicting PyTorch or CUDA-related packages installed:

   pip list | grep torch
   pip list | grep cuda
   

   Remove any conflicting packages if found.

By following these steps, users should be able to successfully install PyTorch 2.x with CUDA support on their Jetson Orin Nano. If the issue persists, it may be necessary to consult the official NVIDIA developer forums or support channels for further assistance.