Torchaudio Build Failure on Jetson Orin Nano

Issue Overview

Users are experiencing difficulties building the torchaudio package on the Nvidia Jetson Orin Nano 8GB development kit. The build process fails during the compilation of CUDA-related components, specifically when building the l4t-pytorch container. This issue occurs with the following software versions:

• L4T_VERSION: 35.4.1
• JETPACK_VERSION: 5.1.2
• CUDA_VERSION: 11.4.315
• LSB_RELEASE: 20.04 (focal)

The build process terminates with a "Killed" message, suggesting that the system ran out of memory during compilation.

Possible Causes

1. Insufficient system memory: The Jetson Orin Nano has limited RAM, which may not be enough for the torchaudio build process.

2. Lack of swap space: Without adequate swap space, memory-intensive operations like compilation can fail when physical RAM is exhausted.

3. Resource-intensive build process: The torchaudio package, especially with CUDA support, requires significant computational resources during compilation.

4. Outdated or incompatible software versions: Mismatches between CUDA, PyTorch, and torchaudio versions could lead to compilation issues.

Troubleshooting Steps, Solutions & Fixes

1. Increase swap space:

   • Follow the instructions in the Jetson containers documentation to mount additional swap:

     sudo systemctl disable nvzramconfig
     sudo fallocate -l 4G /mnt/4GB.swap
     sudo mkswap /mnt/4GB.swap
     sudo swapon /mnt/4GB.swap
     

   • Add this line to /etc/fstab to make the swap persistent:

     /mnt/4GB.swap  none  swap  sw 0  0
     

2. Disable the desktop GUI:

   • If adding swap doesn’t resolve the issue, try disabling the desktop environment to free up more resources:

     sudo init 3
     

3. Use pre-built container images:

   • Instead of building from scratch, consider using pre-built l4t-pytorch container images available in the Jetson containers repository.

4. Optimize build flags:

   • Modify the build process to use fewer parallel jobs or reduce optimization levels:

     export MAX_JOBS=1
     export TORCH_CUDA_ARCH_LIST="5.3;6.2;7.2;8.7"
     

5. Update software versions:

   • Ensure you’re using the latest compatible versions of JetPack, PyTorch, and torchaudio.
   • Check for any known issues or patches for the specific versions you’re using.

6. Clean build environment:

   • Remove any previous build artifacts and start with a clean environment:

     rm -rf build dist
     pip uninstall torchaudio
     

7. Monitor resource usage:

   • Use tools like top or htop to monitor system resources during the build process and identify bottlenecks.

8. Build in stages:

   • Try building PyTorch and torchvision separately before attempting to build torchaudio.

If the issue persists after trying these solutions, consider reporting the problem to the torchaudio GitHub repository with detailed logs and system information.