Low GPU Compute Power on Nvidia Jetson Orin Nano Dev Board

Issue Overview

Users of the Nvidia Jetson Orin Nano Dev board (8G device) are experiencing significantly lower GPU compute power than expected. The product specification indicates a compute capability of 40 TOPS, but actual test results show only 0.737 TOPS. This discrepancy is causing concern among users who are trying to utilize the full potential of the device for their applications.

The issue appears to be consistent across various testing methods, including the use of gpu_burn and CUDA samples. Users are unsure if the problem stems from incorrect testing methodologies or if there are underlying hardware or software issues affecting the device’s performance.

Possible Causes

1. Incorrect Testing Methodology: The testing methods used may not accurately measure the GPU’s computational performance. The gpu_burn test and bandwidthTest are primarily designed to measure memory operations rather than computational power.

2. Software Configuration: Improper software setup, outdated drivers, or incompatible CUDA versions could potentially limit the GPU’s performance.

3. Power Management Settings: The device may be operating in a power-saving mode, reducing its computational capabilities.

4. Thermal Throttling: If the device is overheating, it may reduce its performance to maintain safe operating temperatures.

5. Hardware Limitations: There could be a hardware issue or limitation specific to the user’s device.

6. Misinterpretation of Results: The user might be misinterpreting the test results, confusing memory bandwidth or other metrics with computational performance.

Troubleshooting Steps, Solutions & Fixes

1. Use Appropriate Benchmarking Tools:

   • The NVIDIA representative recommends using the CUTLASS profiler for accurate benchmarking of computational performance.
   • Install CUTLASS:

     git clone https://github.com/NVIDIA/cutlass.git
     cd cutlass
     

   • Follow the CUTLASS documentation to run appropriate benchmarks for your specific use case.

2. Verify CUDA Installation:

   • Check CUDA version compatibility with your Jetson Orin Nano:

     nvcc --version
     

   • Ensure you have the latest CUDA toolkit installed for your device.

3. Update System Software:

   • Update the Jetson OS and CUDA drivers to the latest versions:

     sudo apt update
     sudo apt upgrade
     

4. Check Power Mode:

   • Verify that the device is not in a power-saving mode:

     sudo nvpmodel -q
     

   • If necessary, set to maximum performance mode:

     sudo nvpmodel -m 0
     

5. Monitor Thermal Performance:

   • Use the Jetson stats tool to monitor temperatures during benchmarking:

     sudo jtop
     

   • Ensure proper cooling and ventilation for the device.

6. Verify Hardware Functionality:

   • Run NVIDIA’s built-in hardware test:

     sudo /opt/nvidia/jetson-io/jetson-io.py
     

   • Select "Hardware Configuration" and then "Run Board Tests"

7. Analyze Device Query Results:

   • Review the output of deviceQuery for any anomalies:

     cd /usr/local/cuda/samples/1_Utilities/deviceQuery
     sudo make
     ./deviceQuery
     

   • Compare the results with the official specifications for your Jetson Orin Nano model.

8. Use NVIDIA’s Official Performance Analysis Tools:

   • Install and use NVIDIA Nsight Systems for detailed performance analysis:

     sudo apt-get install nvidia-nsight-systems
     nsys profile -t cuda,osrt,nvtx,cublas,cudnn -o my_profile ./my_application
     

   • Analyze the profile to identify performance bottlenecks.

9. Contact NVIDIA Support:

   • If the issue persists after trying these steps, reach out to NVIDIA’s official support channels with detailed information about your setup and the steps you’ve taken.

Remember that the gpu_burn and bandwidthTest results provided in the original query are not indicative of computational performance. Focus on using CUTLASS or other NVIDIA-recommended tools for accurate TOPS measurements.