High Performance Benchmarks on Nvidia Jetson Orin Nano Dev Board

Issue Overview

Users of the Nvidia Jetson Orin Nano Dev board have reported confusion regarding the performance benchmarks they are observing, specifically in relation to the floating-point operations per second (FLOPS) calculations. One user noted that their benchmark results using the Cutlass library under FP16 showed a peak performance of approximately 9124 GFlop/s. This raised concerns about the accuracy of these performance metrics, especially when compared to established benchmarks for other Nvidia GPUs like the A100, which achieves 256 FLOP/cycle per tensor core and 1024 FLOP/cycle per Streaming Multiprocessor (SM).

The issue arises during benchmarking tests, leading to questions about whether users are miscalculating expected performance or if there are discrepancies in how performance is reported. The confusion was resolved when it was clarified that the correct metric should consider Fused Multiply-Add (FMA) operations, which effectively double the FLOP count.

The frequency of this issue appears to be consistent among users benchmarking their devices, impacting their understanding of the Jetson Orin Nano’s capabilities compared to other high-performance GPUs.

Possible Causes

• Misunderstanding of Performance Metrics: Users may not fully grasp the difference between FLOPS and FMA operations, leading to inflated expectations.

• Configuration Errors: Incorrect settings or parameters during benchmarking can skew results.

• Driver Issues: Outdated or incompatible drivers may affect performance metrics reported by the system.

• Environmental Factors: Variations in temperature or power supply could impact performance but are less likely to cause significant discrepancies in reported FLOPS.

• User Errors: Incorrect calculations or assumptions about hardware capabilities can lead to confusion regarding performance outputs.

Troubleshooting Steps, Solutions & Fixes

1. Clarify Performance Metrics:

   • Understand the difference between FLOPS and FMA operations. For instance, FMA operations effectively double the FLOP count since they combine multiplication and addition in a single instruction.

2. Verify Benchmarking Setup:

   • Ensure that you are using the latest version of the Cutlass library and that your benchmarking code is correctly configured.
   • Review documentation for any specific setup instructions related to your hardware.

3. Check Driver Versions:

   • Run the following command to check your current driver version:

     nvidia-smi
     

   • Compare your driver version against the latest available on Nvidia’s official website and update if necessary.

4. Recalculate Expected Performance:

   • Use the formula:
     $$\text{Total Performance} = \text{Clock Speed (Hz)} \times \text{Number of SMs} \times \text{FLOP/cycle}$$
   • For example, if your Jetson is running at 625 MHz with 8 SMs:
     $$\text{Total Performance} = 625 \times 10^6 \times 8 \times 1024$$
   • Ensure you account for FMA operations by using:
     $$\text{Total Performance} = 625 \times 10^6 \times 8 \times (2 \times 1024)$$

5. Test with Different Configurations:

   • If possible, benchmark with different workloads or configurations to see if results vary significantly.

6. Consult Documentation:

   • Review Nvidia’s official documentation for any notes on performance expectations specific to the Jetson Orin Nano Dev board.

7. Community Engagement:

   • Engage with forums or user groups dedicated to Nvidia products for shared experiences and solutions.

8. Best Practices:

   • Regularly update your software and drivers.
   • Document your benchmarking setups for future reference and comparisons.

By following these troubleshooting steps, users can better understand their device’s performance capabilities and ensure accurate benchmarking results.