Failed TensorRT.trtexec on Jetson Orin Nano

Issue Overview

Users are experiencing an error when attempting to convert an ONNX model to a TensorRT engine using the command line tool trtexec. The specific error message is "&&&& FAILED TensorRT.trtexec [TensorRT v8502]." This issue arises during the model conversion process, particularly when utilizing the Jetson Orin Nano development board.

The user reported the following specifications:

• CUDA Version: 10.4
• L4T Version: 35.3.1
• TensorRT Version: Initially installed version was 8.5.2.2, but after subsequent installations, it changed to 5.1.1.
• cuDNN Version: Reported as 1.0 after installation.

The problem seems to occur consistently when executing the command:

$ /usr/src/tensorrt/bin/trtexec --onnx=v8n.onnx --saveEngine=nano.engine

The impact of this issue is significant as it prevents users from effectively converting their models for inference, which can hinder development and testing of machine learning applications on the Jetson platform.

Possible Causes

Several potential causes for this issue have been identified:

• Binary Mismatch: The version of trtexec being used may not be compatible with the installed version of TensorRT or CUDA, leading to execution failures.

• Incorrect Installation: The installation of TensorRT may not have completed successfully or may be misconfigured, especially since the user mentioned multiple installations.

• Software Bugs: There may be bugs present in the specific version of TensorRT (5.1.1) that prevent it from functioning correctly with certain ONNX models.

• API Compatibility: If the user is not using code based on the TensorRT API, there could be incompatibilities when trying to use models converted with trtexec.

• Environmental Factors: Issues such as insufficient power supply or overheating could potentially affect performance and lead to errors during execution.

Troubleshooting Steps, Solutions & Fixes

To address the issue, users can follow these troubleshooting steps and potential solutions:

1. Verify TensorRT Installation:

   • Ensure that TensorRT is correctly installed by checking its version:

     dpkg -l | grep nvidia-tensorrt
     

2. Check trtexec Compatibility:

   • Confirm that the trtexec binary matches the installed version of TensorRT:

     /usr/src/tensorrt/bin/trtexec --version
     

3. Reinstall TensorRT:

   • If there are inconsistencies in versions, consider reinstalling TensorRT. Follow NVIDIA’s official installation guide for Jetson devices.

4. Test with a Simple ONNX Model:

   • Attempt to convert a simpler ONNX model to check if the issue persists:

     /usr/src/tensorrt/bin/trtexec --onnx=simple_model.onnx --saveEngine=simple_model.engine
     

5. Use Correct CUDA and cuDNN Versions:

   • Ensure that CUDA and cuDNN versions are compatible with the installed version of TensorRT.

6. Check for Updates:

   • Look for any updates or patches for TensorRT that might address known issues.

7. Consult Documentation:

   • Refer to NVIDIA’s official documentation for troubleshooting tips related to trtexec and model conversion processes.

8. Testing Different Configurations:

   • If possible, test with different configurations or hardware setups to isolate whether the issue is hardware-related.

9. Reach Out for Support:

   • If none of the above steps resolve the issue, consider reaching out to NVIDIA support or community forums with detailed logs and error messages for further assistance.

10. Best Practices:

    • Always ensure that you are using compatible versions of software components (CUDA, cuDNN, TensorRT) when working on Jetson devices.
    • Regularly check for updates from NVIDIA that could enhance compatibility and performance.

By following these steps, users should be able to diagnose and potentially resolve issues related to running trtexec on their Jetson Orin Nano boards.