Jetson Orin Nano Dev Board: TensorRT Action Recognition ONNX File Error

Issue Overview

Users are encountering an error when attempting to run an action-recognition model on the Nvidia Jetson Orin Nano Dev board using the jetson-inference container. The specific command used is:

actionnet --model=python/training/detection/ssd/models/action-recognition/rgb_resnet18_3.onnx --labels=python/training/detection/ssd/models/action-recognition/labels.txt csi://0

The error message received is:

[TRT]    Loaded 59 bytes of code generator cache.
[TRT]    device GPU, building CUDA engine (this may take a few minutes the first time a network is loaded)
[TRT]    4: [network.cpp::validate::3062] Error Code 4: Internal Error (Network has dynamic or shape inputs, but no optimization profile has been defined.)
[TRT]    device GPU, failed to build CUDA engine
[TRT]    device GPU, failed to load python/training/detection/ssd/models/action-recognition/rgb_resnet18_3.onnx
[TRT]    actionNet -- failed to initialize.
actionnet:  failed to initialize actionNet

This issue arises during the execution of the model and indicates a failure to build the CUDA engine due to dynamic shape inputs without an optimization profile defined. The problem consistently affects users attempting to utilize this specific model in their applications, significantly hindering their ability to perform action recognition tasks.

Possible Causes

• Dynamic Inputs: The ONNX model may have dynamic inputs that require an optimization profile for TensorRT to function correctly.

• Missing Optimization Profile: Users may not have defined an optimization profile for the model, which is necessary for handling dynamic shapes in TensorRT.

• Compatibility Issues: There could be compatibility issues between the version of TensorRT and the ONNX file exported from TAO toolkit 5.0.

• Configuration Errors: Incorrect command syntax or misconfigured paths may lead to failures in loading the model.

• Software Bugs: There may be bugs within the jetson-inference container or TensorRT that affect model loading and execution.

Troubleshooting Steps, Solutions & Fixes

1. Define Optimization Profile:

   • Ensure that an optimization profile is defined for your model. This can be done using TensorRT’s API or configuration files.

   Example command to create an optimization profile:

   trtexec --onnx=python/training/detection/ssd/models/action-recognition/rgb_resnet18_3.onnx --minShapes=input_name:1x3x224x224 --optShapes=input_name:1x3x224x224 --maxShapes=input_name:1x3x224x224
   

2. Check Model Compatibility:

   • Verify that the ONNX model is compatible with your version of TensorRT. Consult the documentation for both TensorRT and TAO toolkit for compatibility matrices.

3. Update Software:

   • Ensure that you are using the latest versions of the jetson-inference container and TensorRT. Update them if necessary.

4. Test with Sample Models:

   • Run sample models provided by NVIDIA (e.g., from DeepStream SDK) to confirm that your setup works correctly with known good models.

5. Review Command Syntax:

   • Double-check the command syntax for any errors, particularly in file paths and model specifications.

6. Consult Documentation:

   • Refer to relevant documentation for action recognition models within DeepStream or other NVIDIA resources for guidance on setup and configuration.

7. Isolation Testing:

   • Test with different models or configurations to isolate whether the issue is specific to the rgb_resnet18_3.onnx model or a broader problem with your setup.

8. Community Support:

   • Engage with community forums or NVIDIA support channels for additional insights or similar experiences from other users.

9. Log Additional Information:

   • Use logging options in TensorRT or jetson-inference to capture more detailed error messages that could provide further clues about the failure.

By following these steps, users can systematically diagnose and potentially resolve issues related to running action recognition models on the Nvidia Jetson Orin Nano Dev board. If problems persist, further investigation may be required into specific configurations or software versions in use.