Jetson Orin Nano – Unknown Embedded Device Detected

Issue Overview

Users of the Nvidia Jetson Orin Nano 4GB Dev board have reported encountering a warning message during the process of building a TensorRT engine from an ONNX model. The specific warning is:

[11/03/2022-12:01:57] [W] [TRT] Unknown embedded device detected. Using 2779MiB as the allocation cap for memory on embedded devices.

This issue appears to occur primarily when using TensorRT version 8.4, which does not fully support the Orin Nano 4GB variant, leading to unexpected behavior such as system freezes during engine building. Users have noted that the build process can hang indefinitely, requiring a manual reset of the device.

The problem arises in the context of benchmarking TensorRT engines and is particularly evident when the system’s swap memory becomes fully utilized. This has been confirmed through monitoring tools like tegrastats, which indicated that running out of swap memory was a contributing factor to the freezes.

The impact on user experience is significant, as it disrupts workflows and can lead to data loss if not managed properly. The issue seems to be inconsistent; while some users have successfully built engines under certain configurations, others have faced repeated failures.

Possible Causes

• Hardware Limitations: The Orin Nano 4GB may not have sufficient memory resources for certain models, especially when combined with existing system load.

• Software Bugs: The warning indicates that TensorRT 8.4 does not recognize the Orin Nano 4GB correctly, leading to suboptimal memory allocation strategies.

• Configuration Errors: Incorrect settings for memPoolSize or insufficient swap space can lead to memory allocation issues.

• Driver Issues: The lack of support for the Orin Nano 4GB in TensorRT 8.4 could lead to unexpected behaviors during engine builds.

• Environmental Factors: High workloads or background processes consuming RAM can exacerbate memory shortages.

• User Misconfigurations: Users may inadvertently set parameters that do not align with their hardware capabilities or workload requirements.

Troubleshooting Steps, Solutions & Fixes

1. Monitor System Resources:

   • Use the command:

     sudo tegrastats
     

   • Check for swap usage and overall memory consumption during engine building.

2. Adjust Swap Space:

   • If swap usage is high, consider increasing it:

     sudo dd if=/dev/zero of=/swapfile bs=1G count=4
     sudo chmod 600 /swapfile
     sudo mkswap /swapfile
     sudo swapon /swapfile
     

3. Configure Memory Pool Size:

   • Set memPoolSize to a lower value (e.g., 2GB) when building models:

     /usr/src/tensorrt/bin/trtexec --onnx=model.onnx --memPoolSize=workspace:2048
     

4. Test Different Model Configurations:

   • Experiment with various models and configurations (e.g., FP32 vs FP16) to determine optimal settings.

5. Check for System Responsiveness:

   • If using SSH, be aware that heavy workloads may slow down responsiveness significantly. It may take time for commands to execute.

6. Upgrade TensorRT:

   • If possible, upgrade to TensorRT version 8.5 once it becomes available, as it is expected to resolve compatibility issues with the Orin Nano 4GB.

7. Use Alternative Workflows:

   • If freezing persists, consider running builds on more capable hardware temporarily until stability improves on the Orin Nano.

8. Document Findings:

   • Keep a log of configurations that work or fail for future reference and community sharing.

9. Seek Community Support:

   • Engage with community forums or Nvidia support for additional insights or updates regarding potential fixes or patches.

10. Unresolved Issues:

    • Further investigation may be needed regarding the exact memory cap imposed by TensorRT and whether it can be overridden in future versions or through configuration changes.

By following these steps and adjusting configurations as necessary, users should be able to mitigate issues related to engine building on the Jetson Orin Nano 4GB Dev board effectively.