Issue with ssd-mobilenetv2 using jetson-inference on Jetson Orin Nano 8GB

Issue Overview

Users are experiencing errors when running object detection scripts using the jetson-inference library on a Jetson Orin Nano 8GB device with JetPack 6.0. The main symptoms include:

1. CUDA errors during execution, specifically:

   [cuda] cudaEventElapsedTime(&cuda_time, mEventsGPU[evt], mEventsGPU[evt+1])
   [cuda] device not ready (error 600) (hex 0x258)
   [cuda] /home/trinity/jetson-inference/build/aarch64/include/jetson-inference/tensorNet.h:769
   

2. Failure to detect objects in images when running the detectnet.py script.

3. Incorrect loading of class labels:

   detectNet – number of object classes: 8400
   [TRT] detectNet – maximum bounding boxes: 84
   [TRT] loaded 80 class labels
   [TRT] didn't load expected number of class descriptions (80 of 8400)
   

The issue occurs when trying to use a pre-trained YOLOv8 model in ONNX format with the jetson-inference library.

Possible Causes

1. Incompatibility between the YOLOv8 ONNX model and the jetson-inference library:
   The library expects an ssd-mobilenet-v2 checkpoint exported from PyTorch, not a YOLOv8 model.

2. Incorrect branch selection in the jetson-inference repository:
   The user is on the master branch, which may not be compatible with JetPack 6.0.

3. CUDA configuration issues:
   The CUDA errors suggest potential problems with the CUDA setup or compatibility.

4. Mismatch between the model’s class count and the library’s expectations:
   The error message indicates a discrepancy in the number of object classes.

Troubleshooting Steps, Solutions & Fixes

1. Verify JetPack and L4T versions:
   Use the jtop tool to confirm the installed versions:

   jtop
   

   Ensure it shows "NVIDIA Jetson Orin Nano Developer Kit JetPack 6.0 [L4T 36.3.0]".

2. Check and switch to the correct branch in the jetson-inference repository:

   cd /home/trinity/jetson-inference
   git branch
   

   If not on the correct branch for JetPack 6.0, switch to it (consult the repository documentation for the correct branch).

3. Test with default models:
   Try running the detectnet script with the default ssd-mobilenet-v2 model:

   python3 detectnet.py 'images/object_*.jpg'
   

   If this works, the issue is likely related to the YOLOv8 model compatibility.

4. Use compatible models:
   The jetson-inference library doesn’t support YOLOv8 ONNX models out of the box. Consider using ssd-mobilenet-v2 models trained with the provided train_ssd.py script.

5. Adapt the code for YOLOv8:
   If you must use YOLOv8, you’ll need to modify the pre/post-processing code in jetson-inference/c/detectNet.cpp to support the YOLOv8 architecture. This requires advanced C++ and deep learning knowledge.

6. Consider alternative libraries:
   For YOLOv8 support on Jetson devices, consider using the Ultralytics library, which supports YOLOv8 in TensorRT:
   Ultralytics YOLOv8 – NVIDIA Jetson AI Lab

7. Rebuild the jetson-inference library:
   If you’ve made changes or switched branches, rebuild the library:

   cd /home/trinity/jetson-inference
   mkdir build
   cd build
   cmake ..
   make -j$(nproc)
   sudo make install
   sudo ldconfig
   

8. Check CUDA configuration:
   Ensure CUDA is properly configured for your Jetson device. You may need to reinstall or reconfigure CUDA if issues persist.

9. Verify model compatibility:
   Ensure your ONNX model is compatible with the version of TensorRT installed on your Jetson device. You may need to re-export the model with a compatible ONNX version.

10. Monitor system resources:
    Use jtop to monitor CPU, GPU, and memory usage while running your script. Ensure you’re not exceeding the device’s capabilities.

If the issue persists after trying these steps, consider reaching out to NVIDIA’s developer forums with detailed logs and information about your setup for further assistance.