Sample Code for Object Detection and Semantic Segmentation on Jetson Nano

Issue Overview

Users are seeking sample code for object detection and semantic segmentation on the NVIDIA Jetson Nano board. The primary goal is to experiment with accelerating inference of pre-trained models on the NVIDIA GPU. Two specific requirements have been identified:

1. Converting pre-trained models (not created by NVIDIA) to ONNX format, then to TensorRT, and performing inference.
2. Creating a model similar to the MNIST example, obtaining trained model weights, creating the network, building the engine, and performing inference.

The context of this request is to facilitate further experimentation and understanding of how to leverage the Jetson Nano’s capabilities for these computer vision tasks.

Possible Causes

While this is not an issue per se, the lack of readily available sample code for these specific tasks can be attributed to several factors:

1. Complexity of tasks: Object detection and semantic segmentation are more complex than simple classification tasks like MNIST, requiring more sophisticated models and processing pipelines.
2. Diversity of models: There are numerous pre-trained models for these tasks, making it challenging to provide a one-size-fits-all sample.
3. Rapid evolution of frameworks: The fast-paced development in deep learning frameworks and tools can make it difficult to maintain up-to-date samples.
4. Hardware-specific optimizations: The need for optimizations specific to the Jetson Nano’s hardware architecture may complicate the creation of generic samples.

Troubleshooting Steps, Solutions & Fixes

To address the user’s requirements, here are step-by-step solutions and resources:

Converting Pre-trained Models to TensorRT

1. Convert model to ONNX format:

   • For PyTorch models, use the torch.onnx.export() function.
   • For TensorFlow models, use the tf2onnx package.

2. Convert ONNX to TensorRT:
   Use the trtexec tool provided by NVIDIA:

   /usr/src/tensorrt/bin/trtexec --onnx=[input_model.onnx] --saveEngine=[output_engine.trt]
   

3. Deploy the TensorRT engine:
   Modify the sample code provided in the eLinux wiki. This sample demonstrates how to use OpenCV with a TensorRT plan model.

Creating and Deploying Custom Models

1. Develop your model:
   Create your model using PyTorch or TensorFlow, similar to the MNIST example in /usr/src/tensorrt/samples/python/network_api_pytorch_mnist/.

2. Train the model and save weights.

3. Convert to ONNX:
   Use the appropriate conversion method for your framework (as mentioned in the first solution).

4. Convert to TensorRT and deploy:
   Follow steps 2 and 3 from the previous solution.

Additional Resources

1. Jetson Inference Library:
   Explore the jetson-inference library on GitHub. This provides a comprehensive guide to deploying deep learning inference networks and vision primitives on Jetson devices.

2. Transfer Learning:
   The jetson-inference library also includes examples of transfer learning, which can be useful for adapting pre-trained models to specific tasks.

3. NVIDIA Documentation:
   Refer to NVIDIA’s official documentation for Jetson devices and TensorRT for more detailed information on optimizing models for the Jetson Nano.

4. Community Forums:
   Continue to engage with the NVIDIA Developer forums for specific issues or questions that arise during implementation.

By following these steps and utilizing the provided resources, users should be able to experiment with object detection and semantic segmentation on the Jetson Nano board, including both converting pre-trained models and creating custom ones optimized for TensorRT inference.