The option to method add_matrix_multiply kNONE is not accepted

Issue Overview

Users have reported issues when attempting to use the add_matrix_multiply() function in the Nvidia Jetson Orin Nano Dev board’s TensorRT library. The primary symptoms include receiving a NameError indicating that kNONE is not defined, and a TypeError regarding incompatible function arguments.

These errors typically occur during the implementation of matrix multiplication in neural network definitions. Users are often trying to pass NumPy arrays directly into the function instead of the expected tensor types from TensorRT. The issue appears consistently among users who are new to TensorRT or transitioning from other frameworks, impacting their ability to utilize matrix multiplication effectively.

The specific errors encountered are:

• NameError: name ‘kNONE’ is not defined
• TypeError: add_matrix_multiply(): incompatible function arguments

The problem arises during the setup phase, particularly when defining input tensors for matrix operations. It significantly hampers user experience by preventing them from executing basic operations necessary for building neural networks.

Possible Causes

• Incorrect Variable Usage: Users may mistakenly use kNONE instead of the correct TensorRT enumeration, trt.MatrixOperation.NONE.

• Data Type Mismatch: Attempting to pass NumPy arrays directly instead of TensorRT tensor objects leads to type incompatibility.

• Misconfiguration of Input Tensors: The inputs provided must be created using TensorRT’s API rather than standard NumPy arrays.

• Documentation Gaps: Insufficient guidance on how to properly define and use input tensors in the context of matrix multiplication may lead to confusion.

Troubleshooting Steps, Solutions & Fixes

1. Correct Variable Reference:

   • Ensure you are using trt.MatrixOperation.NONE instead of kNONE. Update your code as follows:

     layer = network.add_matrix_multiply(in1, trt.MatrixOperation.NONE, in2, trt.MatrixOperation.NONE)
     

2. Define Input Tensors Properly:

   • Instead of using NumPy arrays, create input tensors using TensorRT’s API:

     in1 = network.add_input("input1", dtype=trt.float32, shape=input_shape)
     in2 = network.add_input("input2", dtype=trt.float32, shape=input_shape2)
     

3. Review Function Signatures:

   • Make sure that the types of inputs match what is expected by the add_matrix_multiply() method:

     C = network.add_matrix_multiply(in1, trt.MatrixOperation.NONE, in2, trt.MatrixOperation.NONE)
     

4. Check for Errors:

   • If you encounter errors, review the specific error messages for clues on what might be wrong with your inputs or configurations.

5. Consult Documentation:

   • Refer to the official TensorRT documentation for examples and detailed explanations of matrix operations:
     TensorRT Matrix Multiply Documentation

6. Testing with Sample Code:

   • If issues persist, consider copying sample code directly from official examples or community forums where similar problems have been resolved successfully.

7. Best Practices:

   • Always ensure that input tensors are defined through TensorRT’s API.
   • Regularly check for updates or patches to the TensorRT library that may address known issues.

8. Further Investigation:

   • If problems remain unresolved after following these steps, consider reaching out to community forums for additional support or clarification on specific error messages encountered.

By adhering to these troubleshooting steps and solutions, users should be able to effectively resolve issues related to the use of the add_matrix_multiply() function on the Nvidia Jetson Orin Nano Dev board.