GPU Memory Limitations and NVME Usage on Jetson Orin Nano

Issue Overview

Users of the Nvidia Jetson Orin Nano are experiencing limitations when attempting to expand GPU memory using NVME SSDs. The main concerns revolve around the compatibility of NVMEs with the Orin Nano, the potential benefits of using higher-speed NVMEs, and the possibility of utilizing NVME storage as additional GPU memory. Users are particularly interested in using GPUDirect Storage to connect the GPU and NVME for processing large datasets, such as depth maps from stereo cameras, without encountering bottlenecks during data transfer between CPU and GPU.

Possible Causes

1. Hardware limitations: The Jetson Orin Nano may have specific hardware constraints that prevent direct GPU access to NVME storage.

2. Software incompatibility: The current software stack on the Orin Nano might not support GPUDirect Storage for NVMEs as initially thought.

3. Misunderstanding of GPUDirect capabilities: There may be confusion about the exact functionality and limitations of GPUDirect on the Jetson platform.

4. Insufficient documentation: Unclear or outdated documentation may lead to misinterpretations about the capabilities of the Orin Nano regarding NVME and GPU interactions.

Troubleshooting Steps, Solutions & Fixes

1. Understand NVME compatibility:

   • The Jetson Orin Nano supports PCIe 3.0 NVMEs.
   • There is no specific list of compatible NVMEs provided, but any PCIe 3.0 NVME should work.

2. NVME speed considerations:

   • Using NVMEs with capabilities beyond PCIe 3.0 will not provide additional speed benefits, as the Orin Nano only supports PCIe 3.0.

3. GPU memory expansion:

   • Unfortunately, NVME storage cannot be directly used as additional GPU memory on the Jetson Orin Nano.
   • The GPU cannot access NVME storage as extended memory.

4. Alternative memory expansion:

   • NVME storage can be set up as swap space for CPU memory extension, but this does not apply to GPU memory.
   • If your use case requires significant memory, consider upgrading to the Orin Nano 8GB model.

5. OS installation on NVME:

   • While not directly addressed in the discussion, it is generally possible to install the Jetson OS on an NVME drive.
   • You can use both the flashed microSD card and NVME storage in conjunction, typically with the OS on one and data on the other.

6. GPUDirect Storage clarification:

   • The document mentioning GPUDirect support from CUDA 11.4 onwards on Jetson Orin platforms refers to FPGA devices, not NVME SSD storage.
   • For more information on GPUDirect RDMA on Jetson platforms, refer to:
     https://github.com/NVIDIA/jetson-rdma-picoevb

7. Alternative approaches for large dataset processing:

   • Consider optimizing your algorithms to work with smaller chunks of data that fit in the available GPU memory.
   • Implement efficient data streaming techniques to process large datasets in parts.
   • Explore using multiple Jetson Orin Nano devices in a cluster for distributed processing.

8. Stay updated:

   • Regularly check the official Nvidia Jetson forums and documentation for any updates or new features that may address these limitations in future software releases.

While the current hardware and software configuration of the Jetson Orin Nano does not support using NVME storage as extended GPU memory, understanding these limitations can help in designing more efficient solutions within the available resources.