Fail used linux pcie test driver pci-epf-test on nano platform

Issue Overview

Users are experiencing issues with the Nvidia Jetson Orin Nano Dev board when attempting to use the PCIe endpoint test driver (pci-epf-test) for transmitting video data over a 10G connection to an x86 PC. The specific symptoms include repeated "Unhandled context fault" errors related to the ARM System Memory Management Unit (SMMU) and memory controller (MC) errors, particularly when using Direct Memory Access (DMA). The errors occur during data transmission, with successful transfers noted only when DMA is not utilized. The issue appears consistently across multiple attempts, significantly impacting user experience by hindering data transfer capabilities essential for applications requiring high throughput.

The hardware involved includes the Nvidia Jetson Orin Nano platform running kernel version 5.10 and an x86 PC. The problem arises when the Orin Nano operates in Endpoint (EP) mode while the x86 PC functions in Root Port (RP) mode. Users have reported that they can run the pcitest utility successfully without DMA, indicating some underlying issues specifically related to DMA operations.

Possible Causes

1. Hardware Incompatibilities or Defects:

   • The interaction between the Orin Nano and the x86 PC may reveal hardware-level incompatibilities, particularly concerning PCIe configurations.

2. Software Bugs or Conflicts:

   • There may be unresolved bugs in the kernel’s DMA subsystem affecting how the Orin Nano handles DMA requests.

3. Configuration Errors:

   • Incorrect device tree configurations or missing parameters could lead to improper initialization of DMA channels.

4. Driver Issues:

   • Outdated or incompatible drivers may not support the required functionalities for efficient DMA operations.

5. Environmental Factors:

   • Power supply issues or thermal conditions could affect device performance, particularly under high load situations like video data transmission.

6. User Errors or Misconfigurations:

   • Misconfigurations during setup could lead to unexpected behavior, especially with complex systems like PCIe setups.

Troubleshooting Steps, Solutions & Fixes

1. Diagnosing the Problem:

   • Use the following command to check for any relevant logs that might provide more insight into the errors:

     dmesg | grep -i "error"
     

2. Gathering System Information:

   • Collect detailed system information using:

     lspci -vvv
     

3. Isolating the Issue:

   • Test with different PCIe configurations (e.g., switching between EP and RP modes) to determine if the issue persists.
   • Try using different PCIe cables or slots if available.

4. Configuration Adjustments:

   • Review and edit device tree entries related to DMA and IOMMU settings as follows:

     // Example modifications
     iommus = <&smmu_niso0 TEGRA_SID_NISO0_PCIE4>;
     iommu-map = <0x0 &smmu_niso0 TEGRA_SID_NISO0_PCIE4 0x1000>;
     dma-coherent;
     

   • Ensure that all necessary parameters are correctly set in your device tree source files.

5. Driver Updates:

   • Check for updates to relevant drivers and install them using:

     sudo apt-get update && sudo apt-get upgrade
     

6. Testing with Sample Code:

   • Utilize sample code from pci-epf-dma-test.c as a reference for implementing your own tests without relying solely on kernel APIs.

7. Best Practices for Future Prevention:

   • Regularly update software and firmware to ensure compatibility.
   • Maintain detailed documentation of configuration changes to facilitate easier troubleshooting in future instances.

8. Recommended Approach:

   • If multiple users report success with specific configurations or updates, prioritize those solutions in your troubleshooting efforts.

9. Unresolved Aspects:

   • Further investigation is needed regarding specific errors related to "Unexpected global fault" and "MC request violates VPR requirements," as these may require deeper kernel-level debugging or patches from Nvidia.

By following these structured steps, users can systematically approach and potentially resolve issues related to PCIe operations on the Nvidia Jetson Orin Nano platform.