Nvidia Jetson Orin Nano Dev Board: Bootloader Issues During Upgrade

Issue Overview

Users of the Nvidia Jetson Orin Nano Dev board are experiencing bootloader issues when upgrading from R35.5.0 to R36.3.0 or R36.4.0. The problem manifests as the system getting stuck at the bootloader stage, displaying an error message related to TOS (Trusted OS) digest mismatch. This issue occurs during both manual flashing and Over-The-Air (OTA) updates. The problem seems to be intermittent, as some users have reported success after multiple attempts.

Possible Causes

1. Incompatibility between versions: The transition from R35 (Kernel 5.10) to R36 (Kernel 5.15) may introduce compatibility issues that affect the bootloader.

2. Secure boot configuration: The issue appears to be related to secure boot settings, as evidenced by the RSA PSS signature check and TOS digest mismatch errors.

3. Incorrect update procedure: Users attempting manual upgrades or custom OTA methods may be missing crucial steps in the update process.

4. Hardware-specific issues: The problem might be related to specific hardware configurations or SOM (System on Module) variants with secure boot enabled.

5. Firmware or driver inconsistencies: The update process may not be correctly updating all necessary firmware components or drivers.

Troubleshooting Steps, Solutions & Fixes

1. Use official OTA method:

   • Follow the official NVIDIA developer guide for Over-the-Air Updates when upgrading from R35.5 to R36.3.
   • This method ensures all necessary steps are followed and required services are executed.

2. Rebuild EKS image:
   If using custom keys, rebuild the eks_t234.img file:

   python3 source/optee/samples/hwkey-agent/host/tool/gen_ekb/gen_ekb.py -chip t234 -oem_k1_key oem_k1.key -in_sym_key sym_t234.key -in_sym_key2 sym2_t234.key -in_auth_key auth_t234.key -out bootloader/eks_t234.img
   

   Ensure all key files are correctly placed and accessible.

3. Check and update QSPI partitions:

   • The nv-l4t-bootloader-config.service is crucial for checking partition layout and updating QSPI partitions.
   • Ensure this service runs correctly during the upgrade process.

4. Capsule update for Orin NX:
   If using an Orin NX, apply the capsule payload patch mentioned in the Orin NX hang issue thread.

5. Skip OEM config:
   Try skipping the OEM configuration setup if the issue persists.

6. Verify hardware authorization:
   Confirm the board authorization types (PKC, SBK, etc.) and ensure they are correctly configured for the new version.

7. Clean boot failure and retry:
   If the upgrade fails:

   • Clean the boot failure of chain B
   • Reboot the system
   • Attempt the upgrade process again

8. Manual upgrade steps (use with caution):
   If attempting a manual upgrade, ensure you follow these steps:

   • Extract rootfs to rootfs B partition
   • Use dd to write boot.img to boot B partition
   • Copy TEGRA_BL.cap to ESP (EFI System Partition)
   • Trigger update by setting EFI variable
   • Clean boot failure of chain B
   • Reboot
     Note: This method may not include all necessary steps and is not recommended over the official OTA method.

9. Downgrade considerations:

   • Downgrading from R36.3 to R35.5 is not supported via OTA
   • To downgrade, you must re-flash the target with the earlier release

10. Collect comprehensive logs:
    If the issue persists, collect detailed logs including:

    • Flash logs
    • UART logs
    • OTA upgrade logs
      These logs can help NVIDIA support diagnose the problem more effectively.

11. Check for UEFI menu:
    After a failed OTA, verify if the UEFI menu is accessible. Its absence may indicate a more severe bootloader issue.

12. Contact NVIDIA Support:
    If none of the above steps resolve the issue, reach out to NVIDIA support with your collected logs and a detailed description of your upgrade process and hardware configuration.