MB1 BL crash when rebooting to RCM from B slot

Issue Overview

Users are experiencing a crash of the MB1 bootloader when attempting to reboot the Nvidia Jetson Orin Nano Dev board into Recovery Mode (RCM) from an active boot slot labeled B. The crash occurs specifically during the loading of the MB2 Applet after executing the command sudo reboot forced-recovery. This issue has been replicated across different versions of Jetpack, including Jetpack 5 and Jetpack 6, using the sample root filesystem provided by Nvidia.

Symptoms

• The system fails to load the MB2 Applet, resulting in a crash.
• The issue is reproducible when booting from slot B after setting it as active.
• No issues arise when booting from slot A, regardless of whether it is set as active or not.

Context

• The problem occurs during the flashing process after a forced recovery reboot.
• Users have confirmed that this behavior persists across different hardware setups and configurations.

Hardware/Software Specifications

• Jetson Orin Nano Dev Kit with A/B slots enabled.
• Jetpack Versions: Jetpack 5 and Jetpack 6 (Jetson Linux R36.4).
• Commands Used: sudo nvbootctrl set-active-boot-slot 1, sudo nvbootctrl dump-slots-info, and sudo ./flash.sh jetson-orin-nano-devkit mmcblk0p1.

Frequency and Impact

The issue appears consistently when trying to flash from an active slot B setup, significantly impacting user experience by preventing successful flashing and recovery operations.

Possible Causes

1. Hardware Incompatibilities or Defects:

   • Potential issues with the specific hardware configuration or defects in the board could lead to failures during bootloader operations.

2. Software Bugs or Conflicts:

   • Bugs in the Jetpack software versions could cause instability during the flashing process, particularly in transition states like recovery mode.

3. Configuration Errors:

   • Incorrect configurations related to boot slots or root filesystem settings could lead to crashes during recovery operations.

4. Driver Issues:

   • Outdated or incompatible drivers may fail to support the required functionalities during boot processes.

5. Environmental Factors:

   • Power supply inconsistencies or overheating may contribute to unexpected crashes during critical operations.

6. User Errors or Misconfigurations:

   • Missteps in command execution or misunderstanding of boot processes could exacerbate the situation.

Troubleshooting Steps, Solutions & Fixes

Step-by-Step Diagnosis

1. Check Boot Configuration:

   • Execute cat /etc/nv_boot_control.conf to verify current configurations.

2. Dump Slot Information:

   • Use sudo nvbootctrl dump-slots-info and sudo nvbootctrl -t rootfs dump-slots-info to gather detailed information about active slots.

3. Reproduce the Issue:

   • Follow these steps:
     1. Flash with A/B slots enabled using Jetson Linux R36.4.
     2. Boot and perform initial configuration.
     3. Set active boot slot using sudo nvbootctrl set-active-boot-slot 1.
     4. Reboot and repeat initial configuration.
     5. Check slot info again with sudo nvbootctrl dump-slots-info.
     6. Reboot into recovery mode using sudo reboot forced-recovery.

Isolation Methods

• Test with different hardware configurations (e.g., using an SD card instead of NVMe).
• Attempt flashing with only boot-chain A/B enabled without root filesystem A/B.

Recommended Fixes

1. Use Slot A for Recovery:

   • If possible, always initiate recovery from an active slot A to avoid crashes associated with slot B.

2. Firmware Updates:

   • Regularly check for firmware updates that may address known issues with bootloader stability.

3. Flashing Commands:

   • Ensure correct usage of flashing commands; consider using sudo ./tools/kernel_flash/l4t_initrd_flash.sh --initrd jetson-orin-nano-devkit internal as an alternative approach.

4. Documentation Review:

   • Refer to Nvidia’s official documentation for any updates regarding known issues with flashing processes on specific versions.

5. Community Support:

   • Engage with community forums for shared experiences and solutions that have worked for others facing similar issues.

Best Practices for Prevention

• Always ensure that your development environment is updated with the latest software patches.
• Maintain a backup of working configurations before making significant changes to system settings or software versions.

Unresolved Aspects

Further investigation may be required into specific hardware configurations that consistently fail during this process, as well as potential updates from Nvidia regarding software stability improvements in future releases of Jetpack or related tools.

This structured approach aims to assist users in diagnosing and resolving issues related to MB1 crashes when rebooting into recovery mode on the Jetson Orin Nano Dev board effectively.