Unable to boot the customised kernel image

Issue Overview

Users have reported difficulties booting a customized kernel image on the Nvidia Jetson Orin Nano Developer Kit. The primary symptoms include:

• Boot Failure: After replacing the default kernel image with a custom one, users experience issues during the boot process, often indicated by mounting errors.
• Kernel Configuration Issues: Users noted that they did not set the CONFIG_LOCALVERSION correctly, which led to complications in recognizing the new kernel image.
• GadgetFS Mounting Errors: After successfully booting with the customized kernel, users encountered an error when attempting to mount GadgetFS, receiving the message: mount: /dev/gadget: unknown filesystem type ‘gadgetfs’.

The problem typically arises after users have followed a series of steps to build and install a custom kernel from source. The context of the issue is primarily during the setup and configuration of the kernel, particularly after modifications to kernel parameters and installation of modules.

The hardware specifications mentioned include:

• Nvidia Jetson Orin Nano Developer Kit
• NVMe storage

The frequency of these issues appears consistent among users attempting similar customizations, significantly impacting their ability to utilize specific functionalities provided by the customized kernel.

Possible Causes

Several potential causes for the booting and mounting issues have been identified:

• Kernel Configuration Errors: Not setting CONFIG_LOCALVERSION properly can lead to the system failing to recognize or boot into the custom kernel.

• Incomplete Module Installation: If all necessary modules are not built and installed alongside the kernel, this can result in drivers being unavailable during boot.

• Incorrect Boot Configuration: Modifications made to extlinux.conf may not be sufficient if other configurations related to device UUIDs or partitions are not addressed.

• Driver Issues: Missing drivers for NVMe or other components required for booting or mounting filesystems can prevent successful initialization.

• Environmental Factors: Power supply issues or overheating may also contribute to instability during boot processes.

• User Errors: Misconfigurations during the installation process or incorrect command usage can lead to these issues.

Troubleshooting Steps, Solutions & Fixes

To address these issues effectively, follow these comprehensive troubleshooting steps:

1. Verify Kernel Configuration:

   • Ensure that you have run:

     make target_defconfig
     

   • Set CONFIG_LOCALVERSION correctly. For example:

     export CONFIG_LOCALVERSION="-demo"
     

2. Build and Install Modules:

   • After making changes in menuconfig, ensure all modules are built and installed:

     make
     sudo make modules_install
     

3. Check Boot Configuration:

   • Review your extlinux.conf file for accuracy. It should point to your new kernel image correctly.
   • Ensure that you have specified any necessary initial ramdisk (initrd) options if required.

4. Gather Boot Logs:

   • Connect a serial console to capture detailed boot logs. This can help identify where the boot process fails.
   • Use a USB UART connected to header pins (8 and 10) for logging if available.

5. Test Different Configurations:

   • If using NVMe, ensure that all drivers required for NVMe support are included in your build.
   • Consider testing with a simpler configuration or reverting to the original kernel to isolate whether the issue is with your customizations or hardware setup.

6. Mount GadgetFS Correctly:

   • Confirm whether GadgetFS is indeed available as a filesystem type. If it is not recognized, check if it needs additional configuration in /etc/fstab.
   • Use:

     sudo mkdir /dev/gadget
     sudo mount -t gadgetfs gadgetfs /dev/gadget
     

   • If you receive an error about an unknown filesystem type, verify that the gadgetfs module is loaded correctly:

     sudo modprobe gadgetfs
     

7. Review Documentation and Updates:

   • Check for any available updates for Jetpack or specific driver updates that might address known issues.
   • Refer to Nvidia’s official documentation for guidance on building kernels and installing modules specific to Jetson devices.

8. Best Practices for Future Prevention:

   • Always backup original configurations before making changes.
   • Document each step taken during customization for easier troubleshooting in future attempts.
   • Engage with community forums or Nvidia support for insights on common pitfalls encountered by other users.

By following these steps, users should be able to diagnose and resolve issues related to booting customized kernels on their Nvidia Jetson Orin Nano Developer Kit effectively. Further investigation may be needed if problems persist after these troubleshooting efforts.