Error on following “NanoVLM – Efficient Multimodal Pipeline”

Issue Overview

Users have reported encountering an "Exception error" while following the tutorial for Small Language Models (SLM) with the Jetson Orin Nano (8GB) using a 128GB SD card. The issue arises specifically when executing the command to run the NanoVLM model, resulting in a few minutes of screen freezing followed by an error message.

Symptoms and Errors

• Screen Freezing: Users experience temporary freezing of the screen during execution.

• Error Message: The terminal outputs a series of warnings and a final exception indicating that the model configuration lacks information about the maximum sequence length. The relevant part of the error log is:

  Exception: The model config should contain information about maximum sequence length.
  

Context

• Hardware Specifications: Jetson Orin Nano with 8GB RAM and 128GB SD card.
• Software Specifications:
  • L4T Version: 35.5.0
  • Jetpack Version: 5.1
  • CUDA Version: 11.4
• Frequency: This issue appears to be consistently reproducible when following the tutorial steps.

Impact

The problem significantly hinders users’ ability to utilize the NanoVLM tutorial effectively, preventing them from running multimodal pipelines as intended.

Possible Causes

1. Model Configuration Issues:

   • The error indicates that the model configuration file does not specify the maximum sequence length, which is crucial for model operation.

2. Software Bugs or Conflicts:

   • Warnings related to version mismatches in dependencies (e.g., urllib3 and chardet) may suggest underlying compatibility issues.

3. Environmental Factors:

   • Insufficient system resources (like RAM or storage) could lead to freezing during intensive operations.

4. User Errors or Misconfigurations:

   • Incorrect command syntax or parameters may lead to unexpected behavior.

Troubleshooting Steps, Solutions & Fixes

Step-by-Step Instructions

1. Verify Model Configuration:

   • Check the model configuration file for entries related to maximum sequence length. Ensure it includes a line similar to:

     "max_seq_len": 256
     

2. Update Dependencies:

   • Run the following command to ensure all dependencies are up-to-date:

     pip install --upgrade requests transformers
     

3. Check System Resources:

   • Monitor system resources while running the command using:

     top
     

   • Ensure that there is sufficient RAM and CPU available.

4. Test with Simplified Command:

   • Try running a simpler version of the command to isolate issues:

     jetson-containers run $(autotag nano_llm) python3 -m nano_llm.chat --api=mlc --model Efficient-Large-Model/VILA1.5-3b
     

5. Adjust Command Parameters:

   • Experiment with different values for --max-context-len and --max-new-tokens in case they are causing conflicts.

6. Check Docker Configuration:

   • Ensure Docker is properly configured for GPU access and that all necessary volumes are mounted correctly.

7. Review Documentation and Updates:

   • Consult the official documentation for any updates or patches related to NanoVLM or Jetson Orin Nano.

8. Reinstall MLC Framework:

   • If issues persist, consider reinstalling the MLC framework using:

     pip uninstall mlc_llm
     pip install mlc_llm
     

Recommended Approach

Multiple users have found success after ensuring their model configuration files are correctly set up with required parameters, particularly regarding maximum sequence length. This adjustment is highly recommended as a first step in troubleshooting.

Best Practices

• Always verify that your software dependencies are compatible with your hardware specifications.
• Regularly check for updates in both software libraries and models used in projects.
• Maintain a backup of configuration files before making significant changes.

Unresolved Aspects

Further investigation may be needed into potential compatibility issues between different versions of libraries used in conjunction with Jetson Orin Nano, especially if dependency warnings persist after updates.