Shadow Artifact and Gain Issues with IMX728 Module on Nvidia Jetson Orin Nano

Issue Overview

Users of the Nvidia Jetson Orin Nano development board are experiencing problems when integrating the IMX728 camera module. The main issues reported are:

1. Partial shadow artifacts appearing at light sources when the lens is moved quickly during preview using argus_camera.
2. The sensor’s gain is automatically set to the maximum value instead of maintaining the default setting.

These issues occur specifically when shooting light sources and moving the lens rapidly. The problem is observed in the High Conversion Gain (HCG) mode of the IMX728 sensor, which is being used in linear mode.

Possible Causes

1. Incorrect Sensor Mode Configuration: The use of High Conversion Gain mode in combination with linear mode might be causing unexpected behavior in the sensor’s response to light.

2. Driver Implementation Issues: The IMX728 driver may not be correctly implementing gain control, leading to the maximum gain setting.

3. Argus Camera Software Bugs: There could be issues in the argus_camera software that incorrectly handle rapid lens movement or gain settings.

4. Hardware Limitations: The IMX728 module might have physical limitations in handling rapid movements while maintaining proper exposure, especially in high-gain scenarios.

5. Improper Integration: The adaptation of the IMX728 module to the Orin Nano platform may not be optimized, leading to unexpected behavior.

6. Environmental Factors: Extreme lighting conditions combined with rapid movement might be pushing the sensor beyond its designed operating parameters.

Troubleshooting Steps, Solutions & Fixes

1. Disable High Conversion Gain Mode:

   • Try disabling the HCG mode to see if the issue persists.
   • Modify the sensor configuration to use standard conversion gain.

2. Use Dummy Gain Values:

   • Implement a test where you manually set fixed gain values.
   • Observe if the shadow artifacts still occur with controlled gain settings.

3. Verify Sensor Sanity Test:

   • Contact the sensor vendor to ensure the IMX728 has passed all sanity tests, especially for the HCG mode in combination with linear mode.
   • Request any available documentation on known issues or limitations.

4. Check Driver Implementation:

   • Review the IMX728 driver code, focusing on gain control mechanisms.
   • Ensure that the driver is not inadvertently setting the gain to maximum.
   • Implement logging to track gain changes during operation.

5. Argus Camera Debugging:

   • Enable verbose logging in argus_camera to track sensor behavior during rapid movements.
   • Check for any error messages or warnings related to gain settings or exposure control.

6. Test with Different Scenes:

   • Systematically test the camera in various lighting conditions and movement speeds.
   • Document the conditions under which the issue occurs and those where it doesn’t.

7. Firmware and Software Updates:

   • Ensure that you are using the latest firmware for the IMX728 module.
   • Update the Jetson Orin Nano’s system software and relevant libraries to the latest versions.

8. Alternative Mode Testing:

   • If possible, test the sensor in SDR (Standard Dynamic Range) mode to compare performance.
   • Experiment with different combinations of gain modes and dynamic range settings.

9. Hardware Inspection:

   • Verify that the IMX728 module is properly connected and seated on the Orin Nano board.
   • Check for any signs of physical damage or loose connections.

10. Performance Profiling:

    • Use system monitoring tools to check CPU and GPU usage during camera operation.
    • Ensure that the system is not overloaded, which could affect camera performance.

11. Consult Nvidia Developer Forums:

    • Post detailed findings and logs on the Nvidia Developer Forums for expert assistance.
    • Include information about all tests performed and their outcomes.

If the issue persists after trying these steps, consider opening a new support ticket with Nvidia, providing comprehensive details about the problem and all troubleshooting steps attempted. Additionally, reaching out to the IMX728 module manufacturer for specific guidance on integration with the Jetson platform may yield further insights or solutions.