Pixel Shift Issue with IMX219 CSI Camera on Nvidia Jetson Orin Nano Dev Board

Issue Overview

Users of the Nvidia Jetson Orin Nano Development Kit are experiencing an image coherence problem when using IMX219-based CSI cameras. The issue manifests as a misalignment between odd and even rows in the captured images, requiring a 16-pixel shift to the right for odd rows to achieve proper alignment. This problem is specific to certain sensor modes and does not occur on older Xavier NX boards running R32 release. However, the issue has been confirmed on Xavier NX boards running R35 release, suggesting a software-related cause rather than a hardware-specific problem.

Key points:

• Affects Orin Nano Development Kit and Xavier NX with R35 release
• Occurs with IMX219 CSI cameras in specific sensor modes (0 and 1)
• Requires a 16-pixel shift for odd rows to correct the image
• Does not affect all sensor modes (e.g., 1920×1080 works correctly)
• Impacts raw image capture but not GStreamer pipeline output

Possible Causes

1. VI Alignment Limitation: The most likely cause is a 32-byte alignment limitation in the Video Input (VI) subsystem of the R35 release. This alignment requirement may be causing the pixel shift in certain sensor modes.

2. Driver Incompatibility: The camera driver for the IMX219 sensor may not be fully compatible with the R35 release, leading to incorrect handling of certain sensor modes.

3. Firmware Mismatch: There could be a mismatch between the camera firmware and the R35 release, causing improper interpretation of sensor data.

4. Software Bug: A bug in the R35 release’s camera handling code could be causing the misalignment for specific sensor modes.

5. Hardware Differences: Although less likely, there might be subtle hardware differences between the Orin Nano and Xavier NX that interact with the R35 release to produce this issue.

Troubleshooting Steps, Solutions & Fixes

1. Verify Sensor Mode Compatibility:

   • Test different sensor modes to identify which ones are affected.
   • Use the following command to list available formats:

     v4l2-ctl --list-formats-ext
     

   • Note that sensor mode 0 (3280×2464) and mode 1 (3280×1848) are affected, while mode 2 (1920×1080) works correctly.

2. Implement Pixel Shift Correction:

   • For affected sensor modes, implement a 16-pixel shift correction for odd rows.
   • Use the provided Python script as a reference for the correction:

     shifted_data = np.copy(data)
     for iRow in range(1, data.shape[0], 2):
         shifted_data[iRow] = np.roll(shifted_data[iRow], 16)
     

3. Ensure 32-byte Alignment:

   • When capturing images, ensure that the output width is a multiple of 32 pixels.
   • If necessary, pad the image width to meet this alignment requirement.

4. Use GStreamer for Real-time Applications:

   • As GStreamer pipelines are not affected by this issue, consider using GStreamer for real-time video processing instead of raw capture.

5. Check for Driver Updates:

   • Regularly check for updates to the Jetson BSP (Board Support Package) that might address this issue.
   • Install any available updates using the SDK Manager.

6. Report the Issue:

   • Report this issue to NVIDIA’s developer forums or support channels, providing detailed information about your setup and the observed behavior.

7. Workaround for Specific Sensor Modes:

   • For sensor mode 0 (3280×2464) and mode 1 (3280×1848), use the following command to capture raw frames:

     v4l2-ctl -d /dev/video0 -c override_enable=1 -c bypass_mode=0 \
     --set-ctrl sensor_mode=0 --set-fmt-video=width=3280,height=2464,pixelformat=RG10 \
     -c exposure=33333 -c frame_rate=10000000 \
     --stream-mmap --stream-count=1 --stream-to=frame.raw
     

   • Then use the provided Python script to correct the image:

     ./imgfromraw.py 3280 2464 16
     

8. Consider Alternative Sensor Modes:

   • If possible, use sensor mode 2 (1920×1080) which does not exhibit the shift issue.

9. Monitor for Firmware Updates:

   • Keep an eye out for any firmware updates for the IMX219 camera module that might address this issue.

10. Investigate Hardware-level Workarounds:

    • If software solutions prove insufficient, consult with NVIDIA support about potential hardware-level workarounds or alternative camera modules that may not exhibit this issue.