Issues with nvjpegdec on Jetson Orin Nano and Xavier NX for MJPEG HTTP Streams

Issue Overview

Users are experiencing problems when attempting to connect to MJPEG encoded live HTTP streams using the gstreamer nvjpegdec plugin on Jetson Orin Nano and Jetson Xavier NX devices. The issues vary depending on the output format:

1. With I420 (default) and NV12 formats:

   • The stream freezes after displaying the first frame
   • Hardware utilization (NVJPG0/NVJPG1) appears normal during initialization

2. With RGBA format:

   • The stream flows without freezing
   • The image output is incorrectly displayed as red

These problems occur on JetPack 5.1.2 (r35.4.1) but not on JetPack 5.0.2 (r35.1.0), suggesting a potential bug in the newer version.

Possible Causes

1. Bug in JetPack 5.1.2: The issue appears to be specific to this version, as it doesn’t occur in JetPack 5.0.2.

2. Hardware JPEG decoding issue: A known problem with NvJPEGDecoder has been reported, where it generates the same output if called twice with different input buffers.

3. Incompatibility between nvjpegdec and MJPEG streams: The plugin may not be correctly handling the MJPEG format in the newer JetPack version.

4. Color space conversion problem: The red output in RGBA format suggests an issue with color space conversion or interpretation.

Troubleshooting Steps, Solutions & Fixes

1. Downgrade to JetPack 5.0.2:

   • If possible, revert to JetPack 5.0.2 (r35.1.0) where the issue doesn’t occur.
   • This is a temporary workaround until the problem is resolved in newer versions.

2. Use nvv4l2decoder instead of nvjpegdec:

   • For JetPack 5.1.2, replace nvjpegdec with nvv4l2decoder and add the mjpeg=1 parameter.
   • Modified pipeline example:

     gst-launch-1.0 souphttpsrc location=http://31.160.161.51:8081/mjpg/video.mjpg do-timestamp=1 blocksize=4294967294 http-log-level=0 is-live=1 ! queue ! jpegparse ! nvv4l2decoder mjpeg=1 ! 'video/x-raw(memory:NVMM),format=I420' ! queue ! nvvidconv ! videoconvert ! queue ! xvimagesink sync=false
     

3. Monitor hardware JPEG decoding:

   • Use NVIDIA’s debugging tools to monitor the NVJPG0/NVJPG1 hardware utilization during stream decoding.
   • Look for any anomalies or errors in the decoding process.

4. Test with different MJPEG sources:

   • Try connecting to various MJPEG HTTP streams to determine if the issue is specific to certain sources.

5. Investigate color space conversion:

   • For the RGBA red output issue, check if adding explicit color space conversion elements in the pipeline resolves the problem.
   • Example:

     gst-launch-1.0 souphttpsrc location=http://31.160.161.51:8081/mjpg/video.mjpg do-timestamp=1 blocksize=4294967294 http-log-level=0 is-live=1 ! queue ! jpegparse ! nvjpegdec rgbaoutput=1 ! 'video/x-raw(memory:NVMM),format=RGBA' ! queue ! nvvidconv ! video/x-raw,format=RGBA ! videoconvert ! video/x-raw,format=RGB ! queue ! xvimagesink sync=false
     

6. Report the issue:

   • If the problem persists, report it to NVIDIA’s developer forums or support channels.
   • Provide detailed information about your setup, JetPack version, and the steps to reproduce the issue.

7. Stay updated:

   • Keep an eye on the reported issue: "NvJPEGDecoder generates the same output if called twice with different input buffer"
   • Check for updates or patches from NVIDIA that address this problem.

8. Consider alternative decoding methods:

   • If hardware decoding continues to be problematic, explore software-based JPEG decoding options as a temporary workaround.

Remember to test each solution thoroughly and monitor system performance, as some workarounds may impact overall system efficiency or introduce other issues.