Monday 18 June 2012

Issues with OpenCV and RTSP

For months now I've been fighting with OpenCV to try and get a decent image coming through my internet camera and it's really beginning to drive me mad. I've got a reasonable little program to test the connection which looks like this:
/* 
 * File:   main.cpp
 * Author: Ryan
 *
 * Created on October 27, 2011, 2:23 PM
 */
#include <cstdlib>
#include "opencv2/core/core.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/contrib/contrib.hpp"
#include "opencv2/features2d/features2d.hpp"
#include "opencv2/legacy/legacy.hpp"
#include "opencv2/video/tracking.hpp"
#include <stdio.h> 

using namespace std;
using namespace cv;

int main() {
// CvCapture *camera = cvCreateFileCapture("/home/ryan/NetBeansProjects/OpenRTSP/video-H264-1");
// CvCapture *camera = cvCreateFileCapture("stream_fifo");    
// CvCapture *camera = cvCreateFileCapture("Wildlife.wmv");
   CvCapture *camera = cvCreateFileCapture("rtsp://admin:admin@192.168.1.128/ch1-s1"); //purple cam /?tcp
// CvCapture *camera=cvCreateFileCapture("http://192.168.1.5/image.jpg"); //black cam
// CvCapture *camera = cvCreateCameraCapture(0); //built in cam
// cvWaitKey(10000); 
    if (camera == NULL) {
        printf("camera is null, aborting...");
        return -1;
    }
    printf("camera is not null\n");
    fflush(stdout);
    cvNamedWindow("img");
    int i = 0;
    while (cvWaitKey(100) != 27) {
 // CvCapture *camera=cvCreateFileCapture("http://192.168.1.5/image.jpg");
        IplImage *img = cvQueryFrame(camera);
            if (img == NULL) break;
        cvShowImage("img", img);
// cvReleaseCapture(&camera); 
// printf("Image: %i\n", ++i);
        }
    cvReleaseCapture(&camera);
    return 0;
}



You'll probably notice in there quite a few lines of commented code, this is just me trying out different ways of creating camera connections for different cameras and also a few little tests on reading from file. I know this code works as it loads up and plays the 'Wildlife.wmv' nicely, albeit without sound but that is expected and I can live without sound for now.

The problem comes when I try and play the video at the end of the RTSP link. Everything is set properly in the URL 'rtsp://admin:admin@192.168.1.128/ch1-s1' which follows the 'rtsp://username:password@ip.address.to.media/channel-stream' format and the OpenCV documentation says that this should work but what results is the most broken, artifact-riddled mess I've ever had the misfortune to lay eyes on. I mean look at this:

What on earth am I meant to do with that?

Whilst opening the stream in Quicktime gives me this (and notice this is after 5 hours of continuous streaming and the image has hardly faltered):

Even FFMPEG, the underlying technology used by OpenCV for its RTSP support gives me a decent stream even though it does seem to crash after 10 seconds (created using the command 'ffplay rtsp://admin:admin@192.168.1.128/ch1-s1' more on the crashing later perhaps):

And finally we have the camera's own video player which is embedded in the html (from what I can see it is an OCX plugin calling on a few .dlls). Perhaps unsurprisingly this gives a perfect stream as well:

So why in the world does OpenCV make such a mess of it? I'm still trying to find out...

Filed under in-betweeners as the connection does actually work, it's just rubbish!

4 comments:

  1. Hey Ryan, I am currently experiencing a similar issue with OpenCV 2.4.3 using an Axis IP camera and RTSP protocol on both Windows and Linux.

    Using "ffplay rtsp:/user:password@192.168.2.111/mpeg4/media.amp" the image becomes corrupted after a short period of time. If you enable tcp as the transport mechanism the image is displayed correctly however there is a noticeable lag in the image update/display.

    "ffplay -rtsp_transport rtsp://user:password@192.168.2.111/mpeg4/media.amp"

    Apparently the ability to tag the tcp parameter to the end of the rtsp url has been remove from recent versions of ffmpeg/opencv. By recompiling the ffmpeg library back into opencv it would be possible to change the default transport property. However I need to process the image in realtime so the significant delay in decoding the image with tcp enabled is not an acceptable solution. Regards,XVision

    ReplyDelete
    Replies
    1. Thanks for the heads up.

      Hackeron has posted some helpful comments on this post http://workingwithcomputervision.blogspot.co.uk/2012/07/weve-finally-got-it-going-ill-get-onto.html which does exactly what you suggest I recommend people try that to see if it helps. It's a shame it doesn't in your case though,

      Delete
  2. Hello,
    I know it's a 2 years old post, but I would like to know if you found a great solution ?
    I have to deal with IP camera with rtsp but I encounter the same problems: lots of artefacts or the crash after 10 seconds...
    thanks for your help.

    ReplyDelete
    Replies
    1. Best I ever got is posted on http://workingwithcomputervision.blogspot.co.uk/2012/07/weve-finally-got-it-going-ill-get-onto.html

      Delete