OpenCV Basics parviz

OpenCV Basics Header files

o #include o #include o #include o #include

Image data types

o IPL_DEPTH_(S|U|F) Here possible values for are 1,8,16,32 and 64 S = Signed U = Unsigned F = Float 1 bit depth images should be unsigned 8 bit depth images should be signed or unsigned 16 bit depth images should be signed or unsigned 32 bit depth images should be signed or float 64 bit depth images should be float

o E.g.: IPL_DEPTH_16S means an 16-bit signed image IPL_DEPTH_64F means a 64-bit float image

Matrix data types

o CV_(S|U|F)C Here possible values for are 8,16,32 and 64 S = Signed integer U = Unsigned integer F = Float Here possible values for are 1,2,3,4 8 bit depth matrices should be signed or unsigned 16 bit depth matrices should be signed or unsigned 32 bit depth matrices should be signed or float 64 bit depth matrices should be float

o E.g.: CV_16SC1 means an 16-bit signed single-channel matrix CV_64FC3 means a 64-bit float matrix with 3 channels

Function naming conventions

o cvActionTargetMod(.....) Action = the functionality (e.g. load, show) Target = the target image area (e.g. contour, image) Mod = optional modifiers (e.g. argument type)

o E.g.: cvLoadImage(...) cvFitLine(...)

Capturing Images & Videos Simple OpenCV Program-Displaying an Image from File

Open Microsoft Visual Studio and create a new win32 console application. Then write the following program. If you have not install and configure OpenCV yet, please refer to Installing & Configuring with Visual Studio. /////////////////////////////////////////////////////////////////////////////////////// #include "stdafx.h" #include #include int main() { IplImage* img = cvLoadImage("C:/MyPic.jpg"); cvNamedWindow("MyWindow"); cvShowImage("MyWindow", img); cvWaitKey(0); cvDestroyWindow("MyWindow"); cvReleaseImage(&img); return 0;

} /////////////////////////////////////////////////////////////////////////////////////// Before you run this program, put any image(MyPic.jpg) in C: drive. You can download this OpenCV visual c++ project from here. (The downloaded file is a compressed .rar folder. So, you have to extract it using Winrar or other suitable software)

Result of above OpenCV Application

Explanation

Let's see the new OpenCV functions, found in the above example.

#include "stdafx.h"

#include

#include

These 3 lines are pre-processor directives to include some header files for your application. In almost every program you have to insert these 3 line at the top of your program.

IplImage* img = cvLoadImage("C:/MyPic.jpg");

IplImage is a data structure to store images in the memory. cvLoadImage(...) is a function which loads an image from the location "C:/MyPic.jpg". Supported image formats of cvLoadImage function are BMP, DIB, JPEG, JPG, JPE, PNG, PBM, PGM, PPM, SR, RAS, TIFF and TIF. By default, the loaded image is forced to be a 3-channel color image. This default can be modified by using a flag variable inside the cvLoadImage function. Then IplImage* img = cvLoadImage("C:/MyPic.jpg" , flag);

if flag: >0 the loaded image is forced to be a 3 channel color image =0 the loaded image is forced to be a 1 channel grayscale image

Result of above OpenCV Application

New OpenCV functions which are not found earlier are explained here

cvCreateImage(cvSize(800,500),IPL_DEPTH_8U,1);

This function create a 8 bit depth unsigned single-channel image which has a hight of 800 pixels and width of 500 pixels. The first argument, cvSize(800,500) defines the height and width of the image.

The 2nd argument, IPL_DEPTH_8U defines the bit_depth of the image. Here bit_depth is unsigned 8 bits. Here'U' is stands for 'unsigned'. For more clarification, visit OpenCV Basics.

The 3rd argument defines the number of channels. Here it is single channel.

cvZero(img);

This function assigns zero to all pixel values of the image which is pointed by the 'img variable. All other lines are same as the previous application. Summary

In this application, it is created an image and assigned zero to all pixel values. Therefore we can see a black image as the result of the application. Simple OpenCV Program-Playing a Video From a File

/////////////////////////////////////////////////////////////////////////////////////// #include "stdafx.h" #include #include

int main() { //load the video file to the memory CvCapture *capture = cvCaptureFromAVI("C:/Wildlife.avi");

if( !capture ) return 1;

//obtain the frames per seconds of that video int fps = ( int )cvGetCaptureProperty( capture, CV_CAP_PROP_FPS );

//create a window with the title "Video" cvNamedWindow("Video");

while(true) { //grab and retrieve each frames of the video sequencially IplImage* frame = cvQueryFrame( capture );

if( !frame ) break;

//show the retrieved frame in the "Video" window cvShowImage( "Video", frame );

int c;

if(fps!=0){

//wait for 1000/fps milliseconds c = cvWaitKey(1000/fps); }else{ //wait for 40 milliseconds c = cvWaitKey(40); }

//exit the loop if user press "Esc" key (ASCII value of "Esc" is 27) if((char)c==27 ) break; }

//destroy the opened window cvDestroyWindow("Video"); //release memory cvReleaseCapture( &capture ); return 0;

} /////////////////////////////////////////////////////////////////////////////////////// You can download this OpenCV visual c++ project from here. (The downloaded file is a compressed .rar folder. So, you have to extract it using Winrar or other suitable software)


cvCaptureFromAVI("C:/Wildlife.avi");

The function allocates and initializes the CvCapture structure for reading the video stream from the specified AVI file, "C:/Wildlife.avi" (Other video formats also can be used, but they are not optimized e.g.- wmv,dat,mpg).

cvGetCaptureProperty( capture, CV_CAP_PROP_FPS );

Returns the frames per second of the captured video. If this function fails to get frames per second, it will return zero. For the first argument, it should be given a video capturing structure. For the 2nd argument, there are few options. They are

o CV_CAP_PROP_FPS - gives the frame rate o CV_CAP_PROP_FRAME_WIDTH - gives the width of frames in the video steam o CV_CAP_PROP_FRAME_HEIGHT - gives the height of frames in the video stream o CV_CAP_PROP_FRAME_COUNT - gives the number of frames in the video stream o CV_CAP_PROP_POS_MSEC - gives the current position in miliseconds in the running video stream o CV_CAP_PROP_POS_FRAMES - gives the 0-based index of the frame to be captured next

This method can be used only for capturing from file. It cannot be used for capturing from cameras.

cvQueryFrame( capture );

The function grabs the next frame from the captured video and returns the pointer. The returned image should not be released by the user.

cvWaitKey(1000/fps);

The function waits for 1000/fps milliseconds. If a key was pressed before the specified time, it returns the ASCII value of the pressed key and the program will go to the next line. If any key is not pressed, the function returns -1, after waiting for 1000/fps milliseconds. If cvGetCaptureProperty( capture, CV_CAP_PROP_FPS ) function returns zero, cvWaitKey(40) statement will execute.

cvReleaseCapture( &capture );

The function releases the CvCapture structure allocated by cvCaptureFromAVI function. Important Note:

User should never release images, returned by cvQueryFrame function because it will cause a run time error. Summary

At first, this program captures a video from a file. Then program enters into a infinite loop. In that loop, it grabs frames from the captured video sequentially, shows in a window and wait for 1000/fps milliseconds. Using cvWaitKey function is very important because cvShowImage function need time to show the image and cvWaitkey will give that necessary time. And waiting 1000/fps milliseconds will make sure the frame rate of output video is same as the original

frame rate.

Simple OpenCV Program-Playing a Video From a Webcam or Camera

The only difference of the following program from the above program is the first line after the main function. /////////////////////////////////////////////////////////////////////////////////////// #include "stdafx.h" #include #include int main() { //initialize and allocate memory to load the video stream from camera CvCapture *capture = cvCaptureFromCAM(0);

if( !capture ) return 1;

//create a window with the title "Video" cvNamedWindow("Video");

while(true) { //grab and retrieve each frames of the video sequentially IplImage* frame = cvQueryFrame( capture );

if( !frame ) break;

//show the retrieved frame in the "Video" window cvShowImage( "Video", frame );

//wait for 40 milliseconds int c = cvWaitKey(40); //exit the loop if user press "Esc" key (ASCII value of "Esc" is 27) if((char)c==27 ) break; }

//destroy the opened window cvDestroyWindow("Video"); //release memory cvReleaseCapture( &capture ); return 0;

} /////////////////////////////////////////////////////////////////////////////////////// You can download this OpenCV visual c++ project from here. (The downloaded file is a compressed .rar folder. So, you have to extract it using Winrar or other suitable software)


cvCaptureFromCAM(0);

The function allocates and initialized the CvCapture structure for reading a video stream from the camera. Here the '0' means the index of the camera to be used. You can use 1,2,3.. instead of 0, if your computer is attached to more than 1 camera. If there is only one camera or it does not matter what camera to use, -1 can be passed as the index.

cvWaitKey(40)

cvGetCaptureProperty( capture, CV_CAP_PROP_FPS ) cannot be used for capturing from cameras. Therefore we cannot obtain the frame rate of video stream. Therefore we have to put a custom value for the parameter ofcvWaitKey function. You

can use any value more than 20 for that parameter. But you must try different values for that parameter and see the difference.

Saving Images & Videos In this tutorial, it is explained how to save images and videos to the hard disk using OpenCV. If you have not install and configure OpenCV yet, please refer to Installing & Configuring with Visual Studio.

Simple OpenCV Program-Saving an Image

/////////////////////////////////////////////////////////////////////////////////////// #include "stdafx.h" #include #include int main() { //loads the image IplImage* img = cvLoadImage("C:/MyPic.jpg"); //check whether the image is loaded if(!img){ printf("Could not load image file\n"); } //saves the image as NewMyPic.jpg in the working folder int result=cvSaveImage("D:/NewMyPic.jpg",img); //check whether the image is saved if(!result){ printf("Could not save\n"); } cvNamedWindow("MyWindow"); cvShowImage("MyWindow", img); cvWaitKey(0); cvDestroyWindow("MyWindow"); cvReleaseImage(&img); return 0; }

/////////////////////////////////////////////////////////////////////////////////////// You can download this OpenCV visual c++ project from here. (The downloaded file is a compressed .rar folder. So, you have to extract it using Winrar or other suitable software) New OpenCV functions which are not found earlier are explained here

cvSaveImage("D:/NewMyPic.jpg",img)

The function saves the image in the specified location to the specified file name and returns a non zero value. If this function fails to save the image, it will return 0. The image format is chosen depending on the file name extension. Only 8 bit single channel or 3 channel with 'BGR' channel order images can be saved. If the depth or channel order of the image is different, use'cvCvtScale' and 'cvCvtColor' functions to convert it before using this function.

If you want more explanation about various methods in the above computer application , please refer toCapturing Images & Videos.

Simple OpenCV Program-Saving a Video

/////////////////////////////////////////////////////////////////////////////////////// #include "stdafx.h" #include #include int main() { //load the video file to the memory CvCapture *capture = cvCaptureFromAVI("C:/video.avi"); if( !capture ) return 1; int frame_width=(int) cvGetCaptureProperty(capture, CV_CAP_PROP_FRAME_WIDTH); //get the width of frame int frame_height=(int) cvGetCaptureProperty(capture, CV_CAP_PROP_FRAME_HEIGHT); //get the height of frame int fps = ( int )cvGetCaptureProperty( capture, CV_CAP_PROP_FPS ); //get the frames per second //creating video writer structure CvVideoWriter *writer = 0; writer=cvCreateVideoWriter("savedVideo.avi",CV_FOURCC('M', 'P', '4', '2'),fps,cvSize(frame_width,frame_height),1); if( !writer ) { //release memory cvReleaseCapture( &capture ); return 0; } while(true) { //grab and retrieve each frames of the video sequentially IplImage* frame = cvQueryFrame( capture ); if( !frame ) break; //writing frame by frame to the output file cvWriteFrame(writer,frame); } //release memory cvReleaseVideoWriter(&writer); cvReleaseCapture( &capture ); return 0; }

///////////////////////////////////////////////////////////////////////////////////////

You can download this OpenCV visual c++ project from here. (The downloaded file is a compressed .rar folder. So, you have to extract it using Winrar or other suitable software) New OpenCV functions which are not found earlier are explained here

cvGetCaptureProperty(capture, CV_CAP_PROP_FRAME_WIDTH)

This function returns the width of frames in pixels

cvGetCaptureProperty(capture, CV_CAP_PROP_FRAME_HEIGHT)

This function returns the height of frames in pixels

cvCreateVideoWriter("savedVideo.avi",CV_FOURCC('M', 'P', '4', '2'),fps,cvSize(frame_width,frame_height),1)

This fuction creates a Video Writer structure. The 1st argument, "savedVideo.avi" is the name of the output file. The output file will be saved inside the project folder. Here I have used CV_FOURCC('M', 'P', '4', '2') as the four character code of codec to compress the video. If the output file

cannot be opened, you can try any of the following four character code of codecs.

All four character code of codecs are listed below CV_FOURCC('D', 'I', 'V', '3') for MPEG-4.3 codec CV_FOURCC('M', 'P', '4', '2') for MPEG-4.2 codec CV_FOURCC('D', 'I', 'V', 'X') for MPEG-4 codec CV_FOURCC('P','I','M','1') for MPEG-1 codec CV_FOURCC('U', '2', '6', '3') for H263 codec CV_FOURCC('I', '2', '6', '3') for H263I codec CV_FOURCC('F', 'L', 'V', '1') for FLV1 codec For Windows users, it is possible to use -1 instead of the above codecs in order to choose compression method and additional compression parameters from a dialog box. The 3rd parameter, fps is the frames per seconds of the video stream.

The 4th parameter, cvSize(frame_width,frame_height) defines the size of frames in the video.

If the last parameter is a non zero value the encoder will expect and encode color frames. If it is zero, the encoder will work with grayscale frames. This flag is currently supported on Windows only. Here I have used 1, that's means I am telling the encoder to

encode this video as color frames.

cvWriteFrame(writer,frame)

This function adds frame by frame to the output video file. The 1st parameter defines the Video Writer structure. The 2nd parameter defines the frame which is to be added to the output file.

cvReleaseVideoWriter(&writer)

This function finishes writing to video file and releases the Video Writer structure.

Filtering Images Image filtering is a important part of computer vision. Most of computer vision applications, filtering should be done before anything else. OpenCV has lots of in-build filtering methods for images. Here is the list of methods used in this tutorial for filtering images

Eroding

Dilating

Smoothing

Inverting

Adjusting Brightness

Adjusting Contrast

Here is the original image which I am going to filter using above methods.

Original Image

If you have not install and configure OpenCV yet, please refer to Installing & Configuring with Visual Studio.

Eroding

Eroding is a simple way of filtering images. Here is how it can be done with OpenCV. /////////////////////////////////////////////////////////////////////////////////////// #include "stdafx.h" #include #include int main() { //display the original image IplImage* img = cvLoadImage("C:/MyPic.jpg"); cvNamedWindow("MyWindow"); cvShowImage("MyWindow", img); //erode and display the eroded image cvErode(img, img, 0, 2); cvNamedWindow("Eroded"); cvShowImage("Eroded", img); cvWaitKey(0); //cleaning up cvDestroyWindow("MyWindow"); cvDestroyWindow("Eroded"); cvReleaseImage(&img); return 0;

}

///////////////////////////////////////////////////////////////////////////////////////

You can download this OpenCV visual c++ project from here. (The downloaded file is a compressed .rar folder. So, you have to extract it using Winrar or other suitable software)

Eroded Image


cvErode(img, img, 0, 2)

The 1st parameter is the source image. The 2nd parameter is the destination image which is to be the eroded image. Here the 3rd parameter is the structuring element used for erosion. If it is 0, a 33 rectangular structuring element is used. The 4th parameter is the number of times, erosion is applied. This function can process images in place. That means same variable can be used for the 1st and 2nd parameters. If you want more explanation about various methods in the above computer application , please refer toCapturing Images & Videos. Dilating

Dilating is something like opposite of the eroding an image. Here is the OpenCV code. /////////////////////////////////////////////////////////////////////////////////////// #include "stdafx.h" #include #include int main() { //display the original image IplImage* img = cvLoadImage("C:/MyPic.jpg"); cvNamedWindow("MyWindow"); cvShowImage("MyWindow", img); //dilate and display the dilated image cvDilate(img, img, 0, 2); cvNamedWindow("Dilated"); cvShowImage("Dilated", img); cvWaitKey(0); //cleaning up cvDestroyWindow("MyWindow"); cvDestroyWindow("Dilated");

cvReleaseImage(&img); return 0; }

///////////////////////////////////////////////////////////////////////////////////////


Dilated Image


cvDilate(img, img, 0, 2)

The 1st parameter is the source image. The 2nd parameter is the destination image which is to be the dilated image. Here the 3rd parameter is the structuring element used for dilation. If it is 0, a 33 rectangular structuring element is used. The 4th parameter is the number of times, dilation is applied. This function can process images in place. That means same variable can be used for the 1st and 2nd parameters. Smoothing

/////////////////////////////////////////////////////////////////////////////////////// #include "stdafx.h" #include #include int main() { //display the original image IplImage* img = cvLoadImage("C:/MyPic.jpg"); cvNamedWindow("MyWindow"); cvShowImage("MyWindow", img); //erode and display the smoothed image cvSmooth(img, img, CV_GAUSSIAN,3,3); cvNamedWindow("Smoothed"); cvShowImage("Smoothed", img); cvWaitKey(0); //cleaning up cvDestroyWindow("MyWindow"); cvDestroyWindow("Smoothed"); cvReleaseImage(&img); return 0; }

///////////////////////////////////////////////////////////////////////////////////////


Smoothed Image


cvSmooth(img, img, CV_GAUSSIAN,3,3)

The 1st parameter is the source image. The 2nd parameter is the destination image. The 3rd parameter is the method, used for smoothing the image. There are several methods that you can use for this parameter.

o CV_BLUR_NO_SCALE o CV_BLUR o CV_GAUSSIAN o CV_MEDIAN o CV_BILATERAL

Every of these methods has some features and restrictions listed below.

CV_BLUR_NO_SCALE supports for single channel images only. CV_BLUR and CV_GAUSSIAN support for 1 or 3 channel, 8 and 32 bit floating point images. These two methods can process

images in place. CV_MEDIAN and CV_BILATERAL support for 1 or 3 channel, 8 bit images and can not process images in place.

The 4th parameter is the aperture width. (It is a technical term. It defines how much you are going to smooth the image). You have to use positive odd number (1,3,5,7,...) The 5th parameter is the aperture height. (It is a technical term. It defines how much you are going to smooth the image). This parameter is ignored by CV_MEDIAN and CV_BILATERAL methods. You have to use positive odd number (1,3,5,7,...)

Inverting

Inverting an image is like taking the negative of an image. ///////////////////////////////////////////////////////////////////////////////////////

#include "stdafx.h" #include #include int main() {

//display the original image IplImage* img = cvLoadImage("C:/MyPic.jpg"); cvNamedWindow("MyWindow"); cvShowImage("MyWindow", img); //invert and display the inverted image cvNot(img, img); cvNamedWindow("Inverted"); cvShowImage("Inverted", img); cvWaitKey(0); //cleaning up cvDestroyWindow("MyWindow"); cvDestroyWindow("Inverted"); cvReleaseImage(&img); return 0; }

///////////////////////////////////////////////////////////////////////////////////////


Inverted Image


cvNot(img, img)

This function inverts every bit in every element of the image in the 1st parameter and places the result in the image in the 2nd parameter. This function can process images in place. That means same variable can be used for the 1st and 2nd parameters. e.g - For a 8 bit image, the value 0 will be mapped to (255-0)=255 the value 46 will be mapped to (255-46)=209

For a 16 bit image, the value 0 will be mapped to (65535-0)=65535 the value 46 will be mapped to (65535-46)=65489 Adjusting Brightness

You can adjust brightness of an image by adding or subtracting some constant from the each and every pixel of the original image. Here is the simple example of increasing the brightness of an image with OpenCV.

/////////////////////////////////////////////////////////////////////////////////////// #include "stdafx.h" #include #include

int main() { //display the original image IplImage* img = cvLoadImage("C:/MyPic.jpg"); cvNamedWindow("MyWindow"); cvShowImage("MyWindow", img); //increase the brightness and display the brightened image cvAddS(img, cvScalar(70,70,70), img); cvNamedWindow("Brightened"); cvShowImage("Brightened", img); cvWaitKey(0); //cleaning up cvDestroyWindow("MyWindow"); cvDestroyWindow("Brightened"); cvReleaseImage(&img); return 0; }

///////////////////////////////////////////////////////////////////////////////////////


Brightened Image


cvAddS(img, cvScalar(70,70,70), img)

1st parameter is the source image. 3rd parameter is the destination image. 2nd parameter is the value, we are going to add to each and every pixel value of the source image. Here I have used cvScalar(70,70,70) because the source image consists of 3 channels. (Color images always consists of 3 channels to

represent red, green and blue). If the source image is a gray scale image which consists of 1 channel, you may use cvScalar(70) as the 2nd parameter. If you want to decrease the brightness, you may usecvScalar(-50,-50,-50) as the 2nd

parameter. This function can process images in place. That means same variable can be used for the 1st and 2nd parameters. Adjusting Contrast

You can adjust contrast of an image by multiplying or dividing each and every pixel of the original image by some constant. Here is the simple example of increasing the contrast of an image with OpenCV. /////////////////////////////////////////////////////////////////////////////////////// #include "stdafx.h"

#include #include int main() { //display the original image IplImage* img = cvLoadImage("C:/MyPic.jpg"); cvNamedWindow("MyWindow"); cvShowImage("MyWindow", img); //increase the contrast and display the image cvScale(img, img, 2); cvNamedWindow("Contrast"); cvShowImage("Contrast", img); cvWaitKey(0); //cleaning up cvDestroyWindow("MyWindow"); cvDestroyWindow("Contrast"); cvReleaseImage(&img); return 0; }

/////////////////////////////////////////////////////////////////////////////////////// You can download this OpenCV visual c++ project from here. (The downloaded file is a compressed .rar folder. So, you have to extract it using Winrar or other suitable software)

Contrasted Image


cvScale(img, img, 2)

1st parameter is the source image. 2nd parameter is the destination image. 3rd parameter is the constant by which each and every pixel values are multiplied. If you use value 1, there is no change. If you use a value more than 1, destination image will have a high contrast than the source image. If you use a value less than 1, destination image will have a low contrast than the source image. (e.g - To decrease the contrast, you can use cvScale(img, img, 0.5))

This function can process images in place. That means same variable can be used for the 1st and 2nd parameters.

Track Bars Using Track Bars with Images

Track bars are very useful in lots of occasions. It enables users to change various parameters while the computer application is running. If you have not install and configure OpenCV yet, please refer to Installing & Configuring with Visual Studio. In the following example, I have added a track bar to change the brightness of an image. /////////////////////////////////////////////////////////////////////////////////////// #include "stdafx.h" #include #include int main() { int bright=50; cvNamedWindow("MyWindow"); //create trackbars cvCreateTrackbar("brightness", "MyWindow", &bright, 100, NULL); //load the original image IplImage* img = cvLoadImage("C:/MyPic.jpg");

//create a blank image IplImage* des = cvCreateImage(cvGetSize(img),IPL_DEPTH_8U,3); while(1){ //change the brightness of the image cvAddS(img, cvScalar(bright-50,bright-50,bright-50), des); cvShowImage("MyWindow", des); int c=cvWaitKey(40); if(c==27){ //cleaning up cvDestroyWindow("MyWindow"); cvReleaseImage(&img); cvReleaseImage(&des); break; } } return 0; }

///////////////////////////////////////////////////////////////////////////////////////


Using Track Bars

Explanation

Here are the new OpenCV functions, found in the above c++ application.

cvCreateTrackbar("brightness", "MyWindow", &bright, 100, NULL);

This function creates a track bar in a window, defined by the 2nd parameter.

The 1st parameter is the name of the track bar. As you can see in the above picture, that name is displayed in the left side of the track bar. The 3rd parameter is a pointer to an integer variable. It is the variable which is controlled by the track bar. When the position of the track bar changes, the value of that variable will also be changed. That value is displayed the the left corner of the track bar in front of the name of the track bar. The 4th parameter is the maximum value of the track bar. Minimum is always 0. The 5th parameter is a name of the callback function which will be called whenever the position of the track bar changes. If you want more explanation about various methods in above computer application , refer to Capturing Images & Videos and Filtering Images.

Track Bars with Callback Function

In the above application, I have used 'NULL' for the 5th parameter of the function, cvCreateTrackbar. Here I am going to

explain, how we use a callback function for the 5th parameter. In the following example, I have added a track bar to change the contrast of an image.The advantage of using the callback function is that it is not required to use a while loop as in the above example.

/////////////////////////////////////////////////////////////////////////////////////// #include "stdafx.h" #include #include IplImage* img; IplImage* des;

//callback function void changeContrast(int contrast){ if(contrast=10){ cvScale(img, des, (contrast-9)); }

cvShowImage("MyWindow", des); } int main() { int contrast=10; cvNamedWindow("MyWindow"); //create trackbars cvCreateTrackbar("contrast", "MyWindow", &contrast, 21, changeContrast); //load the original image img = cvLoadImage("C:/MyPic.jpg"); //create a blank image des = cvCreateImage(cvGetSize(img),IPL_DEPTH_8U,3); changeContrast(10); cvWaitKey(0); //cleaning up cvDestroyWindow("MyWindow"); cvReleaseImage(&img); cvReleaseImage(&des); return 0; }

///////////////////////////////////////////////////////////////////////////////////////

You can download this OpenCV visual c++ project from here. (The downloaded file is a compressed .rar folder. So, you have to extract it using Winrar or other suitable software) Explanation

In addition to 'main' method, I have used a another method,'changeContrast(int contrast)' as the callback function of the track bar.

I have used 2 global variables, img and des because they should be accessed from both methods. Examine closely the 5th parameter of cvCreateTrackbar("contrast", "MyWindow", &contrast, 21, changeContrast). It is just

name of the callback function. This function will be called whenever the position of the track bar changes.

Mouse Events OpenCV supports for detecting mouse events. It is very simple to do that. We can detect the specific mouse event and (x,y) coordinates of the mouse pointer in image coordinates (not window coordinates). Here is a simple example of detecting a mouse left click event and its (x,y) coordinates. /////////////////////////////////////////////////////////////////////////////////////// #include "stdafx.h" #include #include //callback function void mouseEvent(int evt, int x, int y, int flags, void* param){ if(evt==CV_EVENT_LBUTTONDOWN){ printf("%d %d\n",x,y); } } int main() { cvNamedWindow("MyWindow"); //assigning the callback function for mouse events cvSetMouseCallback("MyWindow", mouseEvent, 0); //load and display an image IplImage* img = cvLoadImage("C:/MyPic.jpg"); cvShowImage("MyWindow", img); //wait for key press cvWaitKey(0); //cleaning up cvDestroyWindow("MyWindow"); cvReleaseImage(&img); return 0; }

///////////////////////////////////////////////////////////////////////////////////////


Mouse Events

Summary In this application, it detects mouse left button press on the image window and prints out the (x,y) coordinates.

Here are the new OpenCV functions, found in the above example.

cvSetMouseCallback("MyWindow", mouseEvent, 0)

The 1st parameter is the name of the window with should be monitored for mouse events. The 2nd parameter is the name of the function which will be called whenever mouse events occur in the specified window.This function is prototyped as follows. void mouseEvent(int evt, int x, int y, int flags, void* param)

o For the 1st parameter, it will be one of following CV_EVENT_MOUSEMOVE - when the mouse pointer moves over the specified window CV_EVENT_LBUTTONDOWN - when the left button of the mouse is pressed on the specified

window

CV_EVENT_RBUTTONDOWN - when the right button of the mouse is pressed on the specified window

CV_EVENT_MBUTTONDOWN - when the middle button of the mouse is pressed on the specified window

CV_EVENT_LBUTTONUP - when the left button of the mouse is released on the specified window CV_EVENT_RBUTTONUP - when the right button of the mouse is released on the specified window CV_EVENT_MBUTTONUP - when the middle button of the mouse is released on the specified

window

o For the 2nd and 3rd parameters, it will be the x and y coordinates of the position of mouse pointer as illustrated in the above image.

o For the 4th parameter, it will be one of following CV_EVENT_FLAG_LBUTTON - when the left button of the mouse is pressed on the specified

window

CV_EVENT_FLAG_RBUTTON - when the right button of the mouse is pressed on the specified window

CV_EVENT_FLAG_MBUTTON - when the middle button of the mouse is pressed on the specified window

CV_EVENT_FLAG_CTRLKEY - when the 'CTRL' key is pressed and any mouse event is occured CV_EVENT_FLAG_SHIFTKEY - when the 'SHIFT' key is pressed and any mouse event is occured CV_EVENT_FLAG_ALTKEY - when the 'ALT' key is pressed and any mouse event is occured

o The 5th parameter is user defined parameter passed by the 3rd parameter of cvSetMouseCallbackfunction.

More Advance Visual C++ Example This is also similar to the above example. But this is a little bit advanced. Try to understand the difference and predict the result.

/////////////////////////////////////////////////////////////////////////////////////// #include "stdafx.h" #include #include //callback function void mouseEvent(int evt, int x, int y, int flags, void* param){ if(evt==CV_EVENT_MOUSEMOVE && flags==CV_EVENT_FLAG_CTRLKEY){ printf("%d %d\n",x,y); } } int main() { cvNamedWindow("MyWindow"); //assigning the callback function for mouse events cvSetMouseCallback("MyWindow", mouseEvent, 0); //load and display an image IplImage* img = cvLoadImage("C:/MyPic.jpg"); cvShowImage("MyWindow", img); //wait for key press

cvWaitKey(0); //cleaning up cvDestroyWindow("MyWindow"); cvReleaseImage(&img); return 0; }

/////////////////////////////////////////////////////////////////////////////////////// You can download this visual OpenCV c++ project from here. (The downloaded file is a compressed .rar folder. So, you have to extract it using Winrar or other suitable software)

Rotating Images & Videos

Rotating Images

There are several ways to rotate images with OpenCV. The main problem in rotating images is it leaves some blank space in

the four corners of the image. There are several ways to cope with the blank area

Here is an example of rotating an image while preserving its size.

///////////////////////////////////////////////////////////////////////////////////////

#include "stdafx.h"

#include

#include

IplImage *rotateImage(const IplImage *src, int angleDegrees)

{

IplImage *imageRotated = cvCloneImage(src);

if(angleDegrees!=0){

CvMat* rot_mat = cvCreateMat(2,3,CV_32FC1);

// Compute rotation matrix

CvPoint2D32f center = cvPoint2D32f( cvGetSize(imageRotated).width/2, cvGetSize(imageRotated).height/2 );

cv2DRotationMatrix( center, angleDegrees, 1, rot_mat );

// Do the transformation

cvWarpAffine( src, imageRotated, rot_mat );

}

return imageRotated;

}

int main()

{

IplImage* img;

IplImage* rotated_img;

int angle=0;

//creating the window with a track bar

cvNamedWindow("MyWindow");

cvCreateTrackbar("Angle", "MyWindow", &angle, 360, 0);

while(true){

//load the original image

img = cvLoadImage("C:/MyPic.jpg");

//rotate the image

rotated_img=rotateImage(img,angle);

//display the rotated image

cvShowImage("MyWindow", rotated_img);

//clean up

cvReleaseImage(&img);

cvReleaseImage(&rotated_img);

//if user press 'ESC' button, program quit the while loop

int c=cvWaitKey(50);

if(c==27) break;

}

cvDestroyWindow("MyWindow");

return 0;

}

///////////////////////////////////////////////////////////////////////////////////////

You can download this OpenCV visual c++ project from here. (The downloaded file is a compressed .rar folder. So, you have to

extract it using Winrar or other suitable software)

Rotated Image

Explanation


cvCloneImage(const IplImage* src)

Makes a identical(cloned) copy of the 'src' image including header and region of interrest(ROI)

Returns IplImage* points to the cloned image

Arguements -

o IplImage* src - source image to be cloned

cvCreateMat(int rows, int columns, int types)

Creates matrix header and allocates matrix data

Returns CvMat* points to the matrix

Arguements -

o int rows - number of rows of the matrix o int columns - number of columns of the matrix o int types - CV_C

= 8,16,32, ....

= 'S' for signed data, 'U' for unsigned data and 'F' for float data

= 1,2,3

eg. : CV_8UC1 = 8 bit unsigned image with a single channel

CV_32FC3 = 32 bit float image with 3 channels

cvPoint2D32f (float x, float y)

It is a basic structure for 2D point with floating point cordinates

typedef struct CvPoint2D32f

{

float x;

float y;

}

CvPoint2D32f;

/* Constructor */

inline CvPoint2D32f cvPoint2D32f( double x, double y );

cv2DRotationMatrix(CvPoint2D32f center, double angle, double scale, CvMat* mapMatrix)

Computes affine matrix of 2D rotation

Arguements -

o CvPoint2D32f center - specifies the ceter of rotation o double angle - rotation angle in degrees (Positive value means counter clockwise rotation and negative value

means clockwise rotation) o double scale -Isotrophic scale factor o CvMat* mapMatrix - the pointer to the resultant matrix which should be 2x3 matrix

cvWarpAffine(const CvArr* src, CvArr* dst, const CvMat* mapMatrix)

Applies an affine transformation to the source image

Arguements -

o const CvArr* src - source image o CvArr* dst - destination image which is the affine transformed image of the source image o const CvMat* mapMatrix - 2x3 transformation matrix

If you are not familiar with other methods of the above application, please refer to my previous OpenCV tutorials

Rotate with Zoom

This is also very much similar to the above example. Try to understand the code by comparing the above example.

///////////////////////////////////////////////////////////////////////////////////////

#include "stdafx.h"

#include

#include

IplImage *rotateImage(const IplImage *src, int angleDegrees, double zoom)

{

IplImage *imageRotated = cvCloneImage(src);

CvMat* rot_mat = cvCreateMat(2,3,CV_32FC1);

// Compute rotation matrix

CvPoint2D32f center = cvPoint2D32f( cvGetSize(imageRotated).width/2, cvGetSize(imageRotated).height/2 );

cv2DRotationMatrix( center, angleDegrees, zoom, rot_mat );

// Do the transformation

cvWarpAffine( src, imageRotated, rot_mat );


}

int main()

{

IplImage* img;


int angle=0;

int zoom=24;

//creating the window with 2 track bars



cvCreateTrackbar("Zoom", "MyWindow", &zoom, 99, 0);

while(true){



//rotate the image

rotated_img=rotateImage( img, angle, (zoom+1)/25.0 );



//clean up





if(c==27) break;

}


return 0;

}

///////////////////////////////////////////////////////////////////////////////////////



Rotated and Zoomed Image

Explanation

This program is almost similar to the previous one except the fact that the image can be zoomed out or in.

The only differece is in the cv2DRotationMatrix( center, angleDegrees, zoom, rot_mat ) method which has 'zoom' variable as

the third argument of this method instea of 1.

Here the value of 'zoom' variable should be larger than 0.

Rotating Images in a Different Way

///////////////////////////////////////////////////////////////////////////////////////

#include "stdafx.h"

#include

#include

IplImage* rotateImage(const IplImage* src, int angleDegrees)

{

//take the dimention of original image

int w = src->width;

int h = src->height;

// Make a new image for the result

CvSize newSize;

newSize.width = cvRound(w);

newSize.height = cvRound(h);

IplImage *imageRotated = cvCreateImage( newSize, src->depth, src->nChannels );

// Create a map_matrix, where the left 2x2 matrix is the transform and the right 2x1 is the dimensions.

float m[6];

CvMat M = cvMat(2, 3, CV_32F, m);

float angleRadians = angleDegrees * ((float)CV_PI / 180.0f);

m[0] = (float)( cos(angleRadians) );

m[1] = (float)( sin(angleRadians) );

m[3] = -m[1];

m[4] = m[0];

m[2] = w*0.5f;

m[5] = h*0.5f;

// Transform the image

cvGetQuadrangleSubPix( src, imageRotated, &M);


}

int main()

{

IplImage* img;


int angle=0;

//creating the window with a track bar



while(true){



//rotate the image

rotated_img=rotateImage(img,angle);



//clean up





if(c==27) break;

}


return 0;

}

///////////////////////////////////////////////////////////////////////////////////////



Rotated Image

Explanation


cvRound(double x)

Returns nearest nterger value of x

Arguement -

o double x - the input floating point number

cvCreateImage( CvSize size, int depth, int channels)

Creates an image header and allocate image data

Returns IplImage* which points to the created blank image

Arguements -

o CvSize size - size of the cretaed blank image o int depth - depth of the created blank image (8,16,32,...) o int channels - number of channels of the created blank image (1,3)

cvMat(int rows, int columns, int type, void data)

Initialize the matrix header and assigns data to it. The matrix is filled row-wise

Arguements -

o int rows - number of rows of the matrix o int columns - number of columns of the matrix o int types - CV_C

(More details can be found in the explanation of the method, 'cvCreateMat' in the 1st application in the top of this page)

cvGetQuadrangleSubPix(const CvArr* src, CvArr* dst, const CvMat* mapMatrix)

Retrieves the pixel quadrangle from an image with sub-pixel accuracy.

The values of pixels at non-integer coordinates are retrieved using bilinear interpolation.

When the function needs pixels outside of the image, it uses replication border mode to reconstruct the values.

Every channel of multiple-channel images is processed independently.

Arguements -

o const CvArr* src - Source image o CvArr* dst - Destination image o const CvMat*mapMatrix - Transformation 2x3 matrix

Object Detection & Tracking using Color

Object detection and segmaentation is the most important and challenging fundamental task of computer vision. It is a critical

part in many applications such as image search, image auto-annotation and scene understanding. However it is still an open

problem due to the complexity of object classes and images.

The easiest way to detect and segment an object from an image is the color based methods . The colors in the object and the

background should have a significant color difference in order to segment objects sucessfully using color based methods.

Simple Example of Detecting Red objects

In this example, I am going to convet a video into a binary image based on the red color. (Red color area of the video is

assined to '1' and other area is assigned to '0' in the binary image)

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

#include "stdafx.h"

#include

#include

//This function threshold the HSV image and create a binary image

IplImage* GetThresholdedImage(IplImage* imgHSV){

IplImage* imgThresh=cvCreateImage(cvGetSize(imgHSV),IPL_DEPTH_8U, 1);

cvInRangeS(imgHSV, cvScalar(170,160,60), cvScalar(180,256,256), imgThresh);

return imgThresh;

}

int main(){

CvCapture* capture =0;

capture = cvCaptureFromCAM(0);

if(!capture){

printf("Capture failure\n");

return -1;

}

IplImage* frame=0;

cvNamedWindow("Video");

cvNamedWindow("Ball");

//iterate through each frames of the video

while(true){

frame = cvQueryFrame(capture);

if(!frame) break;

frame=cvCloneImage(frame);

cvSmooth(frame, frame, CV_GAUSSIAN,3,3); //smooth the original image using Gaussian kernel

IplImage* imgHSV = cvCreateImage(cvGetSize(frame), IPL_DEPTH_8U, 3);

cvCvtColor(frame, imgHSV, CV_BGR2HSV); //Change the color format from BGR to HSV

IplImage* imgThresh = GetThresholdedImage(imgHSV);

cvSmooth(imgThresh, imgThresh, CV_GAUSSIAN,3,3); //smooth the binary image using Gaussian kernel

cvShowImage("Ball", imgThresh);

cvShowImage("Video", frame);

//Clean up used images

cvReleaseImage(&imgHSV);

cvReleaseImage(&imgThresh);

cvReleaseImage(&frame);

//Wait 50mS

int c = cvWaitKey(10);

//If 'ESC' is pressed, break the loop

if((char)c==27 ) break;

}

cvDestroyAllWindows() ;

cvReleaseCapture(&capture);

return 0;

}

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////



Explanation

OpenCV usually captures images and videos in 8-bit, unsigned integer, BGR format. In other words, captured images can be

considered as 3 matrices, BLUE,RED and GREEN with integer values ranges from 0 to 255.

How BGR image is formed

In the above image, each small box represents a pixel of the image. In real images, these pixels are so small that human eye

cannot differentiate.

Usually, one can think that BGR color space is more suitable for color based segmentation. But HSV color space is the most

suitable color space for color based image segmentation. So, in the above application, I have converted the color space of

original image of the video from BGR to HSV image.

HSV color space is consists of 3 matrices, 'hue', 'saturation' and 'value'. In OpenCV, value range for 'hue', 'saturation' and

'value' are respectively 0-179, 0-255 and 0-255. 'Hue' represents the color, 'saturation' represents the amount to which that

respective color is mixed with white and 'value' represents the amount to which that respective color is mixed with black.

In the above application, I have considered that the red object has 'hue', 'saturation' and 'value' in between 170-180, 160-255,

60-255 respectively. Here the 'hue' is unique for that specific color distribution of that object. But 'saturation' and 'value' may be

vary according to the lighting condition of that environment.

Hue values of basic colors

o Orange 0-22 o Yellow 22- 38 o Green 38-75 o Blue 75-130 o Violet 130-160 o Red 160-179

There is a open source software, called 'ColorWheelHSV', that enables you to find the 'hue', 'saturation' and 'value' for a specific

object easily. You can download it from here.

These are approximate values. You have to find the exact range of 'hue' values according to the color of the object. I found that

the range of 170-179 is perfect for the range of hue values of my object. The 'saturation' and 'value' is depend on the lighting

condition of the environment as well as the surface of the object.

How to find the exact range of 'hue', 'saturation' and 'value' for a object is discussed later in this post.

Now let's discuss new OpenCV methods in the above application.

cvInRangeS(const CvArr* src, CvScalar lower, CvScalar upper, CvArr* dst)

Checks that each array element of 'src' lies between 'lower' and 'upper'. If so, array element in the relevant location of 'dst' is

assigned '255' , otherwise '0'.

Arguments -

o const CvArr* src - source array which is the image o CvScalar lower - inclusive lower bound (In the above application, it is cvScalar(170,160,60) because my lower

bound for 'hue','saturation' and 'value' is 170,160 and 60 respectively) o CvScalar upper - exclusive upper bound (In the above application, it is cvScalar(180,256,256) because my

upper bound for 'hue','saturation' and 'value' is 179,255 and 255 respectively) o CvArr* dst - destination array which is the binary image (must have 8-bit unsigned integer or 8-bit signed

integer type)

Output images of a webcam is little bit noisy. So, I smooth each frame of the video and each binary thresholded images with a

Gaussian 3x3 kernel.

All other methods in the above application have been discussed in early OpenCV tutorials.

Simple Example of Tracking Red objects

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

#include "stdafx.h"

#include

#include

IplImage* imgTracking;

int lastX = -1;

int lastY = -1;




cvInRangeS(imgHSV, cvScalar(170,160,60), cvScalar(180,2556,256), imgThresh);

return imgThresh;

}

void trackObject(IplImage* imgThresh){

// Calculate the moments of 'imgThresh'

CvMoments *moments = (CvMoments*)malloc(sizeof(CvMoments));

cvMoments(imgThresh, moments, 1);

double moment10 = cvGetSpatialMoment(moments, 1, 0);

double moment01 = cvGetSpatialMoment(moments, 0, 1);

double area = cvGetCentralMoment(moments, 0, 0);

// if the area

cvLine(imgTracking, cvPoint(posX, posY), cvPoint(lastX, lastY), cvScalar(0,0,255), 4);

}

lastX = posX;

lastY = posY;

}

free(moments);

}

int main(){



if(!capture){


return -1;

}

IplImage* frame=0;


if(!frame) return -1;

//create a blank image and assigned to 'imgTracking' which has the same size of original video

imgTracking=cvCreateImage(cvGetSize(frame),IPL_DEPTH_8U, 3);

cvZero(imgTracking); //covert the image, 'imgTracking' to black




while(true){


if(!frame) break;


cvSmooth(frame, frame, CV_GAUSSIAN,3,3); //smooth the original image using Gaussian kernel




cvSmooth(imgThresh, imgThresh, CV_GAUSSIAN,3,3); //smooth the binary image using Gaussian kernel

//track the possition of the ball

trackObject(imgThresh);

// Add the tracking image and the frame

cvAdd(frame, imgTracking, frame);







//Wait 10mS




}

cvDestroyAllWindows() ;

cvReleaseImage(&imgTracking);


return 0;

}

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////



Object Tracking

Explanation

In this application, I use moments to calculate the position of the center of the object. We have to calculate 1st order spacial

moments around x-axis and y-axis and the 0th order central moments of the binary image.

0th order central moments of the binary image is equal to the white area of the image in pixels.

X coordinate of the position of the center of the object = 1st order spacial moment around x-axis / 0th order central moment

Y coordinate of the position of the center of the object = 1st order spacial moment around y-axis / 0th order central moment

If there are 2 or more objects in the image, we cannot use this method. And noise of the binary image is also should be at

minimum level to get accurate results.

In the above application, I considered that if the white area of the binary image is less than 1000 in pixels, there is no objects in

the image because there may be noisy white areas that may have been identified as the object. Usually my object always have

an area more than 1000 pixels.

Now, let's discuss new OpenCV methods that can be found in the above application.

malloc(size_t x)

Allocate a block of memory which have x bytes and returns a pointer to the beginning of the block.

The content of the newly allocated block of memory is not initialized, remaining with undetermined values.

cvMoments(

const CvArr* arr, CvMoments* moments, int isBinary=0

)

Calculates all of the spacial and central moments up to the third order

Arguments -

o const CvArr* arr - source image which we are going to find the moment o CvMoments* moments - Pointer to a memory block to store calculated moments o int isBinary - If this argument is equal to non-zero value, all non-zero pixel values are considered as 1 when

calculating moments.

free(void* ptr)

A block of memory previously allocated using a call to malloc is deallocated, making it available again for further allocations.

cvGetSpatialMoment(CvMoments* ptr, int x_order, int y_order)

Retrieve calculated spacial moments from ptr memory block

Arguments -

o CvMoments* ptr - pointer to allocated memory block which store all moments o int x_order - order of x (>=0) o int y_order - order of y (>=0)

e.g. -

cvGetSpatialMoment(moments, 1, 0) retrieves 1st order spacial moment around x-axis

cvGetSpatialMoment(moments, 0, 1) retrieves 1st order spacial moment around y-axis

cvGetCentralMoment ( CvMoments* ptr, int x_order, int y_order )

Retrieve calculated central moments from ptr memory block

Arguments -

CvMoments* ptr - pointer to allocated memory block which store all moments

int x_order - order of x (>=0)

int y_order - order of y (>=0)

e.g. -

cvGetCentralMoment(moments, 0, 0) retrieves 0th order central moment

cvLine (CvArr* img, CvPoint pt1, CvPoint pt2, CvScalar color, int thickness=1)

Draw a line between 2 points, pt1 and pt2

Arguments -

o CvArr* img - source image o CvPoint pt1 - starting point of the line o CvPoint pt2 - ending point of the line o CvScalar color - color of the line (in the Blue, Green, Red order) o int thickness - thickness of the line in pixels

Except to the 'void trackObject(IplImage* imgThresh)' method, all other code in the above application is almost same to the 1st

application in this post.

How to Find Exact Range for 'Hue', 'Saturation' and 'Value' for a Given Object

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

#include "stdafx.h"

#include

#include

int lowerH=0;

int lowerS=0;

int lowerV=0;

int upperH=180;

int upperS=256;

int upperV=256;




cvInRangeS(imgHSV, cvScalar(lowerH,lowerS,lowerV), cvScalar(upperH,upperS,upperV), imgThresh);

return imgThresh;

}

//This function create two windows and 6 trackbars for the "Ball" window

void setwindowSettings(){



cvCreateTrackbar("LowerH", "Ball", &lowerH, 180, NULL);

cvCreateTrackbar("UpperH", "Ball", &upperH, 180, NULL);

cvCreateTrackbar("LowerS", "Ball", &lowerS, 256, NULL);

cvCreateTrackbar("UpperS", "Ball", &upperS, 256, NULL);

cvCreateTrackbar("LowerV", "Ball", &lowerV, 256, NULL);

cvCreateTrackbar("UpperV", "Ball", &upperV, 256, NULL);

}

int main(){



if(!capture){


return -1;

}

IplImage* frame=0;

setwindowSettings();


while(true){


if(!frame) break;











//Wait 80mS




}

cvDestroyAllWindows();


return 0;

}

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////



Finding the optimum 'hue', 'saturation' and 'value' ranges for an object is a 4 step process.

1) At the beginning, track bars are placed so that maximum ranges for 'hue', 'saturation' and 'value' are set. In other

words, 'hue', 'saturation' and 'value'have ranges 0-179, 0-255 and 0-255 respectively. So, we will see a complete white image in

the 'Ball' window.

2) First, adjust 'LowerH' and 'UpperH' track bars so that the gap between 'LowerH' and 'UpperH' is minimized. Here you have to

be careful that white area in 'Ball' window that represents the object should not be affected, while you are trying to minimize the

gap.

3) Then, adjust 'LowerS' and 'UpperS' track bars so that the gap between 'LowerS' and 'UpperS' is minimized. Here you have to

be careful that white area in 'Ball' window that represents the object should not be affected, while you are trying to minimize the

gap.

4) Finally, adjust 'LowerV' and 'UpperV' track bars so that the gap between 'LowerV' and 'UpperV' is minimized. Here you have

to be careful that white area in 'Ball' window that represents the object should not be affected, while you are trying to minimize

the gap.

Now you know the optimum 'hue', 'saturation' and 'value' ranges for the object. It is 163-179, 126-217 and 68-127 in my case as

you can see in the below picture.

Identification of Shapes of Object using Contours

In the previous tutorial, we could detect and track an object using color separation. But we could not identify the shape of the object there. In this tutorial, let's see how to identify a shape and position of an object using contours with OpenCV. Using contours with OpenCV, you can get a sequence of points of vertices of each white patch (White patches are considered as polygons). As example, you will get 3 points (vertices) for a triangle, and 4 points for quadrilaterals. So, you can identify any polygon by the number of vertices of that polygon. You can even identify features of polygons such as convexity, concavity, equilateral and etc by calculating and comparing distances between vertices. Let's see how this can be done with OpenCV. All you need, is a binary image in which your objects should be white and the background should be black.

Now I am going to identify triangles and quadrilaterals and heptagon in the above image using a C++ application with OpenCV. I'll draw a line along the perimeter of every identified polygon with colors blue for triangle, green for quadrilaterals and red for heptagons. Here is the code. /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// #include "stdafx.h" #include #include using namespace std; int main() { IplImage* img = cvLoadImage("C:/Users/SHERMAL/Desktop/FindingContours.png");

//show the original image cvNamedWindow("Raw"); cvShowImage("Raw",img); //converting the original image into grayscale IplImage* imgGrayScale = cvCreateImage(cvGetSize(img), 8, 1); cvCvtColor(img,imgGrayScale,CV_BGR2GRAY); //thresholding the grayscale image to get better results cvThreshold(imgGrayScale,imgGrayScale,128,255,CV_THRESH_BINARY);

CvSeq* contours; //hold the pointer to a contour in the memory block CvSeq* result; //hold sequence of points of a contour CvMemStorage *storage = cvCreateMemStorage(0); //storage area for all contours

//finding all contours in the image cvFindContours(imgGrayScale, storage, &contours, sizeof(CvContour), CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE, cvPoint(0,0));

//iterating through each contour while(contours) { //obtain a sequence of points of contour, pointed by the variable 'contour' result = cvApproxPoly(contours, sizeof(CvContour), storage, CV_POLY_APPROX_DP, cvContourPerimeter(contours)*0.02, 0); //if there are 3 vertices in the contour(It should be a triangle) if(result->total==3 ) { //iterating through each point CvPoint *pt[3]; for(int i=0;i

pt[i] = (CvPoint*)cvGetSeqElem(result, i); }

//drawing lines around the triangle cvLine(img, *pt[0], *pt[1], cvScalar(255,0,0),4); cvLine(img, *pt[1], *pt[2], cvScalar(255,0,0),4); cvLine(img, *pt[2], *pt[0], cvScalar(255,0,0),4);

} //if there are 4 vertices in the contour(It should be a quadrilateral) else if(result->total==4 ) { //iterating through each point CvPoint *pt[4]; for(int i=0;itotal ==7 ) { //iterating through each point CvPoint *pt[7]; for(int i=0;ih_next; } //show the image in which identified shapes are marked cvNamedWindow("Tracked"); cvShowImage("Tracked",img); cvWaitKey(0); //wait for a key press //cleaning up cvDestroyAllWindows(); cvReleaseMemStorage(&storage); cvReleaseImage(&img); cvReleaseImage(&imgGrayScale); return 0; }

/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// You can download this OpenCV visual c++ project from here. (The downloaded file is a compressed .rar folder. So, you have to extract it using Winrar or other suitable software)

As you can see, triangles are marked with blue, quadrilaterals are marked with green and heptagons are marked with red. So, now it is obvious that this method is capable of identifying shapes. Explanation

Here I have converted the original image in to gray scale. It is because this method works only with gray scale image with single channel. To get better results, I threshold the gray-scale image using 'cvThreshold' function. You can use your own way to threshold the image. Then I find all contours in the thresholded image and identify and track all triangles, quadrilaterals and heptagons. Let's discuss new OpenCV functions, found in this application.

cvThreshold( const Mat& src, Mat& dst, double threshVal, double max, int thresholdType )

applies a fix level threshold to the each element of 'src' array write a value to corresponding array element of 'dst' Arguements -

o const Mat& src - Source array (This should be single channel) o Mat& dst - Destination array which has the same size and same type as the 'src' o double threshVal - Threshold value o double max - Maximum value to use with 'THRESH_BINARY' and 'THRESH_BINARY_INV' which are

thresholding types o int thresholdType - You can use one of the following for this arguement

THRESH_BINARY

dst(x,y)=max, if src(x,y) > ThreshVal dst(x,y)=0, if src(x,y) < ThreshVal

THRESH_BINARY_INV

dst(x,y)=0, if src(x,y) > ThreshVal dst(x,y)=max, if src(x,y) < ThreshVal

THRESH_TOZERO

dst(x,y)=src(x,y), if src(x,y) > ThreshVal dst(x,y)=0, if src(x,y) < ThreshVal

THRESH_TOZERO_INV

dst(x,y)=0, if src(x,y) > ThreshVal dst(x,y)=src(x,y), if src(x,y) < ThreshVal

THRESH_TRUNC

dst(x,y)=threshVal, if src(x,y) > ThreshVal dst(x,y)=src(x,y), if src(x,y) < ThreshVal In the above application, I have used 'THRESH_BINARY', because I want to assign 255 (white) where the objects are located and 0 (black) elsewhere.

cvCreateMemStorage(int byteSize)

Creates memory storage which has the capacity specified by the parameter 'byteSize'. But if byteSize=0, the allocated capacity is the default value(usually 64 Kb)

cvFindContours( CvArr* img, CvMemStorage* str, CvSeq** first_contour, int header_size, int mode, int method, CvPoint offset )

Find all contours in a binary image Arguments -

o CvArr* img - Source image (This should be 8 bit single channel). All non-zero pixels are considered as 1 and all zero remain zero.

o CvMemStorage* str - Memory blocks to store all obtained contours o CvSeq** first_contour - store a pointer to the first contour in the memory block, 'str' o int header_size - size of the sequence header o int mode - mode of retrieval of contours from the image

You have to choose one of the following

CV_RETR_LIST - Retrieves all of the contours and put them in a list CV_RETR_EXTERNAL - Retrieves only the extreme outer contours CV_RETR_CCOMP - Retrieves all of the contours and organizes them into a two-level hierarchy: on

the top level are the external boundaries of the components, on the second level are the boundaries of the holes

CV_RETR_TREE - Retrieves all of the contours and reconstructs the full hierarchy of nested contours

o int method - Approximation method

You have to choose one of the following

CV_CHAIN_CODE - Outputs contours in the Freeman chain code CV_CHAIN_APPROX_NONE - Translates all of the points from the chain code into points CV_CHAIN_APPROX_SIMPLE - Compresses horizontal, vertical, and diagonal segments and

leaves only their end points

CV_CHAIN_APPROX_TC89_L1,CV_CHAIN_APPROX_TC89_KCOS - Applies one of the flavors of the Teh-Chin chain approximation algorithm.

CV_LINK_RUNS - uses a completely different contour retrieval algorithm by linking horizontal segments of 1s. Only the 'CV_RETR_LIST' retrieval mode can be used with this method.

o CvPoint offset - Offset by which every contour point should be shifted. This is useful when we have set ROI (Region Of Interest) in the image. Normally we set the offset to 'cvPoint(0,0)'

cvApproxPoly( const void* src, int header_size, CvMemStorage* storage, int method, double para1, int para2 )

Approximate polygonal curves with specified precision arguments -

o const void* src - Sequence of points

o int header_size - size of the sequence header o CvMemStorage* storage - memory block that contains all contours o int method - Approximation method. (The only method, available to use for this argument is

'CV_POLY_APPROX_DP') o double para1 - approximation accuracy o int para2 - Determines whether the single sequence should be approximated or all sequences in the same

level or below

cvGetSeqElem( const CvSeq* seq, int index )

Returns a pointer to the element of 'seq' at 'index'

cvReleaseMemStorage( CvMemStorage** storage )

Deallocate memory blocks which have been allocated by 'cvCreateMemStorage()' function

Real World Example

The above example is not really useful in practical situation. Usually, there are lots of noises in an image such as irregular lighting, shadows, camera irregularities and etc. So, above application as it is, cannot be used to identify shapes in a real image. It should be modified to cope with these noises. And images usually have 3 channels (BGR color). So, it should be converted into grey-scale which has only one channel. Here is a real world image of an arena of a robot soccer, taken from a camera.

Here, we are going to detect and mark the perimeter of each triangle in the image with a blue line. Let's see the modified OpenCV c++ application which accomplish the above task. /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// #include "stdafx.h" #include #include using namespace std; int main() {

IplImage* img = cvLoadImage("C:/Users/SHERMAL/Desktop/DetectingContours.jpg");

//show the original image cvNamedWindow("Original"); cvShowImage("Original",img); //smooth the original image using Gaussian kernel to remove noise cvSmooth(img, img, CV_GAUSSIAN,3,3); //converting the original image into grayscale IplImage* imgGrayScale = cvCreateImage(cvGetSize(img), 8, 1); cvCvtColor(img,imgGrayScale,CV_BGR2GRAY); cvNamedWindow("GrayScale Image"); cvShowImage("GrayScale Image",imgGrayScale); //thresholding the grayscale image to get better results cvThreshold(imgGrayScale,imgGrayScale,100,255,CV_THRESH_BINARY_INV); cvNamedWindow("Thresholded Image"); cvShowImage("Thresholded Image",imgGrayScale); CvSeq* contour; //hold the pointer to a contour CvSeq* result; //hold sequence of points of a contour CvMemStorage *storage = cvCreateMemStorage(0); //storage area for all contours

//finding all contours in the image cvFindContours(imgGrayScale, storage, &contour, sizeof(CvContour), CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE, cvPoint(0,0));

//iterating through each contour while(contour) { //obtain a sequence of points of the countour, pointed by the variable 'countour' result = cvApproxPoly(contour, sizeof(CvContour), storage, CV_POLY_APPROX_DP, cvContourPerimeter(contour)*0.02, 0); //if there are 3 vertices in the contour and the area of the triangle is more than 100 pixels if(result->total==3 && fabs(cvContourArea(result, CV_WHOLE_SEQ))>100 ) { //iterating through each point CvPoint *pt[3]; for(int i=0;ih_next; } //show the image in which identified shapes are marked cvNamedWindow("Tracked"); cvShowImage("Tracked",img); cvWaitKey(0); //wait for a key press //cleaning up cvDestroyAllWindows(); cvReleaseMemStorage(&storage); cvReleaseImage(&img); cvReleaseImage(&imgGrayScale); return 0; }

/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// You can download this OpenCV visual c++ project from here. (The downloaded file is a compressed .rar folder. So, you have to


Gray scale Image

Thresholded Image

Triangles Detected

In the same way, any shapes with any sizes can be detected with OpenCV. Explanation

To reduce the noise level of the original image, I have smoothed the original image with a Gaussian kernel. Further you can change the 5th argument of cvApproxPoly() function to cope with the noise. In the above example, I have used cvContourPerimeter(contour)*0.02 as the 5th argument of cvApproxPoly(). You can try cvContourPerimeter(contour)*0.01 or cvContourPerimeter(contour)*0.04 or any other value and see the difference of

the output yourselves. Still there may be very small triangles, formed due to the noise. Therefore all triangles with areas less than 100 pixels are filtered out. Here are the new OpenCV functions, found in the above example.

cvContourArea(const CvArr* contour, CvSlice slice)

Calculate the area enclosed by sequence of contour points.

o const CvArr* contour - array of vertices of the contour o CvSlice slice - starting and ending point of the contour. 'CV_WHOLE_SEQ' will take the whole contour to

calculate the area

The orientation of contour affects the area sign. So, this function may return a negative value. So, it should be used fabs() function to get the absolute value.

fabs(double x)

This function returns the absolute value of any floating point number. ( This is a C function, not a OpenCV function) Tracking two Triangles in a Video

Here I am going to track the two triangles in a video. The blue triangle is marked with red and the green triangle is marked with blue. /////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// #include "stdafx.h" #include #include using namespace std; IplImage* imgTracking=0;

int lastX1 = -1; int lastY1 = -1; int lastX2 = -1; int lastY2 = -1; void trackObject(IplImage* imgThresh){ CvSeq* contour; //hold the pointer to a contour CvSeq* result; //hold sequence of points of a contour CvMemStorage *storage = cvCreateMemStorage(0); //storage area for all contours

//finding all contours in the image cvFindContours(imgThresh, storage, &contour, sizeof(CvContour), CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE, cvPoint(0,0));

//iterating through each contour while(contour) { //obtain a sequence of points of the countour, pointed by the variable 'countour' result = cvApproxPoly(contour, sizeof(CvContour), storage, CV_POLY_APPROX_DP, cvContourPerimeter(contour)*0.02, 0); //if there are 3 vertices in the contour and the area of the triangle is more than 100 pixels if(result->total==3 && fabs(cvContourArea(result, CV_WHOLE_SEQ))>100 ) { //iterating through each point CvPoint *pt[3]; for(int i=0;ix + pt[1]->x + pt[2]->x )/3; int posY=( pt[0]->y + pt[1]->y + pt[2]->y )/3; if(posX > 360 ){ if(lastX1>=0 && lastY1>=0 && posX>=0 && posY>=0){ // Draw a red line from the previous point to the current point cvLine(imgTracking, cvPoint(posX, posY), cvPoint(lastX1, lastY1), cvScalar(0,0,255), 4); } lastX1 = posX; lastY1 = posY; } else{ if(lastX2>=0 && lastY2>=0 && posX>=0 && posY>=0){ // Draw a blue line from the previous point to the current point cvLine(imgTracking, cvPoint(posX, posY), cvPoint(lastX2, lastY2), cvScalar(255,0,0), 4); } lastX2 = posX; lastY2 = posY; } }

//obtain the next contour contour = contour->h_next; } cvReleaseMemStorage(&storage); } int main(){ //load the video file to the memory CvCapture *capture = cvCaptureFromAVI("E:/Projects/Robot/IESL Robot/robot/a.avi"); if(!capture){ printf("Capture failure\n"); return -1; } IplImage* frame=0; frame = cvQueryFrame(capture); if(!frame) return -1; //create a blank image and assigned to 'imgTracking' which has the same size of original video

imgTracking=cvCreateImage(cvGetSize(frame),IPL_DEPTH_8U, 3); cvZero(imgTracking); //covert the image, 'imgTracking' to black cvNamedWindow("Video");

//iterate through each frames of the video while(true){ frame = cvQueryFrame(capture); if(!frame) break; frame=cvCloneImage(frame); //smooth the original image using Gaussian kernel cvSmooth(frame, frame, CV_GAUSSIAN,3,3); //converting the original image into grayscale IplImage* imgGrayScale = cvCreateImage(cvGetSize(frame), 8, 1); cvCvtColor(frame,imgGrayScale,CV_BGR2GRAY); //thresholding the grayscale image to get better results cvThreshold(imgGrayScale,imgGrayScale,100,255,CV_THRESH_BINARY_INV); //track the possition of the ball trackObject(imgGrayScale); // Add the tracking image and the frame cvAdd(frame, imgTracking, frame); cvShowImage("Video", frame); //Clean up used images cvReleaseImage(&imgGrayScale); cvReleaseImage(&frame); //Wait 10mS int c = cvWaitKey(10); //If 'ESC' is pressed, break the loop if((char)c==27 ) break; } cvDestroyAllWindows(); cvReleaseImage(&imgTracking); cvReleaseCapture(&capture); return 0; }

/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// You can download this OpenCV visual c++ project from here. (The downloaded file is a compressed .rar folder. So, you have to extract it using Winrar or other suitable software)

OpenCV Basics parviz

Documents

Transcript of OpenCV Basics parviz