1 Back Projection {#tutorial_back_projection} 2 =============== 3 4 Goal 5 ---- 6 7 In this tutorial you will learn: 8 9 - What is Back Projection and why it is useful 10 - How to use the OpenCV function @ref cv::calcBackProject to calculate Back Projection 11 - How to mix different channels of an image by using the OpenCV function @ref cv::mixChannels 12 13 Theory 14 ------ 15 16 ### What is Back Projection? 17 18 - Back Projection is a way of recording how well the pixels of a given image fit the distribution 19 of pixels in a histogram model. 20 - To make it simpler: For Back Projection, you calculate the histogram model of a feature and then 21 use it to find this feature in an image. 22 - Application example: If you have a histogram of flesh color (say, a Hue-Saturation histogram ), 23 then you can use it to find flesh color areas in an image: 24 25 ### How does it work? 26 27 - We explain this by using the skin example: 28 - Let's say you have gotten a skin histogram (Hue-Saturation) based on the image below. The 29 histogram besides is going to be our *model histogram* (which we know represents a sample of 30 skin tonality). You applied some mask to capture only the histogram of the skin area: 31  32  33 34 - Now, let's imagine that you get another hand image (Test Image) like the one below: (with its 35 respective histogram): 36  37  38 39 40 - What we want to do is to use our *model histogram* (that we know represents a skin tonality) to 41 detect skin areas in our Test Image. Here are the steps 42 -# In each pixel of our Test Image (i.e. \f$p(i,j)\f$ ), collect the data and find the 43 correspondent bin location for that pixel (i.e. \f$( h_{i,j}, s_{i,j} )\f$ ). 44 -# Lookup the *model histogram* in the correspondent bin - \f$( h_{i,j}, s_{i,j} )\f$ - and read 45 the bin value. 46 -# Store this bin value in a new image (*BackProjection*). Also, you may consider to normalize 47 the *model histogram* first, so the output for the Test Image can be visible for you. 48 -# Applying the steps above, we get the following BackProjection image for our Test Image: 49 50  51 52 -# In terms of statistics, the values stored in *BackProjection* represent the *probability* 53 that a pixel in *Test Image* belongs to a skin area, based on the *model histogram* that we 54 use. For instance in our Test image, the brighter areas are more probable to be skin area 55 (as they actually are), whereas the darker areas have less probability (notice that these 56 "dark" areas belong to surfaces that have some shadow on it, which in turns affects the 57 detection). 58 59 Code 60 ---- 61 62 - **What does this program do?** 63 - Loads an image 64 - Convert the original to HSV format and separate only *Hue* channel to be used for the 65 Histogram (using the OpenCV function @ref cv::mixChannels ) 66 - Let the user to enter the number of bins to be used in the calculation of the histogram. 67 - Calculate the histogram (and update it if the bins change) and the backprojection of the 68 same image. 69 - Display the backprojection and the histogram in windows. 70 - **Downloadable code**: 71 72 -# Click 73 [here](https://github.com/Itseez/opencv/tree/master/samples/cpp/tutorial_code/Histograms_Matching/calcBackProject_Demo1.cpp) 74 for the basic version (explained in this tutorial). 75 -# For stuff slightly fancier (using H-S histograms and floodFill to define a mask for the 76 skin area) you can check the [improved 77 demo](https://github.com/Itseez/opencv/tree/master/samples/cpp/tutorial_code/Histograms_Matching/calcBackProject_Demo2.cpp) 78 -# ...or you can always check out the classical 79 [camshiftdemo](https://github.com/Itseez/opencv/tree/master/samples/cpp/camshiftdemo.cpp) 80 in samples. 81 82 - **Code at glance:** 83 @include samples/cpp/tutorial_code/Histograms_Matching/calcBackProject_Demo1.cpp 84 85 Explanation 86 ----------- 87 88 -# Declare the matrices to store our images and initialize the number of bins to be used by our 89 histogram: 90 @code{.cpp} 91 Mat src; Mat hsv; Mat hue; 92 int bins = 25; 93 @endcode 94 -# Read the input image and transform it to HSV format: 95 @code{.cpp} 96 src = imread( argv[1], 1 ); 97 cvtColor( src, hsv, COLOR_BGR2HSV ); 98 @endcode 99 -# For this tutorial, we will use only the Hue value for our 1-D histogram (check out the fancier 100 code in the links above if you want to use the more standard H-S histogram, which yields better 101 results): 102 @code{.cpp} 103 hue.create( hsv.size(), hsv.depth() ); 104 int ch[] = { 0, 0 }; 105 mixChannels( &hsv, 1, &hue, 1, ch, 1 ); 106 @endcode 107 as you see, we use the function @ref cv::mixChannels to get only the channel 0 (Hue) from 108 the hsv image. It gets the following parameters: 109 110 - **&hsv:** The source array from which the channels will be copied 111 - **1:** The number of source arrays 112 - **&hue:** The destination array of the copied channels 113 - **1:** The number of destination arrays 114 - **ch[] = {0,0}:** The array of index pairs indicating how the channels are copied. In this 115 case, the Hue(0) channel of &hsv is being copied to the 0 channel of &hue (1-channel) 116 - **1:** Number of index pairs 117 118 -# Create a Trackbar for the user to enter the bin values. Any change on the Trackbar means a call 119 to the **Hist_and_Backproj** callback function. 120 @code{.cpp} 121 char* window_image = "Source image"; 122 namedWindow( window_image, WINDOW_AUTOSIZE ); 123 createTrackbar("* Hue bins: ", window_image, &bins, 180, Hist_and_Backproj ); 124 Hist_and_Backproj(0, 0); 125 @endcode 126 -# Show the image and wait for the user to exit the program: 127 @code{.cpp} 128 imshow( window_image, src ); 129 130 waitKey(0); 131 return 0; 132 @endcode 133 -# **Hist_and_Backproj function:** Initialize the arguments needed for @ref cv::calcHist . The 134 number of bins comes from the Trackbar: 135 @code{.cpp} 136 void Hist_and_Backproj(int, void* ) 137 { 138 MatND hist; 139 int histSize = MAX( bins, 2 ); 140 float hue_range[] = { 0, 180 }; 141 const float* ranges = { hue_range }; 142 @endcode 143 -# Calculate the Histogram and normalize it to the range \f$[0,255]\f$ 144 @code{.cpp} 145 calcHist( &hue, 1, 0, Mat(), hist, 1, &histSize, &ranges, true, false ); 146 normalize( hist, hist, 0, 255, NORM_MINMAX, -1, Mat() ); 147 @endcode 148 -# Get the Backprojection of the same image by calling the function @ref cv::calcBackProject 149 @code{.cpp} 150 MatND backproj; 151 calcBackProject( &hue, 1, 0, hist, backproj, &ranges, 1, true ); 152 @endcode 153 all the arguments are known (the same as used to calculate the histogram), only we add the 154 backproj matrix, which will store the backprojection of the source image (&hue) 155 156 -# Display backproj: 157 @code{.cpp} 158 imshow( "BackProj", backproj ); 159 @endcode 160 -# Draw the 1-D Hue histogram of the image: 161 @code{.cpp} 162 int w = 400; int h = 400; 163 int bin_w = cvRound( (double) w / histSize ); 164 Mat histImg = Mat::zeros( w, h, CV_8UC3 ); 165 166 for( int i = 0; i < bins; i ++ ) 167 { rectangle( histImg, Point( i*bin_w, h ), Point( (i+1)*bin_w, h - cvRound( hist.at<float>(i)*h/255.0 ) ), Scalar( 0, 0, 255 ), -1 ); } 168 169 imshow( "Histogram", histImg ); 170 @endcode 171 172 Results 173 ------- 174 175 Here are the output by using a sample image ( guess what? Another hand ). You can play with the 176 bin values and you will observe how it affects the results: 177  178  179  180
