1 How to Use Background Subtraction Methods {#tutorial_background_subtraction} 2 ========================================= 3 4 - Background subtraction (BS) is a common and widely used technique for generating a foreground 5 mask (namely, a binary image containing the pixels belonging to moving objects in the scene) by 6 using static cameras. 7 - As the name suggests, BS calculates the foreground mask performing a subtraction between the 8 current frame and a background model, containing the static part of the scene or, more in 9 general, everything that can be considered as background given the characteristics of the 10 observed scene. 11 12  13 14 - Background modeling consists of two main steps: 15 16 -# Background Initialization; 17 -# Background Update. 18 19 In the first step, an initial model of the background is computed, while in the second step that 20 model is updated in order to adapt to possible changes in the scene. 21 22 - In this tutorial we will learn how to perform BS by using OpenCV. As input, we will use data 23 coming from the publicly available data set [Background Models Challenge 24 (BMC)](http://bmc.univ-bpclermont.fr/) . 25 26 Goals 27 ----- 28 29 In this tutorial you will learn how to: 30 31 -# Read data from videos by using @ref cv::VideoCapture or image sequences by using @ref 32 cv::imread ; 33 -# Create and update the background model by using @ref cv::BackgroundSubtractor class; 34 -# Get and show the foreground mask by using @ref cv::imshow ; 35 -# Save the output by using @ref cv::imwrite to quantitatively evaluate the results. 36 37 Code 38 ---- 39 40 In the following you can find the source code. We will let the user chose to process either a video 41 file or a sequence of images. 42 43 Two different methods are used to generate two foreground masks: 44 -# cv::bgsegm::BackgroundSubtractorMOG 45 -# @ref cv::BackgroundSubtractorMOG2 46 47 The results as well as the input data are shown on the screen. 48 The source file can be downloaded [here ](https://github.com/Itseez/opencv/tree/master/samples/cpp/tutorial_code/video/bg_sub.cpp). 49 50 @include samples/cpp/tutorial_code/video/bg_sub.cpp 51 52 Explanation 53 ----------- 54 55 We discuss the main parts of the above code: 56 57 -# First, three Mat objects are allocated to store the current frame and two foreground masks, 58 obtained by using two different BS algorithms. 59 @code{.cpp} 60 Mat frame; //current frame 61 Mat fgMaskMOG; //fg mask generated by MOG method 62 Mat fgMaskMOG2; //fg mask fg mask generated by MOG2 method 63 @endcode 64 -# Two @ref cv::BackgroundSubtractor objects will be used to generate the foreground masks. In this 65 example, default parameters are used, but it is also possible to declare specific parameters in 66 the create function. 67 @code{.cpp} 68 Ptr<BackgroundSubtractor> pMOG; //MOG Background subtractor 69 Ptr<BackgroundSubtractor> pMOG2; //MOG2 Background subtractor 70 ... 71 //create Background Subtractor objects 72 pMOG = createBackgroundSubtractorMOG(); //MOG approach 73 pMOG2 = createBackgroundSubtractorMOG2(); //MOG2 approach 74 @endcode 75 -# The command line arguments are analysed. The user can chose between two options: 76 - video files (by choosing the option -vid); 77 - image sequences (by choosing the option -img). 78 @code{.cpp} 79 if(strcmp(argv[1], "-vid") == 0) { 80 //input data coming from a video 81 processVideo(argv[2]); 82 } 83 else if(strcmp(argv[1], "-img") == 0) { 84 //input data coming from a sequence of images 85 processImages(argv[2]); 86 } 87 @endcode 88 -# Suppose you want to process a video file. The video is read until the end is reached or the user 89 presses the button 'q' or the button 'ESC'. 90 @code{.cpp} 91 while( (char)keyboard != 'q' && (char)keyboard != 27 ){ 92 //read the current frame 93 if(!capture.read(frame)) { 94 cerr << "Unable to read next frame." << endl; 95 cerr << "Exiting..." << endl; 96 exit(EXIT_FAILURE); 97 } 98 @endcode 99 -# Every frame is used both for calculating the foreground mask and for updating the background. If 100 you want to change the learning rate used for updating the background model, it is possible to 101 set a specific learning rate by passing a third parameter to the 'apply' method. 102 @code{.cpp} 103 //update the background model 104 pMOG->apply(frame, fgMaskMOG); 105 pMOG2->apply(frame, fgMaskMOG2); 106 @endcode 107 -# The current frame number can be extracted from the @ref cv::VideoCapture object and stamped in 108 the top left corner of the current frame. A white rectangle is used to highlight the black 109 colored frame number. 110 @code{.cpp} 111 //get the frame number and write it on the current frame 112 stringstream ss; 113 rectangle(frame, cv::Point(10, 2), cv::Point(100,20), 114 cv::Scalar(255,255,255), -1); 115 ss << capture.get(CAP_PROP_POS_FRAMES); 116 string frameNumberString = ss.str(); 117 putText(frame, frameNumberString.c_str(), cv::Point(15, 15), 118 FONT_HERSHEY_SIMPLEX, 0.5 , cv::Scalar(0,0,0)); 119 @endcode 120 -# We are ready to show the current input frame and the results. 121 @code{.cpp} 122 //show the current frame and the fg masks 123 imshow("Frame", frame); 124 imshow("FG Mask MOG", fgMaskMOG); 125 imshow("FG Mask MOG 2", fgMaskMOG2); 126 @endcode 127 -# The same operations listed above can be performed using a sequence of images as input. The 128 processImage function is called and, instead of using a @ref cv::VideoCapture object, the images 129 are read by using @ref cv::imread , after individuating the correct path for the next frame to 130 read. 131 @code{.cpp} 132 //read the first file of the sequence 133 frame = imread(fistFrameFilename); 134 if(!frame.data){ 135 //error in opening the first image 136 cerr << "Unable to open first image frame: " << fistFrameFilename << endl; 137 exit(EXIT_FAILURE); 138 } 139 ... 140 //search for the next image in the sequence 141 ostringstream oss; 142 oss << (frameNumber + 1); 143 string nextFrameNumberString = oss.str(); 144 string nextFrameFilename = prefix + nextFrameNumberString + suffix; 145 //read the next frame 146 frame = imread(nextFrameFilename); 147 if(!frame.data){ 148 //error in opening the next image in the sequence 149 cerr << "Unable to open image frame: " << nextFrameFilename << endl; 150 exit(EXIT_FAILURE); 151 } 152 //update the path of the current frame 153 fn.assign(nextFrameFilename); 154 @endcode 155 Note that this example works only on image sequences in which the filename format is \<n\>.png, 156 where n is the frame number (e.g., 7.png). 157 158 Results 159 ------- 160 161 - Given the following input parameters: 162 @code{.cpp} 163 -vid Video_001.avi 164 @endcode 165 The output of the program will look as the following: 166 167  168 169 - The video file Video_001.avi is part of the [Background Models Challenge 170 (BMC)](http://bmc.univ-bpclermont.fr/) data set and it can be downloaded from the following link 171 [Video_001](http://bmc.univ-bpclermont.fr/sites/default/files/videos/evaluation/Video_001.zip) 172 (about 32 MB). 173 - If you want to process a sequence of images, then the '-img' option has to be chosen: 174 @code{.cpp} 175 -img 111_png/input/1.png 176 @endcode 177 The output of the program will look as the following: 178 179  180 181 - The sequence of images used in this example is part of the [Background Models Challenge 182 (BMC)](http://bmc.univ-bpclermont.fr/) dataset and it can be downloaded from the following link 183 [sequence 111](http://bmc.univ-bpclermont.fr/sites/default/files/videos/learning/111_png.zip) 184 (about 708 MB). Please, note that this example works only on sequences in which the filename 185 format is \<n\>.png, where n is the frame number (e.g., 7.png). 186 187 Evaluation 188 ---------- 189 190 To quantitatively evaluate the results obtained, we need to: 191 192 - Save the output images; 193 - Have the ground truth images for the chosen sequence. 194 195 In order to save the output images, we can use @ref cv::imwrite . Adding the following code allows 196 for saving the foreground masks. 197 @code{.cpp} 198 string imageToSave = "output_MOG_" + frameNumberString + ".png"; 199 bool saved = imwrite(imageToSave, fgMaskMOG); 200 if(!saved) { 201 cerr << "Unable to save " << imageToSave << endl; 202 } 203 @endcode 204 Once we have collected the result images, we can compare them with the ground truth data. There 205 exist several publicly available sequences for background subtraction that come with ground truth 206 data. If you decide to use the [Background Models Challenge (BMC)](http://bmc.univ-bpclermont.fr/), 207 then the result images can be used as input for the [BMC 208 Wizard](http://bmc.univ-bpclermont.fr/?q=node/7). The wizard can compute different measures about 209 the accuracy of the results. 210 211 References 212 ---------- 213 214 - [Background Models Challenge (BMC) website](http://bmc.univ-bpclermont.fr/) 215 - A Benchmark Dataset for Foreground/Background Extraction @cite vacavant2013benchmark 216
