Home | History | Annotate | Download | only in akaze_tracking 
      1 AKAZE and ORB planar tracking {#tutorial_akaze_tracking}
      2 =============================
      3 
      4 Introduction
      5 ------------
      6 
      7 In this tutorial we will compare *AKAZE* and *ORB* local features using them to find matches between
      8 video frames and track object movements.
      9 
     10 The algorithm is as follows:
     11 
     12 -   Detect and describe keypoints on the first frame, manually set object boundaries
     13 -   For every next frame:
     14     -#  Detect and describe keypoints
     15     -#  Match them using bruteforce matcher
     16     -#  Estimate homography transformation using RANSAC
     17     -#  Filter inliers from all the matches
     18     -#  Apply homography transformation to the bounding box to find the object
     19     -#  Draw bounding box and inliers, compute inlier ratio as evaluation metric
     20 
     21 ![](images/frame.png)
     22 
     23 Data
     24 ----
     25 
     26 To do the tracking we need a video and object position on the first frame.
     27 
     28 You can download our example video and data from
     29 [here](https://docs.google.com/file/d/0B72G7D4snftJandBb0taLVJHMFk).
     30 
     31 To run the code you have to specify input and output video path and object bounding box.
     32 @code{.none}
     33 ./planar_tracking blais.mp4 result.avi blais_bb.xml.gz
     34 @endcode
     35 
     36 Source Code
     37 -----------
     38 
     39 @include cpp/tutorial_code/features2D/AKAZE_tracking/planar_tracking.cpp
     40 
     41 Explanation
     42 -----------
     43 
     44 ### Tracker class
     45 
     46 This class implements algorithm described abobve using given feature detector and descriptor
     47 matcher.
     48 
     49 -   **Setting up the first frame**
     50     @code{.cpp}
     51     void Tracker::setFirstFrame(const Mat frame, vector<Point2f> bb, string title, Stats& stats)
     52     {
     53         first_frame = frame.clone();
     54         (*detector)(first_frame, noArray(), first_kp, first_desc);
     55         stats.keypoints = (int)first_kp.size();
     56         drawBoundingBox(first_frame, bb);
     57         putText(first_frame, title, Point(0, 60), FONT_HERSHEY_PLAIN, 5, Scalar::all(0), 4);
     58         object_bb = bb;
     59     }
     60     @endcode
     61     We compute and store keypoints and descriptors from the first frame and prepare it for the
     62     output.
     63 
     64     We need to save number of detected keypoints to make sure both detectors locate roughly the same
     65     number of those.
     66 
     67 -   **Processing frames**
     68 
     69     -#  Locate keypoints and compute descriptors
     70         @code{.cpp}
     71         (*detector)(frame, noArray(), kp, desc);
     72         @endcode
     73 
     74         To find matches between frames we have to locate the keypoints first.
     75 
     76         In this tutorial detectors are set up to find about 1000 keypoints on each frame.
     77 
     78     -#  Use 2-nn matcher to find correspondences
     79         @code{.cpp}
     80         matcher->knnMatch(first_desc, desc, matches, 2);
     81         for(unsigned i = 0; i < matches.size(); i++) {
     82             if(matches[i][0].distance < nn_match_ratio * matches[i][1].distance) {
     83                 matched1.push_back(first_kp[matches[i][0].queryIdx]);
     84                 matched2.push_back(      kp[matches[i][0].trainIdx]);
     85             }
     86         }
     87         @endcode
     88         If the closest match is *nn_match_ratio* closer than the second closest one, then it's a
     89         match.
     90 
     91     -#  Use *RANSAC* to estimate homography transformation
     92         @code{.cpp}
     93         homography = findHomography(Points(matched1), Points(matched2),
     94                                     RANSAC, ransac_thresh, inlier_mask);
     95         @endcode
     96         If there are at least 4 matches we can use random sample consensus to estimate image
     97         transformation.
     98 
     99     -#  Save the inliers
    100         @code{.cpp}
    101         for(unsigned i = 0; i < matched1.size(); i++) {
    102             if(inlier_mask.at<uchar>(i)) {
    103                 int new_i = static_cast<int>(inliers1.size());
    104                 inliers1.push_back(matched1[i]);
    105                 inliers2.push_back(matched2[i]);
    106                 inlier_matches.push_back(DMatch(new_i, new_i, 0));
    107             }
    108         }
    109         @endcode
    110         Since *findHomography* computes the inliers we only have to save the chosen points and
    111         matches.
    112 
    113     -#  Project object bounding box
    114         @code{.cpp}
    115         perspectiveTransform(object_bb, new_bb, homography);
    116         @endcode
    117 
    118         If there is a reasonable number of inliers we can use estimated transformation to locate the
    119         object.
    120 
    121 Results
    122 -------
    123 
    124 You can watch the resulting [video on youtube](http://www.youtube.com/watch?v=LWY-w8AGGhE).
    125 
    126 *AKAZE* statistics:
    127 @code{.none}
    128 Matches      626
    129 Inliers      410
    130 Inlier ratio 0.58
    131 Keypoints    1117
    132 @endcode
    133 
    134 *ORB* statistics:
    135 @code{.none}
    136 Matches      504
    137 Inliers      319
    138 Inlier ratio 0.56
    139 Keypoints    1112
    140 @endcode
    141