Improve slides and examples after KRoC 2017 tutorial

Author: sunglok

3dv_tutorial.pdf

@@ -0,0 +1,60 @@
+## An Invitation to 3D Vision: A Tutorial for Everyone
+
+_An Invitation to 3D Vision_ is a introductory tutorial on 3D vision (a.k.a. visual geometry or geometric vision). It aims to make beginners understand basic theory of 3D vision and implement their own applications using [OpenCV][]. In addition to tutorial slides, we provide a set of example codes. The example codes are written as short as possible (__less than 100 lines__) to improve readability and involve many interesting and practical applications.
+
+* Download tutorial slides [Link][https://github.com/sunglok/3dv_tutorial/releases/download/misc/3dv_slides.pdf]
+* Download example codes in a ZIP file [Link][https://github.com/sunglok/3dv_tutorial/archive/master.zip]
+* Download binaries and headers for [OpenCV][] and [CLAPACK][] for Windows [Link][https://github.com/sunglok/3dv_tutorial/releases/download/misc/EXTERNAL4Windows.32bit.zip]
+
+### What does its name come from?
+* The tutorial title, _An Invitation to 3D Vision_, came from [a legendary book by Yi Ma, Stefano Soatto, Jana Kosecka, and Shankar S. Sastry][http://vision.ucla.edu/MASKS/]. We wish that this tutorial will be the first gentle invitation card for beginners in 3D vision and explorers from other fields.
+* The subtitle, _for everyone_, came from [Prof. Kim's online lecture][https://hunkim.github.io/ml/] (in Korean). This tutorial is also intended not only for students and researchers in academia, but also for hobbyists and developers in industries. We tried to describe important and typical problems and their solutions in [OpenCV][]. We hope readers understand it without serious mathematical background.
+
+### Example descriptions
+ * __Single-view Geometry__
+  * Camera Projection Model
+    * Simple Camera Calibration and Object Localization
+    * Image Formation: [image_formation.cpp][]
+    * Geometric Distortion Correction: [distortion_correction.cpp][]
+   * General 2D-3D Geometry
+    * Camera Calibration: [camera_calibration.cpp][]
+    * Camera Pose Estimation (Chessboard Version): [pose_estimation_chessboard.cpp][]
+ * __Two-view Geometry__
+  * Planar 2D-2D Geometry (Projective Geometry)
+    * Perspective Distortion Correction: [perspective_correction.cpp][]
+    * Planar Image Stitching: [image_stitching.cpp][]
+    * 2D Video Stabilization: [video_stabilization.cpp][]
+  * General 2D-2D Geometry (Epipolar Geometry)
+    * Monocular Visual Odometry (Epipolar Version): [visual_odometry_epipolar.cpp][]
+    * Triangulation (Two-view Reconstruction): [triangulation.cpp][]
+ * __Multi-view Geometry__
+  * Bundle Adjustment using cvsba (Multiple-view Reconstruction): [bundle_adjustment.cpp][]
+  * Sparse and Dense 3D Reconstruction using VisualSFM
+ * __Correspondence Problem__
+  * Line Fitting with RANSAC: [ransac_line.cpp][]
+
+### Dependency
+ * [OpenCV][] (> 3.0.0, 3-clause BSD License)
+  * _OpenCV_ is a base of all example codes for linear algebra, vision algorithms, image/video manipulation, and GUI.
+ * [cvsba][] (GPL): An OpenCV wrapper for sba library
+  * _cvsba_ is used for bundle adjustment and already included in EXTERNAL directory.
+ * [CLAPACK][] (Public Domain): f2c'ed version of LAPACK
+  * _CLAPACK_ is necessary for cvsba.
+
+### Contact
+ * [Sunglok Choi](http://sites.google.com/site/sunglok/) (sunglok AT hanmail DOT net)
+
+[OpenCV][http://opencv.org/]
+[cvsba][https://www.uco.es/investiga/grupos/ava/node/39]
+[CLAPACK][http://www.netlib.org/clapack/]
+[image_formation.cpp][https://github.com/sunglok/3dv_tutorial/blob/master/src/image_formation.cpp]
+[distortion_correction.cpp][https://github.com/sunglok/3dv_tutorial/blob/master/src/distortion_correction.cpp]
+[camera_calibration.cpp][https://github.com/sunglok/3dv_tutorial/blob/master/src/camera_calibration.cpp]
+[pose_estimation_chessboard.cpp][https://github.com/sunglok/3dv_tutorial/blob/master/src/pose_estimation_chessboard.cpp]
+[perspective_correction.cpp][https://github.com/sunglok/3dv_tutorial/blob/master/src/perspective_correction.cpp]
+[image_stitching.cpp][https://github.com/sunglok/3dv_tutorial/blob/master/src/image_stitching.cpp]
+[video_stabilization.cpp][https://github.com/sunglok/3dv_tutorial/blob/master/src/video_stabilization.cpp]
+[visual_odometry_epipolar.cpp][https://github.com/sunglok/3dv_tutorial/blob/master/src/visual_odometry_epipolar.cpp]
+[triangulation.cpp][https://github.com/sunglok/3dv_tutorial/blob/master/src/triangulation.cpp]
+[bundle_adjustment.cpp][https://github.com/sunglok/3dv_tutorial/blob/master/src/bundle_adjustment.cpp]
+[ransac_line.cpp][https://github.com/sunglok/3dv_tutorial/blob/master/src/ransac_line.cpp]

README.md

@@ -13,7 +13,7 @@ Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "camera_calibration", "camer
 EndProject
 Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "distortion_correction", "distortion_correction\distortion_correction.vcxproj", "{E5444B55-196A-4A71-9BDE-C42E830B87B0}"
 EndProject
-Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pose_estimation", "pose_estimation\pose_estimation.vcxproj", "{E28D077A-B8C3-4391-AE2E-757E72C87BF6}"
+Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pose_estimation_chessboard", "pose_estimation_chessboard\pose_estimation_chessboard.vcxproj", "{E28D077A-B8C3-4391-AE2E-757E72C87BF6}"
 EndProject
 Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "perspective_correction", "perspective_correction\perspective_correction.vcxproj", "{53A4EA3F-D463-4D4F-A991-77A16129D664}"
 EndProject

README.mdk

@@ -84,9 +84,9 @@
     </Link>
   </ItemDefinitionGroup>
   <ItemGroup>
-    <ClCompile Include="..\..\src\pose_estimation.cpp" />
+    <ClCompile Include="..\..\src\pose_estimation_chessboard.cpp" />
   </ItemGroup>
   <Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
   <ImportGroup Label="ExtensionTargets">
   </ImportGroup>
-</Project>
+</Project>

msvs/3dv_tutorial.sln

@@ -8,7 +8,7 @@ int main(void)
     int n_views = 5;
 
     // Load multiple views of 'box.xyz'
-    // c.f. You need to run 'image_formation%02d.cpp' to generate point observation.
+    // c.f. You need to run 'image_formation.cpp' to generate point observation.
     //      You can apply Gaussian noise by change value of 'camera_noise' if necessay.
     std::vector<std::vector<cv::Point2d> > xs;
     for (int i = 0; i < n_views; i++)
@@ -92,4 +92,4 @@ int main(void)
         fprintf(fout, "%f %f %f\n", Xs[i].x, Xs[i].y, Xs[i].z);
     fclose(fout);
     return 0;
-}
+}

…/pose_estimation/pose_estimation.vcxproj …board/pose_estimation_chessboard.vcxprojmsvs/pose_estimation/pose_estimation.vcxproj renamed to msvs/pose_estimation_chessboard/pose_estimation_chessboard.vcxproj msvs/pose_estimation/pose_estimation.vcxproj renamed to msvs/pose_estimation_chessboard/pose_estimation_chessboard.vcxproj

@@ -70,7 +70,6 @@ int main(void)
         fclose(fout);
     }
 
-    std::cout << "Press any key to terminate tihs program!" << std::endl;
     cv::waitKey(0);
     return 0;
-}
+}

…_estimation/pose_estimation.vcxproj.user …/pose_estimation_chessboard.vcxproj.usermsvs/pose_estimation/pose_estimation.vcxproj.user renamed to msvs/pose_estimation_chessboard/pose_estimation_chessboard.vcxproj.user msvs/pose_estimation/pose_estimation.vcxproj.user renamed to msvs/pose_estimation_chessboard/pose_estimation_chessboard.vcxproj.user

@@ -11,25 +11,22 @@ int main(void)
     cv::Mat gray1, gray2;
     if (image1.channels() > 1)
     {
-        cv::cvtColor(image1, gray1, CV_RGB2GRAY);
-        cv::cvtColor(image2, gray2, CV_RGB2GRAY);
+        cv::cvtColor(image1, gray1, cv::COLOR_RGB2GRAY);
+        cv::cvtColor(image2, gray2, cv::COLOR_RGB2GRAY);
     }
     else
     {
         gray1 = image1.clone();
         gray2 = image2.clone();
     }
-    cv::Ptr<cv::FeatureDetector> detector = cv::xfeatures2d::SURF::create();  // SURF features
+    cv::Ptr<cv::FeatureDetector> fdetector = cv::ORB::create();
     std::vector<cv::KeyPoint> keypoint1, keypoint2;
-    detector->detect(gray1, keypoint1);
-    detector->detect(gray2, keypoint2);
-    cv::Ptr<cv::FeatureDetector> extractor = cv::xfeatures2d::SURF::create(); // SURF descriptors
     cv::Mat descriptor1, descriptor2;
-    extractor->compute(gray1, keypoint1, descriptor1);
-    extractor->compute(gray2, keypoint2, descriptor2);
-    cv::FlannBasedMatcher matcher; // Approximate Nearest Neighbors (ANN) matcher
+    fdetector->detectAndCompute(gray1, cv::Mat(), keypoint1, descriptor1);
+    fdetector->detectAndCompute(gray2, cv::Mat(), keypoint2, descriptor2);
+    cv::Ptr<cv::DescriptorMatcher> fmatcher = cv::DescriptorMatcher::create("BruteForce-Hamming");
     std::vector<cv::DMatch> match;
-    matcher.match(descriptor1, descriptor2, match);
+    fmatcher->match(descriptor1, descriptor2, match);
 
     // Calculate planar homography and merge them
     std::vector<cv::Point2f> points1, points2;
@@ -38,14 +35,20 @@ int main(void)
         points1.push_back(keypoint1.at(match.at(i).queryIdx).pt);
         points2.push_back(keypoint2.at(match.at(i).trainIdx).pt);
     }
-    cv::Mat H = cv::findHomography(points2, points1, cv::RANSAC);
-    cv::Mat merge;
-    cv::warpPerspective(image2, merge, H, cv::Size(image1.cols * 2, image1.rows));
-    merge.colRange(0, image1.cols) = image1 * 1; // Copy
+    cv::Mat inlier_mask;
+    cv::Mat H = cv::findHomography(points2, points1, inlier_mask, cv::RANSAC);
+
+    cv::Mat merged;
+    cv::warpPerspective(image2, merged, H, cv::Size(image1.cols * 2, image1.rows));
+    merged.colRange(0, image1.cols) = image1 * 1; // Copy
 
     // Show the merged image
-    cv::imshow("3DV Tutorial: Image Stitching", merge);
-    printf("Press any key to terminate tihs program!\n");
+    cv::Mat original, matched;
+    cv::drawMatches(gray1, keypoint1, gray2, keypoint2, match, matched, cv::Scalar::all(-1), cv::Scalar::all(-1), inlier_mask);
+    cv::hconcat(image1, image2, original);
+    cv::vconcat(original, matched, matched);
+    cv::vconcat(matched, merged, merged);
+    cv::imshow("3DV Tutorial: Image Stitching", merged);
     cv::waitKey(0);
     return 0;
-}
+}

src/bundle_adjustment.cpp

@@ -32,30 +32,32 @@
 #include <opencv2/viz.hpp>
 
 // Include extra (contrib) modules
-#include <opencv2/opencv.hpp>
-#include <opencv2/aruco.hpp>
-#include <opencv2/bgsegm.hpp>
-#include <opencv2/bioinspired.hpp>
-#include <opencv2/ccalib.hpp>
-#include <opencv2/dataset.hpp>
-#include <opencv2/dpm.hpp>
-#include <opencv2/face.hpp>
-#include <opencv2/fuzzy.hpp>
-#include <opencv2/line_descriptor.hpp>
-#include <opencv2/optflow.hpp>
-#include <opencv2/phase_unwrapping.hpp>
-#include <opencv2/plot.hpp>
-#include <opencv2/rgbd.hpp>
-#include <opencv2/saliency.hpp>
-#include <opencv2/stereo.hpp>
-#include <opencv2/structured_light.hpp>
-#include <opencv2/surface_matching.hpp>
-#include <opencv2/text.hpp>
-#include <opencv2/tracking.hpp>
-#include <opencv2/xfeatures2d.hpp>
-#include <opencv2/ximgproc.hpp>
-#include <opencv2/xobjdetect.hpp>
-#include <opencv2/xphoto.hpp>
+#ifdef USE_OPENCV_CONTRIB
+#   include <opencv2/opencv.hpp>
+#   include <opencv2/aruco.hpp>
+#   include <opencv2/bgsegm.hpp>
+#   include <opencv2/bioinspired.hpp>
+#   include <opencv2/ccalib.hpp>
+#   include <opencv2/dataset.hpp>
+#   include <opencv2/dpm.hpp>
+#   include <opencv2/face.hpp>
+#   include <opencv2/fuzzy.hpp>
+#   include <opencv2/line_descriptor.hpp>
+#   include <opencv2/optflow.hpp>
+#   include <opencv2/phase_unwrapping.hpp>
+#   include <opencv2/plot.hpp>
+#   include <opencv2/rgbd.hpp>
+#   include <opencv2/saliency.hpp>
+#   include <opencv2/stereo.hpp>
+#   include <opencv2/structured_light.hpp>
+#   include <opencv2/surface_matching.hpp>
+#   include <opencv2/text.hpp>
+#   include <opencv2/tracking.hpp>
+#   include <opencv2/xfeatures2d.hpp>
+#   include <opencv2/ximgproc.hpp>
+#   include <opencv2/xobjdetect.hpp>
+#   include <opencv2/xphoto.hpp>
+#endif
 
 #ifdef _WIN32
 #   ifdef _DEBUG