OE 22. Module Point Cloud

Introduce a point cloud processing module in opencv

• Author: Vadim Pisarevsky
• Link: The feature request
• Status: Draft
• Platforms: All
• Complexity: a few man-months

Introduction and Rationale

In opencv_contrib there is rgbd module that does some point cloud processing. Besides, there is well-known Point Cloud Library under BSD license, which contains a lot of good stuff that can be ported to OpenCV. While we do not need to move every single piece of the code around the world into OpenCV, it makes sense to build a better foundation for 3D vision and robotics in OpenCV.

Proposed solution

Create a dedicated pclouds module in the main repository with some assorted algorithms from rgbd, calib3d and maybe PCL. Some initial set of functionality may include:

• algorithm to compute surface normals
• RGBD/visual odometry (compute camera trajectory based on the sequence of RGBD frames it observes)
• registration of (establishing correspondence between) RGB data and depth data coming from different sensors
• 3D filtering (smooth surfaces)
• 3D feature detection (corners)
• maybe kinect fusion (?)
• maybe DTAM
• ICP or such
• surface registration (there is dedicated module in opencv_contrib, but it implements a patented algorithm, maybe there are non-patented alternatives)
• 3D shape fitting
• structure from motion? (there is dedicated sfm module in opencv_contrib for that)
• building blocks for SLAM (or make it a separate module?)
• ...

Impact on existing code, compatibility

Overall, this is a loosely covered part of CV in OpenCV, so it's unlikely that many people use the existing rgbd module.

Possible alternatives

Perhaps, it should be more than 1 module and perhaps sfm and some other parts should stay in opencv_contrib for now.

References

TBD