Mobile 3D Reconstruction and Planar Region Detection Paradigms

Abstract

Context is critical. It determines how people relate to each other, how intelligent agents should act and react to their surroundings, and facilitates the absolute transfer of knowledge. In the interest of transferring as much information as possible, we as a society should be interested in finding increasingly novel and information-rich ways of capturing and sharing contexts. Fortunately, modern advances in affordable consumer sensors are making it increasingly possible for us to investigate new ways of context-sharing. In particular, the recent advent of affordable depth sensors makes 3D reconstruction a very realistic option for such information transfer. With this in mind, along with cutting-edge techniques for real-time 3D reconstruction, we propose an infrastructure that would support a two-fold iOS application. This application would gather information from an attached depth sensor and utilize it in order to construct a 3D reconstruction, intelligently identify regions of missing data, and alert the user to these regions. As a proof of concept and precursor to such an infrastructure, we construct an iOS application that utilizes raw depth and surface normal information from the depth sensor in conjunction with RGB information from an iPhone 6’s camera in order to gather all the information necessary to create a real-time point cloud based 3D representation, and that further utilizes this information to identify planar regions within the captured depth frames.