Improving Depth Completion With Optimization-Guided Neural Iterations For Better Robustness

Abstract

Depth completion is a computer vision task of generating a dense depth map by predicting the missing or uncertain parts from an RGB image and a sparse depth map. Some of the current depth completion models lack the ability to generalize across diverse scenarios, such as sparsity in depth map or outdoor settings. In this thesis, we discuss the novel solution OMNI-DC, which handles sparse depth maps of varying densities, and is robust to scenarios including indoor, outdoor and urban settings. I also discuss the specific contributions that I have towards OMNI-DC, including experimenting with multi-res DDI variants, implementing gradient matching loss, 3D visualizer, and generating visualizations and running experiments.