Synthetic Data Generation for 6DOF Object Rearrangement

Abstract

State of the art methods for object rearrangement tasks will need better labeled, and fuller distributions of, data to train on. Many difficult object classes, like shiny or transparent objects, are important for deployment scenarios like kitchens and factories. Furthermore, challenges to labeling, like occlusions, are essential for a model to be able to handle in more complex or cluttered environments. We approach these challenges in several ways: improved data collection techniques, semi-automatic data labeling pipelines, and synthetic data generation. We believe that synthetic data will continue to close the sim to real gap, and due to its better coverage of the underlying distribution and perfect labels, will become a more cost effective solution to training vision problems like 6DOF object rearrangement. This paper focuses on techniques for synthetic data generation for the object rearrangement task as well as methods to improve real world data collection for benchmarking synthetic datasets.