Diving In: A Model Interpretability Approach to Sensor Fusion for Accurate and Efficient Human Activity Recognition Tasks.

Abstract

Human Activity Recognition (HAR) is a promising field of research with applications that extend to areas such as surveillance-based security, health care, and human-computer interaction. Advances in sensing technology as well as increasing demands for solutions to real-world problems have driven researchers to approach this task in novel ways. As a result, there are currently a breadth of methods, sensing technologies, and tools available for use. However, accurate and efficient activity recognition remains a challenging task which has led some researchers to the field of Sensor Fusion as a potential solution to these issues. Having the ability to combine and integrate information across sensors introduces certain advantages, such as robustness and reduced uncertainty in our data, we can achieve compared with having just a single input. However, combining data from different sensors, where data and semantic information are encoded in different ways, is a complex task of itself. We approach the goal of designing an accurate and data-efficient Sensor Fusion model through the lens of model interpretability. To our knowledge, this is the first attempt at doing so. In this paper, we present our approach towards understanding the behavior of a deep learning model, lay the foundation for a Sensor Fusion model based on our newfound insights, and propose other areas of research that promise improvements in fields such as sample efficiency, inference, and context adaptation.