Moving from Recognition to Reasoning in Image Captioning

Abstract

Image captioning is an artificial intelligence (AI) task at the intersection of vision and language. Current approaches to the task are recognition-based, leading to models that struggle to reason about image content and context. We explore the current state of image captioning and offer solutions for advancing the task from recognition to reasoning, specifically through the use of unpaired training data and evaluation based on a novel discriminativeness metric.