Consistency Correction through Diffusion-based Edits

Abstract

Consistency is a crucial aspect of high-quality text generation. In this paper, we propose and evaluate several approaches to improve consistency in generated text, focusing on the task of rewriting short stories to maintain consistency while making minimal changes. We modify the SSD-LM diffusion-based architecture with the addition of a consistency classifier, and incorporate a minimum-edit loss function to encourage minimal edits. We compare our methods against several baseline models and observe significant improvements in terms of perplexity and consistency. However, we also identify limitations in our evaluation metrics and note that larger autoregressive models do outperform our model. This work provides a foundation for future research on consistency correction in text generation tasks.