On the Two-Sample Statistic Approach to Generative Adversarial Networks

Abstract

We investigate the deeper use of maximum mean discrepancy (MMD), a statistical measure of the distance between distributions, in training generative adversarial networks (GAN), a framework for generative modeling using deep neural networks. The algorithm that uses MMD as a criterion to train generative models parametrized by deep neural network is called generative moment matching networks (GMMN).One of the goals of this work is to understand when MMD is a more effective loss function for training neural samplers than the GAN objective. By performing experiments with simulated data, we found that the original GAN actually performs worse than GMMN when the data does not have low-dimensional structure.We explore using extensions of MMD as the loss criterion in GMMN. In particular, these extensions are adaptive to the data. Our results suggest we could achieve state-of-the-art results with GMMN by using more sophisticated variants of MMD. We also show that MMD can be used as a regularizer to improve the stability of GANs.