Generative Modeling for Chemistry: Unconditional Atomic Diffusion Models for Small Molecules

Abstract

Generating chemically valid and geometrically accurate 3D molecular structures is an important challenge in computational chemistry, with direct implications for drug discovery and molecular design. This thesis explores unconditional generation of small molecules by modifying ChefNMR (CHemical Elucidation From NMR), a Diffusion Transformer (DiT) designed for generating 3D molecular structures conditioned on nuclear magnetic resonance (NMR) spectra. We hypothesize that improving the generative model in the unconditional setting will lead to performance benefits when incorporated back into the conditional setting. We first implement a baseline unconditional model adapted directly from ChefNMR called Uncond-ChefNMR. Subsequently, we introduce modifications inspired by recent literature such as removing explicit hydrogens from the training data, adding positional encodings, increasing rotation augmentation, tuning the learning rate for the unconditional regime, and applying Proteina-style SDE sampling at inference time. The resulting model achieves competitive performance on chemical and geometric validity metrics on our filtered subset of GEOM-Drugs. Beyond model performance, our experiments reveal some broader insights: precise coordinate placement remains the primary bottleneck for molecular validity, non-equivariant architectures struggle with explicit hydrogen representations, and scaling experiments suggest potential improvements by increasing dataset and model size. These findings provide the foundation for integrating Uncond-ChefNMR’s improvements back into the conditional framework and for scaling to more complex molecules.