Training Ancient Greek BERT Models to Incorporate Metrical Concerns with Ancient Text Restoration

Abstract

Ancient text restoration is a growing field within machine learning. This project builds on previous large language models trained on Ancient Greek text for the purpose of detecting scribal errors and filling textual gaps. This project addresses the problems posed by texts written in poetic meter, focusing on one specific meter: the iambic trimeter. I first manually assemble a large dataset of just under 40,000 lines in iambic trimeter. I then train multiple new models, fine-tuning an existing model on this dataset, some of which incorporate the scansion of the poetry as an extra type of embedding, which is combined with tokenization methods already incorporated into the model. I evaluate quantitatively on gap filling and detecting artificial errors. With the assistance of multiple philologists, I evaluate qualitatively on unresolved textual problems in the play Hippolytus by Euripides. I find that my new dataset solidly improves model performance, while the effect of metrical embeddings is inconclusive.