Combining Tokenization Methods Towards Improved Machine Learning Models for Ancient Greek

Abstract

This thesis involves using machine learning to find errors in Greek manuscripts and suggest corrections, as well as to replace missing pieces of texts. I combined two existing forms of tokenization (the way of computationally representing language) to create a new method that leverages the benefits of these pre-existing systems. I then applied models trained using this tokenization system to find errors in the works of Michael Psellos and to repair gaps in inscriptions, in order to compare my newer model against existing models. I found this new system to have increased performance benefits for error detection and state-of-the-art performance in gap-infilling compared to a previous model created by Google DeepMind.