Investigating Automated Machine Learning Packages

Abstract

As the field of machine learning receives greater attention in the modern day given its widespread use in a number of fields and applications in addition to its limitless potential and powerful capabilities for forecasting, prediction, and classification, a desire for faster access to improved performance grows. Currently, creating and training models can be a lengthy, time-consuming process that requires some base level of knowledge to do effectively. Automated machine learning packages seek to provide a solution to this, allowing users to efficiently create models and ensembles without having to manually set hyperparameters while maintaining a high level of performance relative to manually trained models regardless of the given data set or task at hand, but much research still has to be done on their individual effectiveness as well as their performance in competition. I gathered several such packages – auto-sklearn, AutoGluon, and mljar-supervised – which I tested against each other across a variety of data sets and tasks in order to see which ones have more potential utility in a given situation. From the results, I found that while the packages all did relatively well overall, AutoGluon tended to perform the best in both regression and classification tasks, mljar-supervised’s Optuna mode tended to provide minimal gains in performance over the Explain mode and the other AutoML packages at the expense of much more time, and auto-sklearn provided average to slightly worse performance across the board compared to the other packages with a slightly longer but consistent training time. I conclude that all of the packages serve as valuable resources depending on the task but that AutoGluon’s performance combined with its shorter training times makes it the most prominent automated machine learning package for use currently.