Generative models are powerful machine learning models that have the ability to probabilistically generate unique, realistic samples of data. These models can generate anything from hyper-realistic images of faces to 3D models of chairs. Recent publicly-available generative models such as ChatGPT and DALL-E have gained widespread attention and usage. However, generative models often have high-dimensional latent space representations of the data, which results in the latent space vectors used to generate outputs being difficult to interpret.

In this paper, we introduce an application for exploring generative models in hyperbolic space. We adopt the hyperboloid model of hyperbolic geometry and implement a regular tiling system for this model. We implement the hyperbolic world in the Unity game engine and link the hyperbolic world with a Flask server that produces output images from LAFITE, a state-of-the-art generative model. Finally, we run simulated experiments to evaluate the effectiveness of our system as a tool for human-in-the-loop optimization of generative models of varying dimensions.

Consider a graph, G = (V, E), with vertices V and edges E. A graph, G, is generically rigid if all continuous deformations on generic realizations of G preserve the distance between any two points, whether they are adjacent or not. While the case of graphs embedded in R 2 is well-established and solvable in polynomial time, no such algorithm exists for determining generic rigidity in R 3 . Graph neural networks (GNNs) are a class of deep learning designed to operate on graphs and extract structural information with message passing. Therefore, we develop a synthetic dataset of fixed-size graphs and train GNNs to predict generic rigidity for both 2D and 3D data. We find that GNNs significantly outperform non-neural baselines and are strong predictors of generic rigidity for most classes of graphs where rigidity can be determined within a node’s immediate local neighborhood; however, GNN performance begins to drop as the graph’s complexity increases (both in number of nodes and dimension). Though our models fail to predict generic rigidity perfectly, our experiments compare the expressive power of three di↵erent popular GNN architectures, highlight types of graphs that are challenging to predict, and propose areas of particular interest for future work. The ramifications of our work extend to several fields, including civil engineering, chemistry, and biology. Our code is available at https://github.com/sahiljain01/thesis.

Fitting a model to heterogeneous data incurs a high computational cost for models that have high predictive power. However, for small smart devices, computational capacity is a highly constraining factor. These constraints result in a computational complexity-accuracy trade-off that produces discriminatory results, particularly in applications that use models trained with data sets exhibiting domain shifts that skew towards a majority population, resulting in underrepresented minorities. This domain shift necessitates a model that is able to flexibly fit data in a computationally effective way. MoE is a type of ensemble learning that trains multiple models to master certain subtasks as well as an additional gating model that decides which model can perform the best for a given input. Since the submodels specialize in a certain task, they tend to be simpler than a singular model designed to fit all the domains in the data, which is useful for fitting heterogeneous data. In this paper, I implement a Mixture of Experts (MoE) model with a sparse gating network to mitigate the effects of this trade-off as inducing sparsity has been shown to minimize computational costs. The gating model implemented specifically is designed to use only one expert at a time instead of just a select few. The model was able to achieve high accuracies on image classification tasks for data sets with differing synthesized heterogeneity, however, due to the simplicity of the data set utilized, there was not much of a difference in accuracy when comparing results of a Multi-layer Perceptron (MLP) model. I conclude that it is possible that my specific implementation requires more fine-tuning to obtain the cost-effectiveness observed in other sparse models.

Morphological computation is the idea that intelligent systems offload computation from their control systems to their morphologies. Towards the goal of designing complex passively-adaptable intelligent systems, we learn simple adaptive functions and quantify the extent to which a learned morphology induces a passive-automatic control system. Key to our approach is the neuromechanical autoencoder framework, used to co-learn the morphology and controls of a system with a gradient based approach.

Hopfield networks as a mathematical model for associative memory have been well studied and used in a variety of applications since their inception by John Hopfield in his 1982 paper "Neural networks and physical systems with emergent collective computational abilities". In the past few years, significant improvements have been made to the storage capacity, retrieval accuracy, and efficiency of these Hopfield networks. This paper aims to highlight the recent developments in Hopfield networks by demonstrating their improved properties and some example use-cases for problems in signal filtering and classification. This was done by building a variety of Hopfield network implementations, testing and verifying their capabilities, and then applying them to settings including audio pitch denoising, MNIST handwritten digit classification, and NSynth musical instrument pitch classification. I was able to demonstrate the improved memory storage and retrieval capabilities of the continuous modern Hopfield network when compared to the simple binary Hopfield network, as well as the exponential in the number of neurons storage capacity of the new continuous Hopfield networks. Furthermore, I achieved practically usable classification accuracy results of 97.0% and 86.1% for the MNIST and NSynth datasets respectively, highlighting that Hopfield networks now present a useful framework to consider for other machine learning problems.