Finding the Right Group: Difficulty Measurement, False Group Analysis, and Automated Solving in NYT Connections

Abstract

NYT Connections is a daily word-grouping puzzle that requires players to sort 16 words into four thematically linked groups of four. The puzzle’s difficulty arises from semantic ambiguity and deliberate misdirection, including “false groups” — sets of four words that form plausible but incorrect categories. This thesis analyzes Connections puzzles through three lenses using natural language processing methods. First, embedding-based metrics are developed to quantify puzzle difficulty. Group cohesion (mean within-group cosine similarity) and silhouette score correlate negatively with ground-truth difficulty ratings from the NYT, confirming that harder puzzles have less-separated semantic clusters. These correlations are modest, indicating that a substantial portion of perceived difficulty lies beyond what embeddings can capture. Second, false groups are formalized as a computable object in embedding space. A strong false group is defined as any four-word subset whose cohesion exceeds that of the weakest true group by a fixed margin δ. Applied to 226 labeled puzzles, this definition identifies strong false groups in 40.7% of puzzles. An overlap signature taxonomy reveals that near-miss groups — differing from a true solution by exactly one word — are the most common false group type at 46%. Third, an automated solver pipeline is developed and evaluated through a five-solver ablation study. Starting from a greedy baseline that achieves 0% solve rate, beam search, WordNet lexical augmentation, and iterative feedback are added incrementally. The full feedback-aware pipeline achieves a 15.2% full puzzle solve rate on a held-out test set of 46 puzzles, with top-1 accuracy of 56.5%.