Peeling Back the Layers: Application- and Network-Level Characterization of ChatGPT

Abstract

Large language models are rapidly becoming a significant component of Internet traffic. On Princeton University's campus network, Anthropic's Claude AI Chatbot climbed from the 73rd to the 20th largest service by outbound traffic volume in just nine months. Yet surprisingly little is known about what this traffic looks like on the wire. This thesis presents the first fine-grained, multi-layer characterization of a production LLM service's network behavior, using ChatGPT as a case study. We examine ChatGPT sessions from two complementary vantage points: the browser's application-layer view, where every request is visible in plaintext, and the network operator's view, where the same sessions appear as encrypted packets. At the application layer, we find that a single conversation turn triggers a fixed multi-stage protocol---prepare, SSE stream, status polling, and speculative prepare---surrounded by substantial startup, telemetry, and bot mitigation traffic. At the packet layer, we identify the exact inference stream and show that conversation turns, and even internal turn sub-stages, are individually identifiable in encrypted traces through differences in packet directionality, timing, and burst structure. All inference traffic is carried over a single persistent HTTP/2 connection with downstream packet sizes consistently below the TCP maximum segment size, indicating that token delivery is shaped by the serving stack rather than the network. These findings establish a baseline for understanding LLM inference as a network workload and provide a methodology designed to be extended to other providers, modalities, and network environments as this rapidly evolving workload continues to take an increasingly larger share of all traffic.