Modernizing Scalable Application-Level Quantum Benchmarking: Extending SupermarQ to CUDA-Q and Newer Quantum Machines

Abstract

As quantum computers continue to grow in qubit counts and capabilities, such as improved gate fidelities and longer coherence times, it is essential to develop scalable evaluation tools that accurately characterize performance and limitations across technologies. My senior thesis modernizes and scales SupermarQ, a hardware-agnostic, application-level quantum benchmarking suite, by enabling and evaluating benchmarks of increasing system size and scaling on new software and hardware platforms. We analyze feature vector coverage under scaling and updated specifications, finding that even small changes in features or circuit implementations can significantly alter feature maps, exposing sensitivities and gaps in capturing all properties of the suite. We implement the SupermarQ suite in NVIDIA’s CUDA-Q programming framework and show that GPU-accelerated simulation enables efficient scaling compared to CPU execution, though incurring tradeoffs in flexibility. We extend evaluation to real quantum hardware, including one of Duke University’s trapped-ion STAQ machines, running the first application-level programs and contributing to the iterative design-and-evaluation process. When run at increased sizes on modern IBM quantum processors, benchmarks such as GHZ and Mermin-Bell show improved scalability, while error-correction-inspired benchmarks remain challenging, and QAOA results reveal that compilation can significantly impact observed performance. Overall, we show that while quantum hardware has advanced since SupermarQ's creation in 2022, performance is still strongly governed by the full software-hardware stack as well as benchmark specification, motivating continued refinement of evaluation methodologies to better capture end-to-end system behaviors of quantum computers.