Hierarchical Networks and Low-Swing Signaling to Reduce Delay in IMC Architectures

Abstract

MC (in-memory computing) is a ground-breaking paradigm for machine learning acceleration. While several architectures have been proposed, most IMC architectures revolve around weight-stationary SRAM crossbar arrays. While these arrays can provide throughput and efficiency gains, they are often bottlenecked by activation- and weight- loading costs. By analyzing how activations travel through a hierarchical IMC on-chip network, we hope to develop a model for power and energy usage attributed to activation loading. This paper establishes several parameters that are important to determining how activations move through IMC networks. Based on these parameters, it derives a high-level model for power and energy usage for different mappings of an ML (transformer) model to CIMUs (compute-in-memory units). The different parallelism parameters (Mo, Mi, and D), along with model size, strongly affect the activation-loading overhead. Furthermore, this paper looks to low-swing signaling as a potential avenue to reduce power, but also uses multiple drivers and receivers to reduce delay. The two receivers are the StrongARM latch (clocked) and an unclocked low-swing receiver.