Cost-aware Replacement Policies for the iTLB

Abstract

Server-scale applications often utilize extremely large codebases, straining processor instruction memory hierarchy. As this development has led to an increasing proportion of pipeline stalls resulting from front-end operations, new micro-architectural improvements have been studied to address this issue. In this study, we evaluate how previous micro-architectural improvements have paved the way to improve front-end processing through protocols such as Fetch Directed Prefetching (FDP) and cost- aware replacement policies for caches such as EMISSARY. Our work investigates the effectiveness of extending cost-aware replacement policies to the instruction Translation Look-Aside Buffer (iTLB), a cache-like table for storing translations between virtual and physical memory addresses. The cost-aware replacement policy evaluates which translations are most important based on the performance cost to the pipeline if they were to be missed. Since the iTLB is also an essential part of accessing the cache, implementing a cost-aware replacement policy in the iTLB provides further opportunity for improvement when paired with proven cost-aware cache replacement policies. Our results show promise for cost-aware cache-replacement policies with certain iTLB parameters, showing slim across-benchmark performance improvements with minimal hardware overhead. Future work on this topic may show even further processor speedup with better profiling of iTLB miss costs and optimized size, priority, and random parameters.