Sharif Digital Repository

For many years, computer designers benefitted from Moore's law and Dennard scaling to significantly improve the speed of single-core processors. The failure of Dennard scaling pushed the computer industry toward homogenous multicore processors for the performance improvement to continue without significant increase in power consumption. Unfortunately, even homogeneous multicore processors cannot offer the level of energy efficiency required to operate all the cores at the same time in today's and especially tomorrow's technologies. As a result of lack of energy efficiency, not all the cores in a multicore processor can be functional at the same time. This phenomenon is referred to as dark... 

Many applications, including big-data server applications, frequently encounter data misses. Consequently, they lose significant performance potential. Fortunately, data accesses of many of these applications follow temporal correlations, which means data accesses repeat over time. Temporal correlations occur because applications usually consist of loops, and hence, the sequence of instructions that constitute the body of a loop repeats many times, leading to data access repetition. Temporal data prefetchers take advantage of temporal correlation to predict and prefetch future memory accesses. In this chapter, we introduce the concept of temporal prefetching and present two instances of... 

Many applications extensively use data objects with a regular and fixed layout, which leads to the recurrence of access patterns over memory regions. Spatial data prefetching techniques exploit this phenomenon to prefetch future memory references and hide their long latency. Spatial prefetchers are particularly of interest because they usually only need a small storage budget. In this chapter, we introduce the concept of spatial prefetching and present two instances of spatial data prefetchers, SMS and VLDP. © 2021 Elsevier Inc  

Many applications, including big-data server applications, frequently encounter data misses. Consequently, they lose significant performance potential. Fortunately, data accesses of many of these applications follow temporal correlations, which means data accesses repeat over time. Temporal correlations occur because applications usually consist of loops, and hence, the sequence of instructions that constitute the body of a loop repeats many times, leading to data access repetition. Temporal data prefetchers take advantage of temporal correlation to predict and prefetch future memory accesses. In this chapter, we introduce the concept of temporal prefetching and present two instances of... 

Many applications extensively use data objects with a regular and fixed layout, which leads to the recurrence of access patterns over memory regions. Spatial data prefetching techniques exploit this phenomenon to prefetch future memory references and hide their long latency. Spatial prefetchers are particularly of interest because they usually only need a small storage budget. In this chapter, we introduce the concept of spatial prefetching and present two instances of spatial data prefetchers, SMS and VLDP. © 2022 Elsevier Inc  

Chip multiprocessors (CMPs) require a low-latency interconnect fabric network-on-chip (NoC) to minimize processor stall time on instruction and data accesses that are serviced by the last-level cache (LLC). While packet-switched mesh interconnects sacrifice performance of many-core processors due to NoC-induced delays, existing circuit-switched interconnects do not offer lower network delays as they cannot hide the time it takes to set up a circuit. To address this problem, this work introduces CIMA - a hybrid circuit-switched and packet-switched mesh-based interconnection network that affords low LLC access delays at a small area cost. CIMA uses virtual cut-through (VCT) switching for short... 

Server workloads benefit from execution on many-core processors due to their massive request-level parallelism. A key characteristic of server workloads is the large instruction footprints. While a shared last-level cache (LLC) captures the footprints, it necessitates a low-latency network-on-chip (NOC) to minimize the core stall time on accesses serviced by the LLC. As strict quality-of-service requirements preclude the use of lean cores in server processors, we observe that even state-of-the-art single-cycle multi-hop NOCs are far from ideal because they impose significant NOC-induced delays on the LLC access latency, and diminish performance. Most of the NOC delay is due to per-hop... 

Data prefetching, i.e., the act of predicting an application's future memory accesses and fetching those that are not in the on-chip caches, is a well-known and widely used approach to hide the long latency of memory accesses. The fruitfulness of data prefetching is evident to both industry and academy: Nowadays, almost every high-performance processor incorporates a few data prefetchers for capturing various access patterns of applications; besides, there is a myriad of proposals for data prefetching in the research literature, where each proposal enhances the efficiency of prefetching in a specific way. In this survey, we evaluate the effectiveness of data prefetching in the context of... 

Applications extensively use data objects with a regular and fixed layout, which leads to the recurrence of access patterns over memory regions. Spatial data prefetching techniques exploit this phenomenon to prefetch future memory references and hide the long latency of DRAM accesses. While state-of-the-art spatial data prefetchers are effective at reducing the number of data misses, we observe that there is still significant room for improvement. To select an access pattern for prefetching, existing spatial prefetchers associate observed access patterns to either a short event with a high probability of recurrence or a long event with a low probability of recurrence. Consequently, the... 

This paper proposes Blenda, a dynamically-partitioned memory-cache blend architecture for giga-scale die-stacked DRAMs. Blenda architects the stacked DRAM partly as memory and partly as cache, and dynamically adjusts each part's size to workloads' demands. The memory part hosts hot data objects and serves requests to them efficiently (i.e., without metadata overheads). The cache part captures transient data and filters requests to bandwidth-limited off-chip DRAM. Blenda provides three key contributions: (i) Blenda partitions stacked DRAM's capacity in a workload-aware manner: different workloads enjoy different memory-cache configurations. (ii) Blenda is reactive: the configuration is... 

Memory-intensive workloads operate on massive amounts of data that cannot be captured by last-level caches (LLCs) of modern processors. Consequently, processors encounter frequent off-chip misses, and hence, lose significant performance potential. One of the components of a modern processor that has a prominent influence on the off-chip miss traffic is LLC's replacement policy. Existing processors employ a variation of least recently used (LRU) policy to determine the victim for replacement. Unfortunately, there is a large gap between what LRU offers and that of Belady's MIN, which is the optimal replacement policy. Belady's MIN requires selecting a victim with the longest reuse distance,... 

Memory-intensive workloads operate on massive amounts of data that cannot be captured by last-level caches (LLCs) of modern processors. Consequently, processors encounter frequent off-chip misses, and hence, lose significant performance potential. One of the components of a modern processor that has a prominent influence on the off-chip miss traffic is LLC's replacement policy. Existing processors employ a variation of least recently used (LRU) policy to determine the victim for replacement. Unfortunately, there is a large gap between what LRU offers and that of Belady's MIN, which is the optimal replacement policy. Belady's MIN requires selecting a victim with the longest reuse distance,... 

L1 instruction(L1-l) cache misses are a source of performance bottleneck. While many instruction prefetchers have been proposed, most of them leave a considerable potential uncovered. In 2011, Proactive Instruction Fetch (PIF) showed that a hardware prefetcher could effectively eliminate all instruction-cache misses. However, its enormous storage cost makes it impractical. Consequently, reducing the storage cost was the main research focus in instruction prefetching in the past decade. Several instruction prefetchers, including RDIP and Shotgun, were proposed to offer PIF-level performance with significantly lower storage overhead. However, our findings show that there is a considerable... 

We introduced several styles of data prefetching in the past three chapters. The introduced data prefetchers were known for a long time, sometimes for decades. In this chapter, we introduce several state-of-the-art data prefetchers, which have been introduced in the past few years. In particular, we introduce DOMINO, BINGO, MLOP, and RUNAHEAD METADATA. © 2022 Elsevier Inc  

We introduced several data prefetchers and qualitatively discussed their strengths and weaknesses. Without quantitative evaluation, the true strengths and weaknesses of a data prefetcher are still vague. To shed light on the strengths and weaknesses of the introduced data prefetchers and to enable the readers to better understand these prefetchers, in this chapter, we quantitatively compare and contrast them. © 2022 Elsevier Inc  

L1 instruction (L1-I) cache misses are a source of performance bottleneck. While many instruction prefetchers have been proposed over the years, most of them leave a considerable potential uncovered. In 2011, Proactive Instruction Fetch (PIF) showed that a hardware prefetcher could effectively eliminate all instruction-cache misses. However, its enormous storage cost makes it an impractical solution. Consequently, reducing the storage cost was the main research focus in instruction prefetching in the past decade. Several instruction prefetchers, including RDIP and Shotgun, were proposed to offer PIF-level performance with significantly lower storage overhead. However, our findings show that...