Loading...

Cross-core data sharing for energy-efficient gpus

Falahati, H ; Sharif University of Technology | 2024

51 Viewed
  1. Type of Document: Article
  2. DOI: 10.1145/3653019
  3. Publisher: 2024
  4. Abstract:
  5. Graphics Processing Units (GPUs) are the accelerator of choice in a variety of application domains, because they can accelerate massively parallel workloads and can be easily programmed using general-purpose programming frameworks such as CUDA and OpenCL. Each Streaming Multiprocessor (SM) contains an L1 data cache (L1D) to exploit the locality in data accesses. L1D misses are costly for GPUs for two reasons. First, L1D misses consume a lot of energy as they need to access the L2 cache (L2) via an on-chip network and the off-chip DRAM in case of L2 misses. Second, L1D misses impose performance overhead if the GPU does not have enough active warps to hide the long memory access latency. We observe that threads running on different SMs share 55% of the data they read from the memory. Unfortunately, as the L1Ds are in the non-coherent memory domain, each SM independently fetches data from the L2 or the off-chip memory into its L1D, even though the data may be currently available in the L1D of another SM. Our goal is to service L1D read misses via other SMs, as much as possible, to cut down costly accesses to the L2 or the off-chip DRAM. To this end, we propose a new data-sharing mechanism, called Cross-Core Data Sharing (CCDS). CCDS employs a predictor to estimate whether the required cache block exists in another SM. If the block is predicted to exist in another SM’s L1D, then CCDS fetches the data from the L1D that contain the block. Our experiments on a suite of 26 workloads show that CCDS improves average energy and performance by 1.30× and 1.20×, respectively, compared to the baseline GPU. Compared to the state-of-the-art data-sharing mechanism, CCDS improves average energy and performance by 1.37× and 1.11×, respectively. © 2024 Copyright held by the owner/author(s)
  6. Keywords:
  7. Cache memory ; Computer graphics equipment ; Data assimilation ; Data integration ; Data privacy ; Data streams ; Dynamic random access storage ; Multiprocessing systems ; Network security ; Semiconductor storage ; Static random access storage ; Text messaging ; Average energy ; Cache miss ; Data caches ; Data sharing ; Graphics processing ; Off-chip ; Performance ; Processing units ; Sharing mechanism ; Streaming multiprocessors ; Graphics processing unit
  8. Source: ACM Transactions on Architecture and Code Optimization ; Volume 21, Issue 3 , 2024 ; 15443566 (ISSN)
  9. URL: https://dl.acm.org/doi/10.1145/3653019