1. CMPE 421 Parallel Computer Architecture MEMORY SYSTEM

2. 2 Memory System Outline Memory Systems Basics –Memory Technologies DRAM vs SRAM –WHY Hierarchical? –Principle of locality CACHES –Mapping, –blocks, –write policy, –Replacement policy Virtual Memory –Paging –Improving address translation segmentation Unifying principles of memory hierarchy

3. Memory System PRIMARY TECHNOLOGIES used in building memory hierarchy Random Access: –“Random” is good: access time is the same for all locations  DRAM: Dynamic Random Access Memory High density, low power, cheap, but slow Dynamic: needs to be “refreshed” regularly  SRAM: Static Random Access Memory Low density, high power, expensive, fast Static: content will last “forever”(until lose power) Often used applications requiring high-speed access “Not-so-random” Access Technology: Access time varies from location to location and from time to time Examples: Disk, CDROM Sequential Access Technology: access time linear in location (e.g. Tape) The Main Memory: DRAMs Caches: SRAMs 3

4. Memory System 4 Memory hierarchy takes advantage of the principle of locality by implementing the memory of a computer Because of these differences in cost and access time, it is advantageous to build memory as a hierarchy of levels Standard cost / performance trade off o Inverse relationship between access time and cost

5. The problem on Memory The Von Neumann Bottleneck –Logic continues to get faster –Memory capacity is getting larger But memory is not getting faster as logic –How can we keep the CPU busy all of the time? –Many aspects of the computer are dependent on the memory performance 5

6. The problem on Memory Cost vs. Performance –Fast memory is expensive –Slow memory can significantly affect performance Design philosophy –Use a hybrid solution which uses aspects of both –Keep frequently used things in a small of amount of fast/expensive memory, called a cache –Place everything else in slower/inexpensive memory –Make the common case fast 6

7. Slide #7 The Solution Memory can be arranged as hierarchies The goal is to provide the illusion of lots of fast memory Control Datapath Memory Processor Memory Fastest Slowest Smallest Biggest Highest Lowest Speed: Size: Cost: As we away from the processor the levels takes progressively longer to access At any given time, data is copied between only 2 adjacent level Upper level (cache) the one closer to processor Lower level : one further away from the processor

8. Cache Levels Often refer to “upper” to refer to memory closer to the processor Use the term “lower” to refer to memory farther from the processor Control Secondary Storage (DISK) Processor Main Memory (DRAM) Second Level Cache (SDRAM) Registers Datapath On-Chip Cache Faster Slower Inexpensive Expensive Remote Secondary Storage distributed file systems

9. As we move from the higher to low level –At the Highest level (L0) are small number of fast CPU registers that can access in a single clock cycle –Next are one or more small to moderate-sized SRAM-based cache memories that can be accessed in a few CPU clock cycles Next are slow but enormous local disk 9 Cache Levels

10. Memory Hierarchy Suppose that you are writing term project about computer hardware history. You will need many book under your hand. To find exact information that you need, you should go back to shelves and look for an additional books once you have a good selection of books in front of you, you may spend most of your time just using the books on the desk without going back to the shelves Problem: How to access all books within small duration or accessing large memory that we can access as fast as a very small memory –So we should find a way to make it possible for us to fit al the library books on our desk and still find what you want quickly 10

11. Solution: Principle of Locality –Program access a relatively small portion of their address space at any instant of time, just as you accessed a very small portion of the library’s collection Temporal locality (Locality in time) –If an item is referenced “now” it will be tend to be referenced again in the near feature Spatial locality (locality in space) –If an item is referenced “now” other items with addresses close to it will be tended to be referenced soon 11 Memory Hierarchy

12. Ex1: If you recently brought a book to your desk to look at, you will probably need to look at it again soon (temporal locality) Ex2: When you brought out the book on early Electrical Computers, You also notice that there was another book shelved next to it about early mechanical computers –So books on same topic must be shelved together in the library to increase spatial locality 12

13. Principle of Locality 13 Why does code have locality? Look at the common structures and behaviors of program executions -Temporal locality Most programs contain loops, so instruction and data are likely to be accessed repeatedly, showing high amounts of temporal locality Keep more recently accessed data items closer to the processor -Spatial locality Since instructions are normally accessed sequentially, programs show high spatial locality Accesses to data also exhibit a natural spatial locality, e.g., elements of an array Move block consisting of multiple contiguous words in memory to upper levels of the hierarchy.

14. Programs and Locality Program tends to exhibit a great deal of locality in memory access –Array, structure/record access –Subroutines (instructions are near each other) –Local variables (counters, pointers, etc.) are often referenced memory many times Most programs contain loops, so instructions and data are likely to be accessed repeatedly, showing high amounts of temporal locality Since instructions are normally accessed sequentially, programs show high spatial locality Access to data (elements of array) exhibit a natural spatial locality Take the advantage of the principle of locality by implementing the memory of a computer as a memory hierarchy 14