1. CS1104: Computer Organisation http://www.comp.nus.edu.sg/~cs1104 http://www.comp.nus.edu.sg/~cs1104 School of Computing National University of Singapore

2. CS1104-P2-8Memory Hierarchy2 PII Lecture 8: Memory Hierarchy  Memory Technologies  Memory Hierarchy Analogy  Illusion of Large, Fast, Cheap Memory  Principle of Locality  Terminology

3. CS1104-P2-8Memory Hierarchy3 PII Lecture 8: Memory Hierarchy  Reading:  Chapter 8 of textbook, which is Chapter 5 in “Computer Organization” by Hamacher, Vranesic and Zaky.

4. CS1104-P2-8Memory Hierarchy4 Memory Technologies  RAM (Random Access Memory): access time is the same for all locations (in nanoseconds).  DRAM: Dynamic RAM  High density, low power, cheap, slow (access time: 60-120 ns).  Dynamic: needs to be “refreshed” regularly.  SRAM: Static RAM  Low density, high power, expensive, fast (access time: 5-25 ns).  Static: non-volatile – content lasts “forever” (until power is removed).

5. CS1104-P2-8Memory Hierarchy5 Memory Technologies (2)  ROM (Read-Only Memory): non-volatile, allows for read only.  PROM: Programmable ROM  EPROM: Erasable PROM  EEPROM: Electrically EPROM  Flash Memory  Similar to EEPROM  Difference: read a single cell, but write a block

6. CS1104-P2-8Memory Hierarchy6 Recap: Memory Organisation Another example: Organization of a 2M  32 memory module using 512K  8 static memory chips. Chip select 512K x 8 memory chip 19-bit address 8-bit data input/output

7. CS1104-P2-8Memory Hierarchy7 Technology Trends CapacitySpeed (latency) Processor:--4x in 3 yrs DRAM:4x in 3 yrs2x in 10 yrs Disk:4x in 3 yrs 2x in 10 yrs DRAM YearSizeCycle Time 198064 Kb250 ns 1983256 Kb220 ns 19861 Mb190 ns 19894 Mb165 ns 199316 Mb145 ns 199764 Mb120 ns 1000:1!2:1!

8. CS1104-P2-8Memory Hierarchy8 Technology Trends (2) DRAM 9%/yr. (2X/10 yrs) µProc 60%/yr. (2X/1.5yr) Processor-Memory Performance Gap: (grows 50% / year) 1 10 100 1000 DRAM CPU 198019811983198419851986198719881989199019911992199319941995199619971998199920001982 Time

9. CS1104-P2-8Memory Hierarchy9 Analogy: Term Paper in Library  Working on a paper at a desk in library.  Option 1: Every time a book is needed…  Leave desk to go to shelves (or stacks)  Find the book  Bring one book back to desk  Read section interested in  When done, leave desk and go to shelves carrying book  Put book back on shelf  Return to desk to work

10. CS1104-P2-8Memory Hierarchy10 Analogy: Term Paper in Library (2)  Option 2: Every time a book is needed…  Leave some books on desk after fetching them  Only go to shelves when a book not on desk is needed  At the shelves, fetch related books in case you need them; sometimes you will need to return books not used recently to make space for new books on desk (replacement algorithm)  Return to desk to work  Illusion: whole library on your desk

11. CS1104-P2-8Memory Hierarchy11 Illusion of Large, Fast, Cheap Memory  Registers in processor: Very fast, in order of nanoseconds to picoseconds.  Disks: Very slow, in order of milliseconds.  Large memories (disks) are slow; fast memories (registers) are small.  How do we create a memory that is large, cheap and fast (most of the time)?  Memory hierarchy.

12. CS1104-P2-8Memory Hierarchy12 Memory Hierarchy Pyramid Central Processor Unit (CPU) “Upper” “Lower” Levels in memory hierarchy Increasing distance from CPU, decreasing cost/bit Size of memory at each level Level 1 Level 2 Level 3... Level n (Data cannot be in level i unless it is also in i+1.)

13. CS1104-P2-8Memory Hierarchy13 Principle of Locality  Principle of locality (or locality of reference):  Program accesses a relatively small portion of the address space at any instant of time.  Temporal locality and spatial locality. Address Space 02 n - 1 Probability of reference

14. CS1104-P2-8Memory Hierarchy14 Principle of Locality (2)  Temporal locality (locality in time):   Keep most recently accessed data items closer to the processor.  Library analogy: Recently read books are kept on desk.  Spatial locality (locality in space):   Move blocks consisting of contiguous words to ‘upper’ levels.  Block is a unit of transfer.  Library analogy: Bring back nearby books on shelves when fetching a book, hoping that you might need them soon.

15. CS1104-P2-8Memory Hierarchy15 Principle of Locality (3)  What programming constructs lead to principle of locality? SpatialTemporal Dataarraysloop counters Codeno branch/jumploop

16. CS1104-P2-8Memory Hierarchy16 Memory Hierarchy: Idea  Temporal locality: keep recently accessed data items closer to processor.  Spatial locality: move contiguous words in memory to upper levels of hierarchy.  Use smaller and faster memory technologies closer to the processor  If hit rate is high enough, hierarchy has access time close to the highest (fastest) level and size equal to the lowest (largest) level.

17. CS1104-P2-8Memory Hierarchy17 Memory Hierarchy: Terminology  Hit: data appears in some block in the upper level.  Hit rate: the fraction of memory access found in the upper level.  Analogy: fraction of time you find the book on desk.  Miss: data is not at upper level and needs to be retrieved from a block in the lower level.  Miss rate: 1 – Hit rate  Analogy: fraction of time you need to go to shelves for the book.

18. CS1104-P2-8Memory Hierarchy18 Memory Hierarchy: Terminology (2)  (Average) Hit time: time to access the upper level which consists of  Time to determine hit/miss + memory access time.  Analogy: time to find and pick up book from desk.  (Average) Miss penalty: time to replace a block in the upper level + time to deliver the block to the processor.  Analogy: time to go to shelves, find needed book, and return to your desk.  Hit time << Miss penalty.

19. CS1104-P2-8Memory Hierarchy19 Current Memory Hierarchy Increasing size Processor Primary cache Secondary cache Main memory Magnetic disk secondary memory Register L1 L2 Increasing speed Increasing cost per bit

20. CS1104-P2-8Memory Hierarchy20 Current Memory Hierarchy (2) Control Data path Processor Regs Second- ary Memory L2 Cache L1 $ Main Memory Speed(ns):0.5ns2ns6ns100ns10,000,000ns Size (MB):0.00050.051-4100-1000100,000 Cost ($/MB):--$100$30$1 $0.05 Technology:RegsSRAMSRAMDRAMDisk

21. CS1104-P2-8Memory Hierarchy21 Management of the Hierarchy  Registers  Memory  By compiler (or assembly programmer).  Cache  Main memory  By the hardware.  Main memory  Disks  By the hardware and operating system.  By the programmer (through files).

22. CS1104-P2-8Memory Hierarchy22 Virtual Memory  Main memory smaller than address space  Example: 32-bit address allows an address space of 4G bytes, but main memory may only be a few hundred megabytes.  Parts of program not in main memory are stored on secondary storage devices, such as disks.  Operating system moves programs and data automatically between the physical main memory and secondary storage (virtual memory).

23. CS1104-P2-8Memory Hierarchy23 Things To Remember  Principle of locality and hierarchy of memories of different speed and cost; exploit locality to improve cost- performance.  Terms: Hit, miss, hit rate, miss rate, hit time, miss penalty, block, upper-level memory, lower-level memory.

24. 24 End of file