CS1104: Computer Organisation School of Computing National University of Singapore

CS1104-P2-8Memory Hierarchy2 PII Lecture 8: Memory Hierarchy  Memory Technologies  Memory Hierarchy Analogy  Illusion of Large, Fast, Cheap Memory  Principle of Locality  Terminology

CS1104-P2-8Memory Hierarchy3 PII Lecture 8: Memory Hierarchy  Reading:  Chapter 8 of textbook, which is Chapter 5 in “Computer Organization” by Hamacher, Vranesic and Zaky.

CS1104-P2-8Memory Hierarchy4 Memory Technologies  RAM (Random Access Memory): access time is the same for all locations (in nanoseconds).  DRAM: Dynamic RAM  High density, low power, cheap, slow (access time: ns).  Dynamic: needs to be “refreshed” regularly.  SRAM: Static RAM  Low density, high power, expensive, fast (access time: 5-25 ns).  Static: non-volatile – content lasts “forever” (until power is removed).

CS1104-P2-8Memory Hierarchy5 Memory Technologies (2)  ROM (Read-Only Memory): non-volatile, allows for read only.  PROM: Programmable ROM  EPROM: Erasable PROM  EEPROM: Electrically EPROM  Flash Memory  Similar to EEPROM  Difference: read a single cell, but write a block

CS1104-P2-8Memory Hierarchy6 Recap: Memory Organisation Another example: Organization of a 2M  32 memory module using 512K  8 static memory chips. Chip select 512K x 8 memory chip 19-bit address 8-bit data input/output

CS1104-P2-8Memory Hierarchy7 Technology Trends CapacitySpeed (latency) Processor:--4x in 3 yrs DRAM:4x in 3 yrs2x in 10 yrs Disk:4x in 3 yrs 2x in 10 yrs DRAM YearSizeCycle Time Kb250 ns Kb220 ns Mb190 ns Mb165 ns Mb145 ns Mb120 ns 1000:1!2:1!

CS1104-P2-8Memory Hierarchy8 Technology Trends (2) DRAM 9%/yr. (2X/10 yrs) µProc 60%/yr. (2X/1.5yr) Processor-Memory Performance Gap: (grows 50% / year) DRAM CPU Time

CS1104-P2-8Memory Hierarchy9 Analogy: Term Paper in Library  Working on a paper at a desk in library.  Option 1: Every time a book is needed…  Leave desk to go to shelves (or stacks)  Find the book  Bring one book back to desk  Read section interested in  When done, leave desk and go to shelves carrying book  Put book back on shelf  Return to desk to work

CS1104-P2-8Memory Hierarchy10 Analogy: Term Paper in Library (2)  Option 2: Every time a book is needed…  Leave some books on desk after fetching them  Only go to shelves when a book not on desk is needed  At the shelves, fetch related books in case you need them; sometimes you will need to return books not used recently to make space for new books on desk (replacement algorithm)  Return to desk to work  Illusion: whole library on your desk

CS1104-P2-8Memory Hierarchy11 Illusion of Large, Fast, Cheap Memory  Registers in processor: Very fast, in order of nanoseconds to picoseconds.  Disks: Very slow, in order of milliseconds.  Large memories (disks) are slow; fast memories (registers) are small.  How do we create a memory that is large, cheap and fast (most of the time)?  Memory hierarchy.

CS1104-P2-8Memory Hierarchy12 Memory Hierarchy Pyramid Central Processor Unit (CPU) “Upper” “Lower” Levels in memory hierarchy Increasing distance from CPU, decreasing cost/bit Size of memory at each level Level 1 Level 2 Level 3... Level n (Data cannot be in level i unless it is also in i+1.)

CS1104-P2-8Memory Hierarchy13 Principle of Locality  Principle of locality (or locality of reference):  Program accesses a relatively small portion of the address space at any instant of time.  Temporal locality and spatial locality. Address Space 02 n - 1 Probability of reference

CS1104-P2-8Memory Hierarchy14 Principle of Locality (2)  Temporal locality (locality in time):   Keep most recently accessed data items closer to the processor.  Library analogy: Recently read books are kept on desk.  Spatial locality (locality in space):   Move blocks consisting of contiguous words to ‘upper’ levels.  Block is a unit of transfer.  Library analogy: Bring back nearby books on shelves when fetching a book, hoping that you might need them soon.

CS1104-P2-8Memory Hierarchy15 Principle of Locality (3)  What programming constructs lead to principle of locality? SpatialTemporal Dataarraysloop counters Codeno branch/jumploop

CS1104-P2-8Memory Hierarchy16 Memory Hierarchy: Idea  Temporal locality: keep recently accessed data items closer to processor.  Spatial locality: move contiguous words in memory to upper levels of hierarchy.  Use smaller and faster memory technologies closer to the processor  If hit rate is high enough, hierarchy has access time close to the highest (fastest) level and size equal to the lowest (largest) level.

CS1104-P2-8Memory Hierarchy17 Memory Hierarchy: Terminology  Hit: data appears in some block in the upper level.  Hit rate: the fraction of memory access found in the upper level.  Analogy: fraction of time you find the book on desk.  Miss: data is not at upper level and needs to be retrieved from a block in the lower level.  Miss rate: 1 – Hit rate  Analogy: fraction of time you need to go to shelves for the book.

CS1104-P2-8Memory Hierarchy18 Memory Hierarchy: Terminology (2)  (Average) Hit time: time to access the upper level which consists of  Time to determine hit/miss + memory access time.  Analogy: time to find and pick up book from desk.  (Average) Miss penalty: time to replace a block in the upper level + time to deliver the block to the processor.  Analogy: time to go to shelves, find needed book, and return to your desk.  Hit time << Miss penalty.

CS1104-P2-8Memory Hierarchy19 Current Memory Hierarchy Increasing size Processor Primary cache Secondary cache Main memory Magnetic disk secondary memory Register L1 L2 Increasing speed Increasing cost per bit

CS1104-P2-8Memory Hierarchy20 Current Memory Hierarchy (2) Control Data path Processor Regs Second- ary Memory L2 Cache L1 $ Main Memory Speed(ns):0.5ns2ns6ns100ns10,000,000ns Size (MB): ,000 Cost ($/MB):--$100$30$1 $0.05 Technology:RegsSRAMSRAMDRAMDisk

CS1104-P2-8Memory Hierarchy21 Management of the Hierarchy  Registers  Memory  By compiler (or assembly programmer).  Cache  Main memory  By the hardware.  Main memory  Disks  By the hardware and operating system.  By the programmer (through files).

CS1104-P2-8Memory Hierarchy22 Virtual Memory  Main memory smaller than address space  Example: 32-bit address allows an address space of 4G bytes, but main memory may only be a few hundred megabytes.  Parts of program not in main memory are stored on secondary storage devices, such as disks.  Operating system moves programs and data automatically between the physical main memory and secondary storage (virtual memory).

CS1104-P2-8Memory Hierarchy23 Things To Remember  Principle of locality and hierarchy of memories of different speed and cost; exploit locality to improve cost- performance.  Terms: Hit, miss, hit rate, miss rate, hit time, miss penalty, block, upper-level memory, lower-level memory.

24 End of file