Caching Chapter 7.

Name: Caching Chapter 7.
Uploaded: 2017-10-08T23:59:29+00:00
Duration: PTM37S54
Channel: Michael Montgomery
Description: Caching Chapter 7.

Caching Chapter 7

Memory Hierarchy Size Smallest Largest Cost/bit Highest Lowest Tech
SRAM (logic) DRAM (capacitors) Speed Fastest Slowest CPU L1 L2 Cache DRAM

Two design decisions What shall we put in the cache?
How shall we organize cache to find things quickly hold the most important data freezer or backpack….

What to put in cache? Try to apply a similar problem’s solution
Can we predict what data we will use?

What to put in cache? Can we predict what data we will use?
Instead of predicting branch direction, predict next memory address request

Instead of predicting branch direction, predict next memory address request Like branch prediction, use previous behavior

Instead of predicting branch direction, predict next memory address request Like branch prediction, use previous behavior Keep a prediction for every load? Fetch stage for load is *TOO LATE* Keep a prediction per-memory address?

Instead of predicting branch direction, predict next memory address request Like branch prediction, use previous behavior Keep a prediction for every load? Fetch stage for load is *TOO LATE* Keep a prediction per-memory address? Given address, guess next likely address

Instead of predicting branch direction, predict next memory address request Like branch prediction, use previous behavior Keep a prediction for every load? Fetch stage for load is *TOO LATE* Keep a prediction per-memory address? Given address, guess next likely address Too many choices – table too large or fits too few

Program Characteristics Find out more about programs
Temporal Locality Spatial Locality

Program Characteristics
Temporal Locality If you use one item, you are likely to use it again soon Spatial Locality

Program Characteristics
Temporal Locality If you use one item, you are likely to use it again soon Spatial Locality If you use one item, you are likely to use its neighbors soon

Locality Programs tend to exhibit spatial & temporal locality. Just a fact of life. How can we use this knowledge of program behavior to design a cache?

What does that mean?!? 1. Design cache that takes advantage of spatial & temporal locality

What does that mean?!? 1. Design cache that takes advantage of spatial & temporal locality 2. When you program, place data together that is used together to increase spatial & temporal locality

What does that mean?!? 1. Design cache that takes advantage of spatial & temporal locality 2. When you program, place data together that is used together to increase locality Java - difficult to do C - more control over data placement Note: Caches exploit locality. Programs have varying degrees of locality. Caches do not have locality!

Cache Design Temporal Locality Spatial Locality

Cache Design Temporal Locality Spatial Locality
When we obtain the data, store it in the cache. Spatial Locality

Cache Design Temporal Locality Spatial Locality
When we obtain the data, store it in the cache. Spatial Locality Transfer large block of contiguous data to get item’s neighbors. Block (Line): Amount of data transferred for a single miss (data plus neighbors)

Where do we put data? Searching whole cache takes time & power
Direct-mapped Limit each piece of data to one possible position Search is quick and simple

What is our “key” for lookup?
Tools are sorted by tool-type Books are sorted by subject (Dewey-Decimal) Old LISP machine sorted by data type Modern machines have no information – can only sort by address

Each box corresponds to one word (4 bytes)
Direct-Mapped Each box corresponds to one word (4 bytes) 000000 000100 Index 010000 00 010100 01 10 11 100000 100100 Cache 110000 110100 Memory

Direct-Mapped 000000 000100 One block (line) Index 010000 00 010100 01
11 100000 100100 Cache 110000 110100 Memory

Show what addresses go where
Draw on the board!!! Show what addresses go where Direct-Mapped 000000 One block (line) 000100 Index 010000 00 010100 01 10 11 100000 100100 Cache 110000 110100 Memory

Direct-Mapped cache Block (Line) size = 2 words or 8 bytes
Byte Address 0b Index Data 00 01 10 11 Where do we look in the cache? How do we know if it is there?

Byte Address 0b Index Data 00 01 Where is it within the block? 10 Block Address 11 Where do we look in the cache? BlockAddress mod #sets BlockAddress & (#sets-1) How do we know if it is there?

Byte Address 0b Valid Tag Data 00 1 1001 M[ ] M[ ] 01 Where is it within the block? 10 Tag Index 11 Where do we look in the cache? BlockAddress mod #slots BlockAddress & (#slots-1) How do we know if it is there? We need a tag & valid bit

Splitting the Address Direct-Mapped Cache Valid Tag Data 0b1010001 00
01 10 Tag 11 Index Block Offset Byte Offset

Definitions Byte Offset: Which _____ within _____?
Block Offset: Which _____ within ______? Set: Group of ______ checked each access Index: Which ______ within cache? Tag: Is this the right one?

Definitions Byte Offset: Which byte within word
Block Offset: Which _____ within ______? Set: Group of ______ checked each access Index: Which ______ within cache? Tag: Is this the right one?

Block Offset: Which word within block Set: Group of ______ checked each access Index: Which ______ within cache? Tag: Is this the right one?

Block Offset: Which word within block Set: Group of blocks checked each access Index: Which ______ within cache? Tag: Is this the right one?

Block Offset: Which word within block Set: Group of blocks checked each access Index: Which set within cache? Tag: Is this the right one? (All of the upper bits)

Definitions Block (Line) Hit Miss Hit time / Access time Miss Penalty

Definitions Block - unit of data transfer – bytes/words Hit Miss
Hit time / Access time Miss Penalty

Definitions Block - unit of data transfer – bytes/words
Hit - data found in this cache Miss Hit time / Access time Miss Penalty

Hit - data found in this cache Miss - data not found in this cache Send request to lower level Hit time / Access time Miss Penalty

Hit - data found in this cache Miss - data not found in this cache Send request to lower level Hit time / Access time Time to access this cache – look for item, return data Miss Penalty

Hit - data found in this cache Miss - data not found in this cache Send request to lower level Hit time / Access time Time to access this cache Miss Penalty Time to receive block from lower level Not always constant

Example 1 – Direct-Mapped Block size=2 words
Direct-Mapped Cache Valid Tag Data 00 0x 01 10 Tag 11 Index Block Offset Byte Offset

Direct-Mapped Cache Reference Stream: Hit/Miss 0b 0b 0b 0b 0b Valid Tag Data 00 01 10 11 Tag Byte Offset Miss Rate: Index Block Offset

Direct-Mapped Cache Reference Stream: Hit/Miss 0b M 0b 0b 0b 0b Valid Tag Data 00 01 1 10 M[76-79] M[72-75] 10 11 Tag Byte Offset Miss Rate: Index Block Offset

Direct-Mapped Cache Reference Stream: Hit/Miss 0b M 0b 0b 0b 0b Valid Tag Data 00 01 1 10 M[76-79] M[72-75] 10 1 00 M[20-23] M[16-19] 11 Tag Byte Offset Miss Rate: Index Block Offset

Direct-Mapped Cache Reference Stream: Hit/Miss 0b M 0b M 0b 0b 0b 0b Valid Tag Data 00 01 1 10 M[76-79] M[72-75] 10 1 00 M[20-23] M[16-19] 11 Tag Byte Offset Miss Rate: Index Block Offset

Direct-Mapped Cache Reference Stream: Hit/Miss 0b M 0b M 0b M 0b 0b 0b Valid Tag Data 00 01 1 10 M[76-79] M[72-75] 10 1 00 M[20-23] M[16-19] 11 1 01 M[60-63] M[56-59] Tag Byte Offset Miss Rate: Index Block Offset

Direct-Mapped Cache Reference Stream: Hit/Miss 0b M 0b M 0b M 0b H 0b 0b Valid Tag Data 00 01 1 10 M[76-79] M[72-75] 10 1 00 M[20-23] M[16-19] 11 1 01 M[60-63] M[56-59] Tag Byte Offset Miss Rate: Index Block Offset

Direct-Mapped Cache Reference Stream: Hit/Miss 0b M 0b M 0b M 0b H 0b H 0b Valid Tag Data 00 01 1 10 M[76-79] M[72-75] 10 1 00 M[20-23] M[16-19] 11 1 01 M[60-63] M[56-59] Tag Byte Offset Miss Rate: Index Block Offset

Direct-Mapped Cache Reference Stream: Hit/Miss 0b M 0b M 0b M 0b H 0b H 0b M Valid Tag Data 00 01 1 10 M[76-79] M[72-75] 10 1 01 M[20-23] M[16-19] 11 1 01 M[60-63] M[56-59] Tag Byte Offset Miss Rate: Index Block Offset

Direct-Mapped Cache Reference Stream: Hit/Miss 0b M 0b M 0b M 0b H 0b H 0b M Valid Tag Data 00 1 01 M[36-39] M[32-35] 01 1 10 M[76-79] M[72-75] 10 1 01 M[20-23] M[16-19] 11 1 01 M[60-63] M[56-59] Tag Byte Offset Miss Rate: Index Block Offset

Direct-Mapped Cache Reference Stream: Hit/Miss 0b M 0b M 0b M 0b H 0b H 0b M Valid Tag Data 00 1 01 M[36-39] M[32-35] 01 1 10 M[76-79] M[72-75] 10 1 01 M[20-23] M[16-19] 11 1 01 M[60-63] M[56-59] Tag Byte Offset Miss Rate: 4 / 6 = 67% Hit Rate: 2 / 6 = 33% Index Block Offset

Implementation Byte Address 0b100100100 Byte Offset Tag Index
Block offset Valid Tag Data 00 01 10 11 = MUX Hit? Data

Example 2 You are implementing a 64-Kbyte cache, 32-bit address
The block size (line size) is 16 bytes. Each word is 4 bytes How many bits is the block offset? How many bits is the index? How many bits is the tag?

Example 2 You are implementing a 64-Kbyte cache
The block size (line size) is 16 bytes. Each word is 4 bytes How many bits is the block offset? 16 / 4 = 4 words -> 2 bits How many bits is the index? How many bits is the tag?

The block size (line size) is 16 bytes. Each word is 4 bytes, address 32 bits How many bits is the block offset? 16 / 4 = 4 words -> 2 bits How many bits is the index? 64*1024 / 16 = > 12 bits How many bits is the tag?

The block size (line size) is 16 bytes. Each word is 4 bytes, address 32 bits How many bits is the block offset? 16 / 4 = 4 words -> 2 bits How many bits is the index? 64*1024 / 16 = > 12 bits How many bits is the tag? 32 - ( ) = 16 bits

How caches work Classic abstraction
Each level of hierarchy has no knowledge of the configuration of lower level L2 cache’s perspective L1 cache’s perspective Me Me L1 L2 Cache Memory Memory L2 Cache DRAM DRAM

Memory Operation at any level
Address 1. 1. Cache receives request Me Cache Memory

Memory operation at any level
Address 1. 1. Cache receives request 2. Look for item in cache 2. Me Cache Memory

Address Data 1. 3. 1. Cache receives request 2. Look for item in cache Hit - return data 2. Me Cache Memory

Address 1. 1. Cache receives request 2. Look for item in cache Hit - return data Miss - request memory 2. Me Cache 3. Memory

Address 1. 1. Cache receives request 2. Look for item in cache Hit - return data Miss - request memory receive data update cache 2. Me Cache 3. 4. Memory

Address Data 1. 1. Cache receives request 2. Look for item in cache Hit - return data Miss – 3. request memory 4. receive data 5. update cache 5. return data 5. 2. Me Cache 3. 4. Memory

Timing Address 1. Cache receives request Me Cache Memory

Timing Address 1. Cache receives request 2. Look for item in cache
Access Time Me Cache Memory

Address Data 1. Cache receives request 2. Look for item in cache Hit - return data Access Time Me Cache Memory

Address 1. Cache receives request 2. Look for item in cache Hit - return data Miss - request memory Access Time Me Cache Memory

Address 1. Cache receives request 2. Look for item in cache Hit - return data Miss - request memory receive block update cache Access Time Me Cache Memory Miss Penalty

Address Data 1. Cache receives request 2. Look for item in cache Hit - return data Miss - request memory receive block update cache return data Access Time Me Cache Memory Miss Penalty

Performance Hit: latency = Miss: latency = Goal: minimize misses!!!

Performance Hit: latency = access time Miss: latency =
Goal: minimize misses!!!

Performance Hit: latency = access time
Miss: latency = access time + miss penalty Goal: minimize misses!!!

Caching Chapter 7.

Similar presentations

Presentation on theme: "Caching Chapter 7."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Caching Chapter 7.

Similar presentations

Presentation on theme: "Caching Chapter 7."— Presentation transcript:

Similar presentations

About project

Feedback