2. ICC Module 3 Lesson 2 – Memory Hierarchies 1 / 25 © 2015 Ph. Janson Information, Computing & Communication Memory Hierarchies – Clip 8 – Example School of Computer Science & Communications B. Falsafi (charts), Ph. Janson (commentary)

3. ICC Module 3 Lesson 2 – Memory Hierarchies 2 / 25 © 2015 Ph. Janson Outline ►Clip 1 – TechnologiesClip 1 ►Clip 2 – ConceptClip 2 ►Clip 3 – PrincipleClip 3 ►Clip 4 – ImplementationClip 4 ►Clip 5 – Reading memoryClip 5 ►Clip 6 – Writing memoryClip 6 ►Clip 7 – Cache management – the Least Recently Used algorithmClip 7 ►Clip 8 – A simulated exampleClip 8 ►Clip 9 – LocalityClip 9 First clipPrevious clipNext clip

4. ICC Module 3 Lesson 2 – Memory Hierarchies 3 / 25 © 2015 Ph. Janson Example from the preceding lesson: sum of integers up to n Sum of n first integers input : n output : m s  0 while n > 0 s  s + n n  n – 1 m  s

5. ICC Module 3 Lesson 2 – Memory Hierarchies 4 / 25 © 2015 Ph. Janson Focus on the loop Sum of n first integers input : n output : m s  0 while n > 0 s  s + n n  n – 1 m  s

6. ICC Module 3 Lesson 2 – Memory Hierarchies 5 / 25 © 2015 Ph. Janson Imagine a hypothetical computer  Cache with 2 blocks  Memory with 4 blocks  Blocks of 4 words each  (Only one register in the processor) Cache ? ? ? ? Main memory 0 4 8 12 Processor

7. ICC Module 3 Lesson 2 – Memory Hierarchies 6 / 25 © 2015 Ph. Janson Assume the following memory layout ►m @3 ►n @13 ►s @14 ►s and m initially 0 ►n initially 2 Cache ? ? ? ? Main memory 0 4 8 12 m=0 n=2 s=0 Processor

8. ICC Module 3 Lesson 2 – Memory Hierarchies 7 / 25 © 2015 Ph. Janson read @ 13 Cache ? ? ? ? Main memory m=0 n=2 s=0 read @13 0 4 8 12 Execution Processor While n > 0 s  s + n n  n – 1

9. ICC Module 3 Lesson 2 – Memory Hierarchies 8 / 25 © 2015 Ph. Janson read @13 @13 not in cache ((de)fault) Cache ? ? ? ? Main memory m=0 n=2 s=0 0 4 8 12 Execution Processor While n > 0 s  s + n n  n – 1

10. ICC Module 3 Lesson 2 – Memory Hierarchies 9 / 25 © 2015 Ph. Janson read @13 @13 not in cache ((de)fault) load block @12 Cache ? ? ? ? Main memory m=0 n=2 s=0 load block @12 0 4 8 12 Execution Processor While n > 0 s  s + n n  n – 1

11. ICC Module 3 Lesson 2 – Memory Hierarchies 10 / 25 © 2015 Ph. Janson read @13 @13 not in cache ((de)fault) load block @12 Cache ? ? ? ? Processor n=2 s=0 12 Execution Main memory m=0 n=2 s=0 0 4 8 12 While n > 0 s  s + n n  n – 1

12. ICC Module 3 Lesson 2 – Memory Hierarchies 11 / 25 © 2015 Ph. Janson read @13 @13 not in cache ((de)fault) load block @12 place block @12 Main memory m=0 n=2 s=0 Processor 0 4 8 12 Execution Cache ? ? 12 n=2 s=0 While n > 0 s  s + n n  n – 1

13. ICC Module 3 Lesson 2 – Memory Hierarchies 12 / 25 © 2015 Ph. Janson read @13 @13 not in cache ((de)fault) load block @12 place block @12 return 2 Processor Execution @13 = 2 Cache ? ? 12 n=2 s=0 Main memory m=0 n=2 s=0 0 4 8 12 While n > 0 s  s + n n  n – 1

14. ICC Module 3 Lesson 2 – Memory Hierarchies 13 / 25 © 2015 Ph. Janson Cache ? ? Processor 12 n=2 s=0 read @14 read @13 @13 not in cache ((de)fault) load block @12 place block @12 return 2 read @14 Main memory m=0 n=2 s=0 0 4 8 12 While n > 0 s  s + n n  n – 1 Execution

15. ICC Module 3 Lesson 2 – Memory Hierarchies 14 / 25 © 2015 Ph. Janson Cache ? ? Processor 12 n=2 s=0 @14 = 0 (in cache!) read @13 @13 not in cache ((de)fault) load block @12 place block @12 return 2 read @14 (in cache!) return 0 Main memory m=0 n=2 s=0 0 4 8 12 While n > 0 s  s + n n  n – 1 Execution

16. ICC Module 3 Lesson 2 – Memory Hierarchies 15 / 25 © 2015 Ph. Janson Cache ? ? Processor 12 n=2 s=0 read @13 Execution read @13 @13 not in cache ((de)fault) load block @12 place block @12 return 2 read @14 (in cache) return 0 read @13 Main memory m=0 n=2 s=0 0 4 8 12 While n > 0 s  s + n n  n – 1

17. ICC Module 3 Lesson 2 – Memory Hierarchies 16 / 25 © 2015 Ph. Janson Cache ? ? Processor 12 n=2 s=0 @13 = 2 (in cache !) Execution read @13 @13 not in cache ((de)fault) load block @12 place block @12 return 2 read @14 (in cache) return 0 read @13 (in cache) return 2 Main memory m=0 n=2 s=0 0 4 8 12 While n > 0 s  s + n n  n – 1

18. ICC Module 3 Lesson 2 – Memory Hierarchies 17 / 25 © 2015 Ph. Janson Cache ? ? Processor 12 n=2 s=0 read @13 @13 not in cache ((de)fault) load block @12 place block @12 return 2 read @14 (in cache) return 0 read @13 (in cache) return 2 add s, n (0 + 2) Main memory m=0 n=2 s=0 0 4 8 12 While n > 0 s  s + n n  n – 1 Execution

19. ICC Module 3 Lesson 2 – Memory Hierarchies 18 / 25 © 2015 Ph. Janson Cache ? ? Processor 12 n=2 s=2 write 2 @14 Execution read @13 @13 not in cache ((de)fault) load block @12 place block @12 return 2 read @14 (in cache) return 0 read @13 (in cache) return 2 add s, n (0 + 2) write 2 @14 (in cache) Main memory m=0 n=2 s=0 0 4 8 12 While n > 0 s  s + n n  n – 1

20. ICC Module 3 Lesson 2 – Memory Hierarchies 19 / 25 © 2015 Ph. Janson Cache ? ? Processor 12 n=2 s=2 read @13 Execution read @13 @13 not in cache ((de)fault) load block @12 place block @12 return 2 read @14 (in cache) return 0 read @13 (in cache) return 2 add s, n (0 + 2) write 2 @14 (in cache) read @13 (in cache) Main memory m=0 n=2 s=0 0 4 8 12 While n > 0 s  s + n n  n – 1

21. ICC Module 3 Lesson 2 – Memory Hierarchies 20 / 25 © 2015 Ph. Janson Cache ? ? Processor 12 n=2 s=2 @13 = 2 Execution read @13 @13 not in cache ((de)fault) load block @12 place block @12 return 2 read @14 (in cache) return 0 read @13 (in cache) return 2 add s, n (0 + 2) write 2 @14 (in cache) read @13 (in cache) return 2 Main memory m=0 n=2 s=0 0 4 8 12 While n > 0 s  s + n n  n – 1

22. ICC Module 3 Lesson 2 – Memory Hierarchies 21 / 25 © 2015 Ph. Janson read @13 @13 not in cache ((de)fault) load block @12 place block @12 return 2 read @14 (in cache) return 0 read @13 (in cache) return 2 add s, n (0 + 2) write 2 @14 (in cache) read @13 (in cache) return 2 add n, -1 (2 – 1) Cache ? ? Processor 12 n=2 s=2 Execution Main memory m=0 n=2 s=0 0 4 8 12 While n > 0 s  s + n n  n – 1

23. ICC Module 3 Lesson 2 – Memory Hierarchies 22 / 25 © 2015 Ph. Janson read @13 @13 not in cache ((de)fault) load block @12 place block @12 return 2 read @14 (in cache) return 0 read @13 (in cache) return 2 add s, n (0 + 2) write 2 @14 (in cache) read @13 (in cache) return 2 add n, -1 (2 – 1) write 1 @ 13 (in cache) Cache ? ? Processor 12 n=1 s=2 Execution Main memory m=0 n=2 s=0 0 4 8 12 write 1 @13 While n > 0 s  s + n n  n – 1

24. ICC Module 3 Lesson 2 – Memory Hierarchies 23 / 25 © 2015 Ph. Janson read @13 @13 not in cache ((de)fault) load block @12 place block @12 return 25 + 1 read @14 (in cache) return 0 read @13 (in cache) return 2 add s, n (0 + 2) write 2 @14 (in cache)1 read @13 (in cache) return 2 add n, -1 (2 – 1) write 1 @ 13 (in cache) Cache ? ? Processor 12 n=1 s=2 How many accesses to the cache and to the main memory ? Main memory m=0 n=2 s=0 0 4 8 12 While n > 0 s  s + n n  n – 1

25. ICC Module 3 Lesson 2 – Memory Hierarchies 24 / 25 © 2015 Ph. Janson read @13 (in cache) load block @12 place block @12 return 2 read @14 (in cache) return 0 read @13 (in cache) return 2 add s, n (0 + 2) write 2 @14 (in cache) read @13 (in cache) return 2 add n, -1 (2 – 1) write 1 @ 13 (in cache) Cache ? ? Processor 12 n=1 s=2 For all subsequent iterations of the loop Main memory m=0 n=2 s=0 0 4 8 12 While n > 0 s  s + n n  n – 1 6 cache accesses, 0 (de)fault

26. ICC Module 3 Lesson 2 – Memory Hierarchies 25 / 25 © 2015 Ph. Janson Overall For a total of 6n memory accesses ►6(n-1) + 5 in cache (at 1ns / access) ►1 cache (de)fault => memory access (at 100ns)  Overall performance => (100 + 6n) ns  Not bad at all if n is large which is typically the case in real programs Cache ? ? Processor 12 n=1 s=2 Main memory m=0 n=2 s=0 0 4 8 12