1. Lucía G. Menezo Valentín Puente José Ángel Gregorio University of Cantabria (Spain) MOSAIC :

2. University of Cantabria Edinburgh - PACT 2013 Motivation Directory Schemas In-cache Sparse MOSAIC Coherence Protocol Examples Evaluation Results Conclusions

3. University of Cantabria Edinburgh - PACT 2013 Performance improvement: more processors per chip Major challenges: off-chip bandwidth wall Introduce cache into the chip Complex on-chip cache hierarchies Coherence protocol: fundamental role to play 3

4. University of Cantabria Edinburgh - PACT 2013 What coherence protocol to use with large number of cores: Broadcast-based protocols high energy requirements Directory-based protocols more storage necessities for sharing information MOSAIC: new coherence protocol Directory without inclusiveness Token Coherence to guarantee correctness 4

5. University of Cantabria Edinburgh - PACT 2013 Motivation Directory Schemas In-cache Sparse MOSAIC Coherence Protocol Examples Evaluation Results Conclusions

6. University of Cantabria Edinburgh - PACT 2013 Each block in LLC includes tag, data and the sharers information LLC receives requests needs precise knowledge Inclusiveness is necessary: any block in the private levels needs to be allocated in LLC Advantage: coherence protocol less complex Disadvantage: all LLC blocks has storage overhead 6

7. University of Cantabria Edinburgh - PACT 2013 @ datasharers @data @ @ @ P Processors and private caches LLC + in-cache directory PPP Interconnection network 7 Overhead!!! @data P @ P @ P @ P @ P @ P @ P @ P

8. University of Cantabria Edinburgh - PACT 2013 @ datasharers @ datasharers LLC + in-cache directory Interconnection network 8 Overhead!!! @data P @ P @ P @ P @ P @ P @ P @ P Overhead!!! Processors and private caches

9. University of Cantabria Edinburgh - PACT 2013 Directory entries separated from data Allocated under demand Overhead proportional to the aggregate private levels size (not LLC) Capacity and associativity has to be sufficient to keep private-level cache tags 9

10. University of Cantabria Edinburgh - PACT 2013 @ datasharers @ data Interconnection network 10 @data P @ P @ P @ P @ P @ P @ P @ P @@ sharers LLC Sparse dir Processors and private caches

11. University of Cantabria Edinburgh - PACT 2013 Duplicate-tag directory: holding all the tags of private levels Example: 16 cores with 4-way 32KB L1 64-way Associativity = # cores * private caches associativity # sets = # private caches sets tag 11

12. University of Cantabria Edinburgh - PACT 2013 tag 12 Decrease Associativity: now << # cores * private caches associativity tag sharers tag One tag may be in various private caches More than 1 tag per entry conflicts Inclusiveness needed invalidate private data (recalls messages) tag Increase number of sets

13. University of Cantabria Edinburgh - PACT 2013 Motivation Directory Schemas In-cache Sparse MOSAIC Coherence Protocol Examples Evaluation Results Conclusions 13

14. University of Cantabria Edinburgh - PACT 2013 In-cache or sparse it doesnt matter No inclusiveness No invalidations of data in private caches Reconstruction of sharing information under demand Uses token counting to avoid extra traffic and guarantee correctness Token Coherence protocol: Initially each block := # tokens (==#procs) Read request: data and 1 token Write request: data and all tokens 14

15. University of Cantabria Edinburgh - PACT 2013 15 I0N/A P0P0 O2DATA P1P1 S1 P2P2 Sharers I Last Level Cache I0N/A Data_slice Dir_slice Memory Controller On-chip network Private Caches 1 2 3 4 5 StateNum. Tokens Data V 2 3 1

16. University of Cantabria Edinburgh - PACT 2013 When data not present in LLC broadcast for reconstruction Private caches inform of num. of held tokens Token counting avoids negative acknowledgements or timeouts Reconstruction message piggybacks type of request and requestor Key: directory may replace silently no invalidations 16

17. University of Cantabria Edinburgh - PACT 2013 P0P1P2 Invalid State IS Read P3 DirLLC State S State O State C Data + token State A Reconstruction Info 1 token Info 2 tokens Owner Unblock (info 1 token) Read Forward GETS to Owner Sharers [P2] Owner: ¿? Sharers [P2, P1] Owner: P1 Sharers [P2, P1, P0] Owner: P1 Data + token 3 tokens1 token Unblock Sharers [P2, P1, P0, P3] Owner: P1 17

18. University of Cantabria Edinburgh - PACT 2013 P0P1P2 Invalid State IS Write P3 DirLLC State S State O State C Data + 3 tokens State A Reconstruction Sharers [P0] Owner: P0 3 tokens1 token State IM State M 1 token Unblock (info all tokens) 18 Directory Eviction

19. University of Cantabria Edinburgh - PACT 2013 Motivation Directory Schemas In-cache Sparse MOSAIC Coherence Protocol Examples Evaluation Results Conclusions 19

20. University of Cantabria Edinburgh - PACT 2013 Core 0Core 1Core 2Core 3 Core 4Core 5Core 6Core 7 RRRR RRRR RRRR RRRR Slice 0Slice 2Slice 1Slice 3 Slice 4Slice 6Slice 5Slice 7 Slice 8Slice 10Slice 9Slice 11 Slice 12Slice 14Slice 13Slice 15 20

21. University of Cantabria Edinburgh - PACT 2013 GEMS: full-system evaluation SLICC: Specification Language for Implementing Cache Coherence 21

22. University of Cantabria Edinburgh - PACT 2013 Normalized execution time 22 128KB 16K entries (8 bytes per entry)

23. University of Cantabria Edinburgh - PACT 2013 23 Normalized num. misses

24. University of Cantabria Edinburgh - PACT 2013 Normalized execution time 24 128KB 16K entries (8 bytes per entry) 16KB 2K entries

25. University of Cantabria Edinburgh - PACT 2013 25 16KB 2K entries

26. University of Cantabria Edinburgh - PACT 2013 Average network link utilization 26

27. University of Cantabria Edinburgh - PACT 2013 27 40%!!

28. University of Cantabria Edinburgh - PACT 2013 28 Normalized link utilization 16 cores configuration

29. University of Cantabria Edinburgh - PACT 2013 Low complexity and great scalability Very low storage overhead No noticeable energy cost Alternative for future many-core cache coherent CMPs Bandwidth scalability of a directory Elegancy of Token Coherence MOSAIC Coherence Protocol 29

30. University of Cantabria Edinburgh - PACT 2013 30

31. University of Cantabria Edinburgh - PACT 2013 31

32. University of Cantabria Edinburgh - PACT 2013 32 Normalized execution time - Same experiment with BASE: 20% impact in some cases L1: 4-way 32KB / L2: 8-way 256KB x2 full dir1/10 full dir

33. University of Cantabria Edinburgh - PACT 2013 33 Normalized Dynamic Energy