Lucía G. Menezo Valentín Puente José Ángel Gregorio University of Cantabria (Spain) MOSAIC :

University of Cantabria Edinburgh - PACT 2013 Motivation Directory Schemas In-cache Sparse MOSAIC Coherence Protocol Examples Evaluation Results Conclusions

University of Cantabria Edinburgh - PACT 2013 Performance improvement: more processors per chip Major challenges: off-chip bandwidth wall Introduce cache into the chip Complex on-chip cache hierarchies Coherence protocol: fundamental role to play 3

University of Cantabria Edinburgh - PACT 2013 What coherence protocol to use with large number of cores: Broadcast-based protocols high energy requirements Directory-based protocols more storage necessities for sharing information MOSAIC: new coherence protocol Directory without inclusiveness Token Coherence to guarantee correctness 4

University of Cantabria Edinburgh - PACT 2013 Motivation Directory Schemas In-cache Sparse MOSAIC Coherence Protocol Examples Evaluation Results Conclusions

University of Cantabria Edinburgh - PACT 2013 Each block in LLC includes tag, data and the sharers information LLC receives requests needs precise knowledge Inclusiveness is necessary: any block in the private levels needs to be allocated in LLC Advantage: coherence protocol less complex Disadvantage: all LLC blocks has storage overhead 6

University of Cantabria Edinburgh - PACT @ P Processors and private caches LLC + in-cache directory PPP Interconnection network P

University of Cantabria Edinburgh - PACT datasharers LLC + in-cache directory Interconnection network P Overhead!!! Processors and private caches

University of Cantabria Edinburgh - PACT 2013 Directory entries separated from data Allocated under demand Overhead proportional to the aggregate private levels size (not LLC) Capacity and associativity has to be sufficient to keep private-level cache tags 9

University of Cantabria Edinburgh - PACT data Interconnection network P sharers LLC Sparse dir Processors and private caches

University of Cantabria Edinburgh - PACT 2013 Duplicate-tag directory: holding all the tags of private levels Example: 16 cores with 4-way 32KB L1 64-way Associativity = # cores * private caches associativity # sets = # private caches sets tag 11

University of Cantabria Edinburgh - PACT 2013 tag 12 Decrease Associativity: now << # cores * private caches associativity tag sharers tag One tag may be in various private caches More than 1 tag per entry conflicts Inclusiveness needed invalidate private data (recalls messages) tag Increase number of sets

University of Cantabria Edinburgh - PACT 2013 Motivation Directory Schemas In-cache Sparse MOSAIC Coherence Protocol Examples Evaluation Results Conclusions 13

University of Cantabria Edinburgh - PACT 2013 In-cache or sparse it doesnt matter No inclusiveness No invalidations of data in private caches Reconstruction of sharing information under demand Uses token counting to avoid extra traffic and guarantee correctness Token Coherence protocol: Initially each block := # tokens (==#procs) Read request: data and 1 token Write request: data and all tokens 14

University of Cantabria Edinburgh - PACT I0N/A P0P0 O2DATA P1P1 S1 P2P2 Sharers I Last Level Cache I0N/A Data_slice Dir_slice Memory Controller On-chip network Private Caches StateNum. Tokens Data V 2 3 1

University of Cantabria Edinburgh - PACT 2013 When data not present in LLC broadcast for reconstruction Private caches inform of num. of held tokens Token counting avoids negative acknowledgements or timeouts Reconstruction message piggybacks type of request and requestor Key: directory may replace silently no invalidations 16

University of Cantabria Edinburgh - PACT 2013 P0P1P2 Invalid State IS Read P3 DirLLC State S State O State C Data + token State A Reconstruction Info 1 token Info 2 tokens Owner Unblock (info 1 token) Read Forward GETS to Owner Sharers [P2] Owner: ¿? Sharers [P2, P1] Owner: P1 Sharers [P2, P1, P0] Owner: P1 Data + token 3 tokens1 token Unblock Sharers [P2, P1, P0, P3] Owner: P1 17

University of Cantabria Edinburgh - PACT 2013 P0P1P2 Invalid State IS Write P3 DirLLC State S State O State C Data + 3 tokens State A Reconstruction Sharers [P0] Owner: P0 3 tokens1 token State IM State M 1 token Unblock (info all tokens) 18 Directory Eviction

University of Cantabria Edinburgh - PACT 2013 Motivation Directory Schemas In-cache Sparse MOSAIC Coherence Protocol Examples Evaluation Results Conclusions 19

University of Cantabria Edinburgh - PACT 2013 Core 0Core 1Core 2Core 3 Core 4Core 5Core 6Core 7 RRRR RRRR RRRR RRRR Slice 0Slice 2Slice 1Slice 3 Slice 4Slice 6Slice 5Slice 7 Slice 8Slice 10Slice 9Slice 11 Slice 12Slice 14Slice 13Slice 15 20

University of Cantabria Edinburgh - PACT 2013 GEMS: full-system evaluation SLICC: Specification Language for Implementing Cache Coherence 21

University of Cantabria Edinburgh - PACT 2013 Normalized execution time KB 16K entries (8 bytes per entry)

University of Cantabria Edinburgh - PACT Normalized num. misses

University of Cantabria Edinburgh - PACT 2013 Normalized execution time KB 16K entries (8 bytes per entry) 16KB 2K entries

University of Cantabria Edinburgh - PACT KB 2K entries

University of Cantabria Edinburgh - PACT 2013 Average network link utilization 26

University of Cantabria Edinburgh - PACT %!!

University of Cantabria Edinburgh - PACT Normalized link utilization 16 cores configuration

University of Cantabria Edinburgh - PACT 2013 Low complexity and great scalability Very low storage overhead No noticeable energy cost Alternative for future many-core cache coherent CMPs Bandwidth scalability of a directory Elegancy of Token Coherence MOSAIC Coherence Protocol 29

University of Cantabria Edinburgh - PACT

University of Cantabria Edinburgh - PACT

University of Cantabria Edinburgh - PACT Normalized execution time - Same experiment with BASE: 20% impact in some cases L1: 4-way 32KB / L2: 8-way 256KB x2 full dir1/10 full dir

University of Cantabria Edinburgh - PACT Normalized Dynamic Energy