1. Token Coherence: Decoupling Performance and Correctness Milo M. D. Martin Mark D. Hill David A. Wood University of Wisconsin-Madison ISCA-30 (2003)

2. Outline Token Coherence Basics –What Is Token Coherence? –What Are Its Advantages? Review of Snooping and Directory-Based Coherence Schemes Token Coherence Details Evaluation Results and Conclusions

3. Token Coherence Basics Decouple Interconnect Performance and Protocol Correctness –Remember Amdahl’s Law: Go Faster In Most Cases At Expense Of Occasional, More Expensive Corner Case Handling Fast, Unordered Bus For Protocol Traffic –Races Allowed, Handling Built-In, But Deferred –Cache Lines Have Tokens To Track State Token Ownership Correlates With Coherence State (MOSI)

4. Token Coherence Benefits Fast Bus Makes Common Case Fast –Some Additional Bus Traffic (Not Too Much) –Common Is Very Common (95+%) Benefits Of Snooping And Directories –Unordered Bus Messages (Directories) –Cache-Cache Transfers Without Indirection (Snooping)

5. Outline Token Coherence Basics –What Is Token Coherence? –What Are Its Advantages? Review of Snooping and Directory-Based Coherence Schemes Token Coherence Details Evaluation Results and Conclusions

6. Coherence Review: Snooping Requests Broadcast, All Agents Snoop –One-Hop Communication (Low Latency) –Every Agent Processes Every Message Bus Is Point Of Synchronization (Total Order) –Hard To Scale To Higher Speeds/Larger Systems P0P1P2P3 Mem 123123 123123 1313 2323 3 2 1 1 2 Total Order of Events: 1 Before 2 Before 3 To All Bus Agents

7. Coherence Review: Directories Directory Manages Ordering Of Requests –Bus Can Be Faster (Unordered) –Adds Level Of Indirection To Cache-Cache Transactions (Extra Latency) –Messages Addressed To Specific Agents P0P1P2P3 Dir 33 3 2 1 12 Bus Order Irrelevant, Ordering At Directory: 1 Before 2 Before 3 33

8. Outline Token Coherence Basics –What Is Token Coherence? –What Are Its Advantages? Review of Snooping and Directory-Based Coherence Schemes Token Coherence Details Evaluation Results and Conclusions

9. Token Coherence Goals Achieve Efficiency Of Snooping –Low-Latency Inter-Cache Transfers Keep Advantage Of Directories –Unordered, Fast Interconnects Allowed How? –Use Of Tokens To Implement “Correctness Substrate” And Protocol(s) –Tokens Associated With Each Cache Line At Least N Tokens Per Line (N = # Processors)

10. Correctness Substrate Enforce Safety – Count Tokens 1.Tokens Preserved (Always T Tokens Per Line) 2.Must Hold All Tokens To Write 3.Must Hold One Token To Read 4.If A Message Has A Token, It Must Have Data Optimization – Add Special Owner Token 1.Tokens Preserved With One Owner Token Per Line 2.Must Hold All Tokens To Write 3.Must Hold One Token And Valid Data To Read 4.If A Message Has Owner Token, It Must Have Data

11. A Token Coherence Protocol TokenB – Token Coherence Using Broadcast –Processors Broadcast Transient Token Requests –Snooping Mayhem Ensues Without Total Ordering Of Transactions On Bus Races Allowed To Occur – Unanswered Messages Retried Last Resort (After Several Retries) Is Aptly-Named Persistent Request

12. Outline Token Coherence Basics –What Is Token Coherence? –What Are Its Advantages? Review of Snooping and Directory-Based Coherence Schemes Token Coherence Details Evaluation Results and Conclusions

13. Performance Evaluation Compared To: Snooping, Directories, And Glueless “Estimated” Protocol Simulation Results Show… –Token Coherence Works, Performs Well, Even –For 16-Processor System, >95% Of First Transient Requests Succeed With TokenB –Potentially Significant Additional Traffic Generated Over Directory Not Ultimately As Scalable As Directories

14. Conclusions And Discussion Token Coherence Provides Novel Solution To Snooping/Directory Hybrid Optimization Problem –Is Additional Space For Token Storage An Issue? –Why Didn’t Hammer/21364 Use This? With Glueless A Trend, Is This Doomed To Academia? –How Does This Compare To Multicast Snooping?