1. Hierarchical Cache Coherence Protocol Verification One Level at a Time through Assume Guarantee Xiaofang Chen, Yu Yang, Michael Delisi, Ganesh Gopalakrishnan and Ching-Tsun Chou Supported in part by SRC Contract TJ1318

2. 2 Project Summary  Past work –FMCAD 2006 –Decompose an M-CMP protocol to 3 abstract protocols –Verification time: over 30 hours  12 hour + 18 hour  This work –Decompose an M-CMP protocol to 4 abstract protocols –Verification time: over 30 hours  6 minutes –Memory usage: 18 GB  1.8 GB

3. 3 Outline  Background  Hierarchical coherence protocol benchmarks  Our compositional approach  Conclusion

4. 4 Multicores Will Be the Future! (Photo courtesy of Intel Corporation) More than 80% of chips shipped will be multi-core

5. 5 Hierarchical Cache Coherence Protocols Chip-level protocols Inter-cluster protocols Intra-cluster protocols dir mem dir mem …

6. 6 Example: Verification Complexity (I) RAC L2 Cache+Local Dir L1 Cache Main Mem Home ClusterRemote Cluster 1Remote Cluster 2 L1 Cache Global Dir RAC L2 Cache+Local Dir L1 Cache RAC L2 Cache+Local Dir L1 Cache

7. 7 Example: Verification Complexity (II)  High level modeling –Murphi, ~2500 LOC  Properties to be verified –Safety properties  Verification –IA-64 machine –18GB memory –40-bit hash compaction –Non-conclusive after >30 hours of state enumeration

8. 8 Outline Background  Hierarchical coherence protocol benchmarks  Our compositional approach  Conclusion

9. 9 An M-CMP Benchmark Protocol RAC L2 Cache+Local Dir L1 Cache Main Mem Home ClusterRemote Cluster 1Remote Cluster 2 L1 Cache Global Dir RAC L2 Cache+Local Dir L1 Cache RAC L2 Cache+Local Dir L1 Cache Inter-cluster Intra-cluster

10. 10 Protocol Features  Both levels use MESI protocols –Intra-cluster: FLASH –Inter-cluster: DASH  Silent drop on non-Modified cache lines  Network channels are non-FIFO  Inclusive caches  ~2500 LOC in Murphi

11. 11 Another Benchmark: Non-inclusive Caches RAC L2 Cache+Local Dir L1 Cache Main Mem Home ClusterRemote Cluster 1Remote Cluster 2 L1 Cache Global Dir RAC L2 Cache+Local Dir L1 Cache RAC L2 Cache+Local Dir L1 Cache

12. 12 Outline Background Hierarchical coherence protocol benchmarks  Our compositional approach  Conclusion

13. 13 Our Compositional Approach Original protocol

14. 14 Our Compositional Approach

15. 15 Workflow of Our Approach

16. 16 Our Approach in FMCAD 2006  Construct three abstract protocols  Each with 1 detailed cluster + 2 abstracted clusters RAC L2 Cache+Local Dir’ Main Mem Home Cluster Remote Cluster 1 Global Dir RAC L2 Cache+Local Dir L1 Cache RAC L2 Cache+Local Dir’ Remote Cluster 2

17. 17 Problems with This Approach  Every abstract protocol contains 2 protocols  Duplicated behaviors in abstract protocols  State space still large 18 636,613,051M2M2 1812 284,088,425M1M1 Mem (GB)Time (hour)# of states

18. 18 A New Way to Decompose Protocols RAC L2 Cache+Local Dir’ Main Mem Home ClusterRemote Cluster 1Remote Cluster 2 RAC L2 Cache+Local Dir’ Global Dir RAC L2 Cache+Local Dir’ Home Cluster Remote Cluster 1 ABS #1 ABS #2 ABS #3 L2 Cache+Local Dir L1 Cache L2 Cache+Local Dir L1 Cache

19. 19 Model Checking Results

20. 20 Details of Our Approach  Abstraction –States –Transitions, properties  Constraining –Assume guarantee reasoning

21. 21 Abstraction on States Intra-cluster Inter-cluster

22. 22  Rule: guard action  guard –Become more permissive  action –Allow more behaviors Abstracting Transitions and Properties

23. 23 An Example of Abstraction RAC L2 Cache+Local Dir L1 Cache RAC L2 Cache+Local Dir’ WB Clusters[c].WbMsg.Cmd = WB Clusters[c].L2.Data := Clusters[c].WbMsg.Data; Clusters[c].L2.HeadPtr := L2; … True Clusters[c].L2.Data := nondet ; … Abstract inter-cluster protocol Abstract intra-cluster protocol

24. 24 Abstraction, Now Constraining

25. 25 An Example of Constraining RAC L2 Cache+Local Dir L1 Cache RAC L2 Cache+Local Dir’ WB Clusters[c].WbMsg.Cmd = WB Clusters[c].L2.State = Excl True & Clusters[c].L2.State = Excl Clusters[c].L2.Data := nondet; …

26. 26 For Non-inclusive Protocols  Using history variables  Add an auxiliary variable for each cluster – IE : boolean (implicit exclusive) – IE = true an exclusive/modified copy exists in the cluster

27. 27 Using History Variables for Non-inclusive Protocols RAC L2 Cache+Local Dir’ Main Mem Home ClusterRemote Cluster 1Remote Cluster 2 RAC L2 Cache+Local Dir’ Global Dir RAC L2 Cache+Local Dir’ Home Cluster Remote Cluster 1 L2 Cache+Local Dir L1 Cache L2 Cache+Local Dir L1 Cache

28. 28 Experimental Results

29. 29 Outline Background Hierarchical coherence protocol benchmarks Our compositional approach  Conclusion

30. 30 Related Work  Our FMCAD 2006 work  Chou et al FMCAD 2004 –A simple method for parameterized verification of cache coherence protocols  Compositional verification –Many previous works including McMillan, Jones, etc.  Token coherence protocol –Decouple correctness from performance

31. 31 Future Work  Automatic recognition of spurious bugs –Interface-aware BFS guided search  Automate guard strengthening –Obtain invariants through simulation

32. 32 Conclusion  Developed several hierarchical protocol benchmarks  Developed a compositional approach  Obtained promising experimental results

33. 33 Thank you.