1. Sparse Coding for Specification Mining and Error Localization Runtime Verification September 26, 2012 Wenchao Li, Sanjit A. Seshia University of California - Berkeley wenchaol@eecs.berkeley.edu

2. Runtime Verification 2012 Assertion-Based Verification 2 Problem: assertions are created manually Simulator Assertions Coverage Tests Circuit/Program Generate stimulus to patch coverage holes Find bugs with assertions “…typically 20% of specifications pass vacuously during the first formal verification runs of a new hardware design…” [IBM Haifa]

3. Runtime Verification 2012 Error Localization 3 Fatal Error 010101010101 011011010101 010111111010 10101 Where? Challenges: Limited observability Long error detection latency Transient and hard-to-reproduce bugs Idea: assertions can provide local observability and correctness checks

4. Runtime Verification 2012 Related Work Specification Mining: –Programs: single-state invariants, pre-/post-conditions, automata learning, alternating patterns –Circuits: fixed-delay pairs, temporal logic patterns –Require templates Error Localization: –Programs: model checking, predicates –Circuits: instruction footprints, SAT-based, mined assertion-based –Require system model and good observability –Require templates 4 Our technique is template-free and does not require having the system model

5. Runtime Verification 2012 What can you tell by just observing a trace? 1001110000 0011100010 1001010000 0010100010 5 Obj1.m1() Obj1.m2() Obj1.m1() Obj2.m1() Cloud  Hardware trace  Program trace  Human interaction/behavior  Sensor network  Distributed system

6. Runtime Verification 2012 A Sparse Coding Approach 6  0.8 * + 0.3 * + 0.5 * x  0.8 *   + 0.3 *   + 0.5 *   Key idea: Express each subtrace as a Boolean combination of a few “basis subtraces”– a (sparsity- constrained) Boolean matrix factorization problem. 11001 00101 Sparsity helps to uncover latent structure of the data

7. Runtime Verification 2012 Contributions and Outline A new formalism for discovering structure in a trace A definition of the sparsity-constrained Boolean matrix factorization problem and an algorithm for solving it Applications to specification mining and error localization –Does not rely on redefined templates –Simultaneous perform error localization and explanation Outline:  Problem formulation  Algorithm  Error localization and explanation  Results 7

8. Runtime Verification 2012 Problem Formulation 8 11001 00101 11 10 00 01 = ○ basis coefficient Multiplication as “AND” Addition as “OR” columns are sparse Subtrace

9. Runtime Verification 2012 Sparsity-Constrained Boolean Factorization 9 C = 2

10. Runtime Verification 2012 Algorithm Idea Observe that the data matrix X can be viewed as the adjacency matrix for a bipartitie graph. Idea: factorization → biclique cover (biclique ↔ basis subtrace) 10 v u

11. Runtime Verification 2012 Algorithm Overview Incrementally generate maximal bicliques –Consensus-based algorithm –Extend to a maximal biclique Keep track of closeness to sparsity constraint Heuristically optimize for basis sharing 11 B C D A C E A C DA E C DA E C DA E

12. Runtime Verification 2012 Y Z X G C D A E F B Algorithm Overview Step 1: start with the set of v-rooted star bicliques Step 2: Pick two stars and form a consensus Step 3: Extend the consensus to a maximal biclique Step 4: Add the biclique to the cover if possible Step 5: update sparsity constraint at the covered nodes 12 B C DAA C E C DA E … C D A E F C D A E F

13. Runtime Verification 2012 An Arbiter Example 13 A 2-input 2-output arbiter with round-robin scheme p0p0 010110…… p1p1 100111…… q0q0 010010…… q1q1 100101…… Sample mined assertions (basis subtrace): 000 100 000 100 010 000 010 000 011 010 001 010 12 Number of subtraces 01 10

14. Runtime Verification 2012 Error Localization and Explanation 14 01 0110…… 10 0111…… 01 0010…… 10 0101……

15. Runtime Verification 2012 All subtraces Example Illustration Error localization and explanation (arbiter example): 15 001 010 001 000 000 010 000 000 10001 11010 10001 01000 Error traceError subtraceError explanation 000 000 000 010 Alternative error Explanation Space spanned by the learned basis Correct subtraces Error

16. Runtime Verification 2012 Experimental Results Chip Multiprocessor Router: –Observe 14 control signals –Subtrace width of 2 cycles –Learn the basis from a single error- free trace of 1000 cycles: 0.243 seconds to obtain 189 basis subtraces from 93 distinct subtraces 16 Error Localization: –Inject a single bit flip at a random cycle for each of 99 error traces –Localize the error to the subtrace (out of 999) where it was injected Comparisons: –Baseline approach (1): hash all distinct subtraces – report error even before an error is injected for the 99 traces –Baseline approach (2): use unit basis – 0% localization –Sparse Coding: 55.6% localization A CMP Router in a NoC

17. Runtime Verification 2012 Conclusion A template-free assertion miner that can explore embedded patterns in digital circuit traces Effective assertion-mining based error localization and explanation Potential applications to other domains, e.g. programs or distributed systems 17 THANK YOU