Download presentation
Presentation is loading. Please wait.
1
Leonardo de Moura Microsoft Research
2
SMT@Microsoft SATTheoriesSMT Arithmetic Bit-vectors Arrays …
3
SMT@Microsoft ArithmeticArithmetic
4
ArithmeticArithmetic Array Theory
5
SMT@Microsoft ArithmeticArithmetic Array Theory Uninterpreted Functions
6
M | F SMT@Microsoft Partial model Set of clauses
7
Guessing SMT@Microsoft p, q | p q, q r p | p q, q r
8
Deducing SMT@Microsoft p, s| p q, p s p | p q, p s
9
Backtracking SMT@Microsoft p, s| p q, s q, p q p, s, q | p q, s q, p q
10
Efficient decision procedures for conjunctions of ground atoms. SMT@Microsoft a=b, a 10 Examples: Congruence closure Dual Simplex Bellman-Ford …
11
SMT@Microsoft Z3 is a new solver developed at Microsoft Research. Development/Research driven by internal customers. Free for academic research. Interfaces: http://research.microsoft.com/projects/z3 Z3 TextC/C++.NETOCaml
12
SMT@Microsoft Linear real and integer arithmetica = 2b + 3 Fixed-size bit-vectorsa & (a – 1) Uninterpreted functionsf(a) = a Extensional arraysread(write(a, i, c), j) Quantifiers x: g(f(x)) = x Model generation
13
SMT@Microsoft Z3 Test case generation Verifying Compiler Predicate Abstraction
14
SMT@Microsoft Z3 Test case generation Verifying Compiler Predicate Abstraction
15
SMT@Microsoft Test (correctness + usability) is 95% of the deal: Dev/Test is 1-1 in products. Developers are responsible for unit tests. Tools: Annotations and static analysis (SAL + ESP) File Fuzzing Unit test case generation
16
Security is critical SMT@Microsoft Security bugs can be very expensive: Cost of each MS Security Bulletin: $600k to $Millions. Cost due to worms: $Billions. The real victim is the customer. Most security exploits are initiated via files or packets. Ex: Internet Explorer parses dozens of file formats. Security testing: hunting for million dollar bugs Write A/V Read A/V Null pointer dereference Division by zero
17
Two main techniques used by “black hats”: Code inspection (of binaries). Black box fuzz testing. Black box fuzz testing: A form of black box random testing. Randomly fuzz (=modify) a well formed input. Grammar-based fuzzing: rules to encode how to fuzz. Heavily used in security testing At MS: several internal tools. Conceptually simple yet effective in practice SMT@Microsoft
18
Execution Path Run Test and Monitor Path Condition Solve seed New input Test Inputs Constraint System Known Paths
19
PEX Implements DART for.NET. SAGE Implements DART for x86 binaries. YOGI Implements DART to check the feasibility of program paths generated statically using a SLAM-like tool. Vigilante Partially implements DART to dynamically generate worm filters. SMT@Microsoft
20
Test input generator Pex starts from parameterized unit tests Generated tests are emitted as traditional unit tests SMT@Microsoft
22
class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }... class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } }
![class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }...](http://images.slideplayer.com/33/8246883/slides/slide_22.jpg "class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } }.")
23
class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }... class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } } Inputs
![class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }...](http://images.slideplayer.com/33/8246883/slides/slide_23.jpg "class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } } Inputs.")
24
(0,null) class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }... class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } }
![(0,null) class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }...](http://images.slideplayer.com/33/8246883/slides/slide_24.jpg "class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } }.")
25
InputsObserved Constraints (0,null)!(c<0) class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }... class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } } c < 0 false
![InputsObserved Constraints (0,null)!(c<0) class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }...](http://images.slideplayer.com/33/8246883/slides/slide_25.jpg "class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } } c < 0 false.")
26
InputsObserved Constraints (0,null)!(c<0) && 0==c class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }... class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } } 0 == c true
![InputsObserved Constraints (0,null)!(c<0) && 0==c class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }...](http://images.slideplayer.com/33/8246883/slides/slide_26.jpg "class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } } 0 == c true.")
27
InputsObserved Constraints (0,null)!(c<0) && 0==c class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }... class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } } item == item true This is a tautology, i.e. a constraint that is always true, regardless of the chosen values. We can ignore such constraints.
![InputsObserved Constraints (0,null)!(c<0) && 0==c class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }...](http://images.slideplayer.com/33/8246883/slides/slide_27.jpg "class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } } item == item true This is a tautology, i.e. a constraint that is always true, regardless of the chosen values. We can ignore such constraints..")
28
Constraints to solve InputsObserved Constraints (0,null)!(c<0) && 0==c !(c<0) && 0!=c class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }... class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } }
![Constraints to solve InputsObserved Constraints (0,null)!(c<0) && 0==c !(c<0) && 0!=c class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }...](http://images.slideplayer.com/33/8246883/slides/slide_28.jpg "class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } }.")
29
Constraints to solve InputsObserved Constraints (0,null)!(c<0) && 0==c !(c<0) && 0!=c(1,null) class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }... class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } }
![Constraints to solve InputsObserved Constraints (0,null)!(c<0) && 0==c !(c<0) && 0!=c(1,null) class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }...](http://images.slideplayer.com/33/8246883/slides/slide_29.jpg "class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } }.")
30
Constraints to solve InputsObserved Constraints (0,null)!(c<0) && 0==c !(c<0) && 0!=c(1,null)!(c<0) && 0!=c class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }... class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } } 0 == c false
![Constraints to solve InputsObserved Constraints (0,null)!(c<0) && 0==c !(c<0) && 0!=c(1,null)!(c<0) && 0!=c class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }...](http://images.slideplayer.com/33/8246883/slides/slide_30.jpg "class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } } 0 == c false.")
31
Constraints to solve InputsObserved Constraints (0,null)!(c<0) && 0==c !(c<0) && 0!=c(1,null)!(c<0) && 0!=c c<0 class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }... class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } }
![Constraints to solve InputsObserved Constraints (0,null)!(c<0) && 0==c !(c<0) && 0!=c(1,null)!(c<0) && 0!=c c<0 class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }...](http://images.slideplayer.com/33/8246883/slides/slide_31.jpg "class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } }.")
32
Constraints to solve InputsObserved Constraints (0,null)!(c<0) && 0==c !(c<0) && 0!=c(1,null)!(c<0) && 0!=c c<0(-1,null) class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }... class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } }
![Constraints to solve InputsObserved Constraints (0,null)!(c<0) && 0==c !(c<0) && 0!=c(1,null)!(c<0) && 0!=c c<0(-1,null) class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }...](http://images.slideplayer.com/33/8246883/slides/slide_32.jpg "class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } }.")
33
Constraints to solve InputsObserved Constraints (0,null)!(c<0) && 0==c !(c<0) && 0!=c(1,null)!(c<0) && 0!=c c<0(-1,null)c<0 class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }... class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } } c < 0 true
![Constraints to solve InputsObserved Constraints (0,null)!(c<0) && 0==c !(c<0) && 0!=c(1,null)!(c<0) && 0!=c c<0(-1,null)c<0 class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }...](http://images.slideplayer.com/33/8246883/slides/slide_33.jpg "class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } } c < 0 true.")
34
Constraints to solve InputsObserved Constraints (0,null)!(c<0) && 0==c !(c<0) && 0!=c(1,null)!(c<0) && 0!=c c<0(-1,null)c<0 class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }... class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } }
![Constraints to solve InputsObserved Constraints (0,null)!(c<0) && 0==c !(c<0) && 0!=c(1,null)!(c<0) && 0!=c c<0(-1,null)c<0 class ArrayList { object[] items; int count; ArrayList(int capacity) { if (capacity < 0) throw...; items = new object[capacity]; } void Add(object item) { if (count == items.Length) ResizeArray(); items[this.count++] = item; }...](http://images.slideplayer.com/33/8246883/slides/slide_34.jpg "class ArrayListTest { [PexMethod] void AddItem(int c, object item) { var list = new ArrayList(c); list.Add(item); Assert(list[0] == item); } }.")
35
Rich Combination Linear arithmetic BitvectorArrays Free Functions Models Model used as test inputs -Quantifier Used to model custom theories (e.g.,.NET type system) API Huge number of small problems. Textual interface is too inefficient. SMT@Microsoft
36
Pex “sends” several similar formulas to Z3. Plus: backtracking primitives in the Z3 API. push pop Reuse (some) lemmas. SMT@Microsoft
37
Given a set of constraints C, find a model M that minimizes the interpretation for x 0, …, x n. In the ArrayList example: Why is the model where c = 2147483648 less desirable than the model with c = 1? !(c<0) && 0!=c Simple solution: Assert C while satisfiable Peek x i such that M[x i ] is big Assert x i < n, where n is a small constant Return last found model SMT@Microsoft
38
Given a set of constraints C, find a model M that minimizes the interpretation for x 0, …, x n. In the ArrayList example: Why is the model where c = 2147483648 less desirable than the model with c = 1? !(c<0) && 0!=c Refinement: Eager solution stops as soon as the system becomes unsatisfiable. A “bad” choice (peek x i ) may prevent us from finding a good solution. Use push and pop to retract “bad” choices. SMT@Microsoft
39
Apply DART to large applications (not units). Start with well-formed input (not random). Combine with generational search (not DFS). Negate 1-by-1 each constraint in a path constraint. Generate many children for each parent run. SMT@Microsoft parent generation 1
40
Starting with 100 zero bytes … SAGE generates a crashing test for Media1 parser SMT@Microsoft 00000000h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000020h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000030h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000040h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000050h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000060h: 00 00 00 00 ;.... Generation 0 – seed file
41
Starting with 100 zero bytes … SAGE generates a crashing test for Media1 parser SMT@Microsoft 00000000h: 52 49 46 46 00 00 00 00 00 00 00 00 00 00 00 00 ; RIFF............ 00000010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000020h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000030h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000040h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000050h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000060h: 00 00 00 00 ;.... Generation 1
42
` Starting with 100 zero bytes … SAGE generates a crashing test for Media1 parser SMT@Microsoft 00000000h: 52 49 46 46 00 00 00 00 ** ** ** 20 00 00 00 00 ; RIFF....***.... 00000010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000020h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000030h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000040h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000050h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000060h: 00 00 00 00 ;.... Generation 2
43
Starting with 100 zero bytes … SAGE generates a crashing test for Media1 parser SMT@Microsoft 00000000h: 52 49 46 46 3D 00 00 00 ** ** ** 20 00 00 00 00 ; RIFF=...***.... 00000010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000020h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000030h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000040h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000050h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000060h: 00 00 00 00 ;.... Generation 3
44
Starting with 100 zero bytes … SAGE generates a crashing test for Media1 parser SMT@Microsoft 00000000h: 52 49 46 46 3D 00 00 00 ** ** ** 20 00 00 00 00 ; RIFF=...***.... 00000010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000020h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000030h: 00 00 00 00 73 74 72 68 00 00 00 00 00 00 00 00 ;....strh........ 00000040h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000050h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000060h: 00 00 00 00 ;.... Generation 4
45
Starting with 100 zero bytes … SAGE generates a crashing test for Media1 parser SMT@Microsoft 00000000h: 52 49 46 46 3D 00 00 00 ** ** ** 20 00 00 00 00 ; RIFF=...***.... 00000010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000020h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000030h: 00 00 00 00 73 74 72 68 00 00 00 00 76 69 64 73 ;....strh....vids 00000040h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000050h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000060h: 00 00 00 00 ;.... Generation 5
46
Starting with 100 zero bytes … SAGE generates a crashing test for Media1 parser SMT@Microsoft 00000000h: 52 49 46 46 3D 00 00 00 ** ** ** 20 00 00 00 00 ; RIFF=...***.... 00000010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000020h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000030h: 00 00 00 00 73 74 72 68 00 00 00 00 76 69 64 73 ;....strh....vids 00000040h: 00 00 00 00 73 74 72 66 00 00 00 00 00 00 00 00 ;....strf........ 00000050h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000060h: 00 00 00 00 ;.... Generation 6
47
Starting with 100 zero bytes … SAGE generates a crashing test for Media1 parser SMT@Microsoft 00000000h: 52 49 46 46 3D 00 00 00 ** ** ** 20 00 00 00 00 ; RIFF=...***.... 00000010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000020h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000030h: 00 00 00 00 73 74 72 68 00 00 00 00 76 69 64 73 ;....strh....vids 00000040h: 00 00 00 00 73 74 72 66 00 00 00 00 28 00 00 00 ;....strf....(... 00000050h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000060h: 00 00 00 00 ;.... Generation 7
48
Starting with 100 zero bytes … SAGE generates a crashing test for Media1 parser SMT@Microsoft 00000000h: 52 49 46 46 3D 00 00 00 ** ** ** 20 00 00 00 00 ; RIFF=...***.... 00000010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000020h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000030h: 00 00 00 00 73 74 72 68 00 00 00 00 76 69 64 73 ;....strh....vids 00000040h: 00 00 00 00 73 74 72 66 00 00 00 00 28 00 00 00 ;....strf....(... 00000050h: 00 00 00 00 00 00 00 00 00 00 00 00 C9 9D E4 4E ;............ÉäN 00000060h: 00 00 00 00 ;.... Generation 8
49
Starting with 100 zero bytes … SAGE generates a crashing test for Media1 parser SMT@Microsoft 00000000h: 52 49 46 46 3D 00 00 00 ** ** ** 20 00 00 00 00 ; RIFF=...***.... 00000010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000020h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000030h: 00 00 00 00 73 74 72 68 00 00 00 00 76 69 64 73 ;....strh....vids 00000040h: 00 00 00 00 73 74 72 66 00 00 00 00 28 00 00 00 ;....strf....(... 00000050h: 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 ;................ 00000060h: 00 00 00 00 ;.... Generation 9
50
Starting with 100 zero bytes … SAGE generates a crashing test for Media1 parser SMT@Microsoft 00000000h: 52 49 46 46 3D 00 00 00 ** ** ** 20 00 00 00 00 ; RIFF=...***.... 00000010h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000020h: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ;................ 00000030h: 00 00 00 00 73 74 72 68 00 00 00 00 76 69 64 73 ;....strh....vids 00000040h: 00 00 00 00 73 74 72 66 B2 75 76 3A 28 00 00 00 ;....strf²uv:(... 00000050h: 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 ;................ 00000060h: 00 00 00 00 ;.... Generation 10 – CRASH
51
SAGE is very effective at finding bugs. Works on large applications. Fully automated Easy to deploy (x86 analysis – any language) Used in various groups inside Microsoft Powered by Z3. SMT@Microsoft
52
Formulas are usually big conjunctions. SAGE uses only the bitvector and array theories. Pre-processing step has a huge performance impact. Eliminate variables. Simplify formulas. Early unsat detection. SMT@Microsoft
53
Z3 Test case generation Verifying Compiler Predicate Abstraction
54
Source Language C# + goodies = Spec# Specifications method contracts, invariants, field and type annotations. Program Logic: Dijkstra’s weakest preconditions. Automatic Verification type checking, verification condition generation (VCG), automatic theorem proving Z3 Spec# (annotated C#) Boogie PL Spec# Compiler VC Generator Formulas Z3
55
V V Static program verifier (Boogie) MSIL Z3 V.C. generator Verification condition “correct” or list of errors Spec# compiler Spec# C Bytecode translator C Boogie VCC HAVOC
56
SMT@Microsoft A tool for specifying and checking properties of systems software written in C. It also translates annotated C into Boogie PL. It allows the expression of richer properties about the program heap and data structures such as linked lists and arrays. HAVOC is being used to specify and check: Complex locking protocols over heap-allocated data structures in Windows. Properties of collections such as IRP queues in device drivers. Correctness properties of custom storage allocators.
57
VCC translates an annotated C program into a Boogie PL program. A C-ish memory model Abstract heaps Bit-level precision Microsoft Hypervisor: verification grand challenge. SMT@Microsoft
58
Quantifiers, quantifiers, quantifiers, … Modeling the runtime h,o,f: IsHeap(h) o ≠ null read(h, o, alloc) = t read(h,o, f) = null read(h, read(h,o,f),alloc) = t SMT@Microsoft
59
Quantifiers, quantifiers, quantifiers, … Modeling the runtime Frame axioms o, f: o ≠ null read(h 0, o, alloc) = t read(h 1,o,f) = read(h 0,o,f) (o,f) M SMT@Microsoft
60
Quantifiers, quantifiers, quantifiers, … Modeling the runtime Frame axioms User provided assertions i,j: i j read(a,i) read(b,j) SMT@Microsoft
61
Quantifiers, quantifiers, quantifiers, … Modeling the runtime Frame axioms User provided assertions Theories x: p(x,x) x,y,z: p(x,y), p(y,z) p(x,z) x,y: p(x,y), p(y,x) x = y SMT@Microsoft
62
Quantifiers, quantifiers, quantifiers, … Modeling the runtime Frame axioms User provided assertions Theories Solver must be fast in satisfiable instances. SMT@Microsoft We want to find bugs!
63
SMT@Microsoft SMT solvers use heuristic quantifier instantiation. E-matching (matching modulo equalities). Example: x: f(g(x)) = x { f(g(x)) } a = g(b), b = c, f(a) c Pattern
64
SMT@Microsoft SMT solvers use heuristic quantifier instantiation. E-matching (matching modulo equalities). Example: x: f(g(x)) = x { f(g(x)) } a = g(b), b = c, f(a) c x=b f(g(b)) = b Equalities and ground terms come from the partial model M
65
SMT@Microsoft Integrates smoothly with DPLL. Software verification problems are big & shallow. Decides useful theories: Arrays Partial orders …
66
SMT@Microsoft E-matching is NP-Hard. In practice
67
SMT@Microsoft Pattern: f(x1, g(x1, a), h(x2), b) Pattern: f(x1, g(x1, a), h(x2), b) Instructions: 1.init(f, 2) 2.check(r4, b, 3) 3.bind(r2, g, r5, 4) 4.compare(r1, r5, 5) 5.check(r6, a, 6) 6.bind(r3, h, r7, 7) 7.yield(r1, r7) Instructions: 1.init(f, 2) 2.check(r4, b, 3) 3.bind(r2, g, r5, 4) 4.compare(r1, r5, 5) 5.check(r6, a, 6) 6.bind(r3, h, r7, 7) 7.yield(r1, r7) CompilerCompiler Similar patterns share several instructions. Combine code sequences in a code tree
68
SMT@Microsoft E-matching needs ground seeds. x: p(x), x: not p(x)
69
SMT@Microsoft E-matching needs ground seeds. Bad user provided patterns: x: f(g(x))=x { f(g(x)) } g(a) = c, g(b) = c, a b Pattern is too restrictive
70
SMT@Microsoft E-matching needs ground seeds. Bad user provided patterns: x: f(g(x))=x { g(x) } g(a) = c, g(b) = c, a b More “liberal” pattern More “liberal” pattern
71
SMT@Microsoft E-matching needs ground seeds. Bad user provided patterns: x: f(g(x))=x { g(x) } g(a) = c, g(b) = c, a b, f(g(a)) = a, f(g(b)) = b a=b
72
SMT@Microsoft E-matching needs ground seeds. Bad user provided patterns. Matching loops: x: f(x) = g(f(x)) {f(x)} x: g(x) = f(g(x)) {g(x)} f(a) = c
73
SMT@Microsoft E-matching needs ground seeds. Bad user provided patterns. Matching loops: x: f(x) = g(f(x)) {f(x)} x: g(x) = f(g(x)) {g(x)} f(a) = c f(a) = g(f(a))
74
SMT@Microsoft E-matching needs ground seeds. Bad user provided patterns. Matching loops: x: f(x) = g(f(x)) {f(x)} x: g(x) = f(g(x)) {g(x)} f(a) = c f(a) = g(f(a)) g(f(a)) = f(g(f(a)))
75
SMT@Microsoft E-matching needs ground seeds. Bad user provided patterns. Matching loops. It is not refutationally complete.
76
SMT@Microsoft Decidable fragments: EPR Array property fragment More coming soon DPLL( ): DPLL + Saturation CEGAR-like loop Z3 2.0 won all -divisions in SMT-COMP’08
77
SMT@Microsoft Inference rule: DPLL( ) is parametric. Examples: Resolution Superposition calculus …
78
DPLL + Theories DPLL + Theories Saturation Solver Saturation Solver SMT@Microsoft Saturation solver ignores non-unit ground clauses. It is still refutanionally complete if: has the reduction property. Ground literals Ground clauses
79
SMT@Microsoft Generate candidate model Model check Instantiate quantifiers
80
SMT@Microsoft Z3 Test case generation Verifying Compiler Predicate Abstraction
81
SMT@Microsoft http://research.microsoft.com/slam/ SLAM/SDV is a software model checker. Application domain: device drivers. Architecture: c2bp C program → boolean program (predicate abstraction). bebop Model checker for boolean programs. newton Model refinement (check for path feasibility) SMT solvers are used to perform predicate abstraction and to check path feasibility. c2bp makes several calls to the SMT solver. The formulas are relatively small.
82
Given a C program P and F = {p 1, …, p n }. Produce a Boolean program B(P, F) Same control flow structure as P. Boolean variables {b 1, …, b n } to match {p 1, …, p n }. Properties true in B(P, F) are true in P. Each p i is a pure Boolean expression. Each p i represents set of states for which p i is true. Performs modular abstraction. SMT@Microsoft
83
Implies F (e) Best Boolean function over F that implies e. ImpliedBy F (e) Best Boolean function over F that is implied by e. ImpliedBy F (e) = not Implies F (not e) SMT@Microsoft
84
minterm m = l 1 ∧... ∧ l n, where l i = p i, or l i = not p i. Implies F (e): disjunction of all minterms that imply e. Naive approach Generate all 2 n possible minterms. For each minterm m, use SMT solver to check validity of m ⇒ e. Many possible optimizations SMT@Microsoft
85
F = { x < y, x = 2} e : y > 1 Minterms over F !x 1 x 1 !x 1 x 1 Implies F (y>1) = x<y x=2 Implies F (y>1) = b 1 b 2 SMT@Microsoft
86
Given an error path p in the Boolean program B. Is p a feasible path of the corresponding C program? Yes: found a bug. No: find predicates that explain the infeasibility. Execute path symbolically. Check conditions for inconsistency using Z3. SMT@Microsoft
87
All-SAT Better (more precise) Predicate Abstraction Unsatisfiable cores Why the abstract path is not feasible? Fast Predicate Abstraction SMT@Microsoft
88
Let S be an unsatisfiable set of formulas. S’ S is an unsatisfiable core of S if: S’ is also unsatisfiable, and There is not S’’ S’ that is also unsatisfiable. Computing Implies F (e) with F = {p 1, p 2, p 3, p 4 } Assume p 1, p 2, p 3, p 4 e is valid That is p 1, p 2, p 3, p 4, e is unsat Now assume p 1, p 3, e is the unsatisfiable core Then it is unnecessary to check: p 1, p 2, p 3, p 4 e p 1, p 2, p 3, p 4 e p 1, p 2, p 3, p 4 e
89
Model programs (M. Veanes – MSRR) Termination (B. Cook – MSRC) Security protocols (A. Gordon and C. Fournet - MSRC) Business Application Modeling (E. Jackson - MSRR) Cryptography (R. Venki – MSRR) Verifying Garbage Collectors (C. Hawblitzel – MSRR) Model Based Testing (L. Bruck – SQL) Semantic type checking for D models (G. Bierman – MSRC) More coming soon… SMT@Microsoft
90
P = NP New Solvers P = NEXPTime EPR P = Ω Superposition Calculus SMT@Microsoft *
91
Z3 2.0 release. ManyZ3 Non linear arithmetic. Better support for bitvectors. Floating point numbers? More decidable fragments. Improved DPLL( ). Non convex optimization. New applications.
92
SMT@Microsoft SMT is hot at Microsoft. Many applications. Z3 is a new and very efficient SMT solver. http://research.microsoft.com/projects/z3 Thank You!
Similar presentations
