1. White Box Testing and Symbolic Execution Written by Michael Beder

2. White Box Testing and Symbolic Execution 2 Agenda What is White Box Testing? Flow Graph and Coverage Types Symbolic Execution: – Formal Definition –Examples – Questions

3. White Box Testing and Symbolic Execution 3 What is White Box Testing? Software testing approach that uses inner structural and logical properties of the program for verification and deriving test data Also called: Clear Box Testing, Glass Box Testing and Structural Testing Manual: Inspection, Walkthrough Automatic: Syntax Parser, Symbolic Execution

4. White Box Testing and Symbolic Execution 4 Pros and Cons Pros: Usage of more information on the tested object (than BlackBox) Inference of real Equivalence Partitioning Structural Coverage Assurance Cons: Expensive Limited Semantic Coverage

5. White Box Testing and Symbolic Execution 5 Example I Sin(x) { if (|x| < eps) return x; if (|x| < 5*eps) return x – (x^3)/6; … } Black Box Testing: Test 0, Pi/2, -Pi/2 White Box Testing: Test 0.5*eps, eps, 3*eps, 5*eps, 7*eps, … Usage of logical properties makes better coverage

6. White Box Testing and Symbolic Execution 6 Example II: Unit Testing f(x)g(x) {h(x) { {…… if (x > 0) return g(x);}} return h(x); } Black Box Testing: Test f(x), g(x), h(x) for every x White Box Testing: Test g(x) for x > 0, h(x) for x <= 0 and verify f(x) Usage of structural properties makes fewer, qualitative tests

7. White Box Testing and Symbolic Execution 7 Flow Graph Abstraction of the program Defines the data and control flow in the program Uniform representation of the program, language independent Simple basic elements: assignment and condition Further analysis is performed using Graph Algorithms

8. White Box Testing and Symbolic Execution 8 Flow Graph – cont. G = (V, E) when V is a set of basic blocks. start, end in V E is a set of control branches Example: 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4If (a^2 > c) 5c = c + a 6a = a - 2 } 1, 2345endstart T T 6 F F Input: b = 2 Output: a = 0, c = 2

9. White Box Testing and Symbolic Execution 9 White Box Coverage Types Statement Coverage:Every statement is executed Branch Coverage: Every branch option is chosen Path Coverage: Every path is executed Basic Path Coverage:Every basic path is executed All definition-use path Coverage: All paths between the definition and a usage of a variable are executed Loops?

10. White Box Testing and Symbolic Execution 10 Basic Path Coverage Let p1, p2 be paths from start to end. Then, p1 < p2 if exists vertex v such that p2 contains v and p1 doesn’t contain v Find maximal set of paths p1, p2, …, pk such that pi < pj for i < j Such set of paths is of size E – N + 2 (Linear Complexity) Each path is called “basic path” Example: p1 = start – 1,2 – 3 – end p2 = start – 1,2 – 3 – 4 – 6 – 3 – end p3 = start – 1,2 – 3 – 4 – 5 – 6 – 3 – end E – N + 2 = 8 – 7 + 2 = 3 1, 2345endstart T T 6 F F

11. White Box Testing and Symbolic Execution 11 Path Function A function, when D is the working domain Represents the current values of the variables as function of their initial values Each variable X is represented by a projection function Function composition: For example:

12. White Box Testing and Symbolic Execution 12 Path Condition A condition that should be fulfilled for going along the path A constraint on the initial values of the variables For Example: p = start – 1,2 – 3 – end. 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4If (a^2 > c) 5c = c + a 6a = a - 2 } The path condition is B <= 1, when B is b’s value at start 1, 2 3 4 5 end start T T 6 F F

13. White Box Testing and Symbolic Execution 13 Symbolic Execution A method for deriving test cases which satisfy a given path Performed as a simulation of the computation on the path Initial path function = Identity function, Initial path condition = true Each vertex on the path concludes a symbolic composition on the path function and a logical constraint on the path condition: If an assignment was made: If a conditional decision was made: path condition path condition branch condition Output: path function and path condition at the final vertex of the path

14. White Box Testing and Symbolic Execution 14 Example III x = x + y y = y + x end The final path function represents the values of X, Y, Z after both assignments as a function of their initial values

15. White Box Testing and Symbolic Execution 15 Concatenation and Associativity If is the path function of path and is the path function of path then is the path function of path The composition is associative: Symbolic Execution is a special case when V1 VnVk

16. White Box Testing and Symbolic Execution 16 Example IV 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4If (a^2 > c) 5c = c + a 6a = a - 2 } Find test case for path: p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end 1, 2 3 4 5 end start T T 6 F F input = b output = c

17. White Box Testing and Symbolic Execution 17 Example IV 1, 2345endstart T T 6 F F 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4If (a^2 > c) 5c = c + a 6a = a - 2 } p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end vertexpath functionpath condition start: (A, B, C) true 1,2 (A, B, C) true 3 (B, B, 0) true 4 (B, B, 0) (true Λ B>1) ↔ B>1 5 (B, B, 0) (B>1 Λ B^2>0) ↔ B>1 input = b output = c

18. White Box Testing and Symbolic Execution 18 Example IV 1, 2345endstart T T 6 F F 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4If (a^2 > c) 5c = c + a 6a = a – 2 } p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end vertex path function path condition 6(B, B, B) B>1 3(B-2, B, B) B>1 4(B-2, B, B)(B>1 Λ B-2>1) ↔ B>3 5(B-2, B, B) (B>3 Λ (B-2)^2>B) ↔ B>4 6(B-2, B, 2B-2) B>4 3(B-4, B, 2B-2) B>4 end (B-4, B, 2B-2) (B>4 Λ B-4<=1) ↔ B=5 input = b output = c

19. White Box Testing and Symbolic Execution 19 Example IV 1, 2 3 4 5 end start T T 6 F F 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4if (a^2 > c) 5c = c + a 6a = a – 2 } p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end end (B-4, B, 2B-2) B=5 Hence the test case is B = 5 and the expected result is 2B-2 = 8. Is there a test case for p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end ? input = b output = c

20. White Box Testing and Symbolic Execution 20 Question (from exam) 1d = b + c; 2if (d > 20) 3a = 3 * a + d; 4if (b < a) { 5 a = 1; 6 if (d < 2 * b) 7 b = 2; 8} 1.Draw program’s Flow Graph 2.Find minimal number of test cases for the following coverage types: a)Statement Coverage b)Path Coverage c)Branch Coverage d)Basic Path Coverage

21. White Box Testing and Symbolic Execution 21 White Box Testing vs. Black Box Testing Given a function f(X1, X2, …, X10) with the following preconditions: 1.Every parameter is odd 2.Every parameter is less or equal to M 3.Some parameter is equal to M The function should report about every precondition that is not fulfilled f examines each parameter in turn using if statements (without else) and handles differently the following cases: a.Exactly one parameter is higher than M b.Two or more parameters are higher than M Check f’s correctness using White/Black Box Testing methods