White Box Testing and Symbolic Execution Written by Michael Beder

White Box Testing and Symbolic Execution 2 Agenda What is White Box Testing? Flow Graph and Coverage Types Symbolic Execution: – Formal Definition –Examples – Questions

White Box Testing and Symbolic Execution 3 What is White Box Testing? Software testing approach that uses inner structural and logical properties of the program for verification and deriving test data Also called: Clear Box Testing, Glass Box Testing and Structural Testing Manual: Inspection, Walkthrough Automatic: Syntax Parser, Symbolic Execution

White Box Testing and Symbolic Execution 4 Pros and Cons Pros: Usage of more information on the tested object (than BlackBox) Inference of real Equivalence Partitioning Structural Coverage Assurance Cons: Expensive Limited Semantic Coverage

White Box Testing and Symbolic Execution 5 Example I Sin(x) { if (|x| < eps) return x; if (|x| < 5*eps) return x – (x^3)/6; … } Black Box Testing: Test 0, Pi/2, -Pi/2 White Box Testing: Test 0.5*eps, eps, 3*eps, 5*eps, 7*eps, … Usage of logical properties makes better coverage

White Box Testing and Symbolic Execution 6 Example II: Unit Testing f(x)g(x) {h(x) { {…… if (x > 0) return g(x);}} return h(x); } Black Box Testing: Test f(x), g(x), h(x) for every x White Box Testing: Test g(x) for x > 0, h(x) for x <= 0 and verify f(x) Usage of structural properties makes fewer, qualitative tests

White Box Testing and Symbolic Execution 7 Flow Graph Abstraction of the program Defines the data and control flow in the program Uniform representation of the program, language independent Simple basic elements: assignment and condition Further analysis is performed using graph algorithms

White Box Testing and Symbolic Execution 8 Flow Graph – cont. G = (V, E) where - V is a set of basic blocks. start, end in V - E is a set of control branches Example: 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4If (a^2 > c) 5c = c + a 6a = a - 2 } 1, 2345endstart T T 6 F F Input: b = 2 Output: a = 0, c = 2

White Box Testing and Symbolic Execution 9 Basic Path Set Let p1, p2 be paths from start to end. Then, p1 < p2 if there exists a vertex v that belongs to p2 and not to p1 A basic path set is a maximal set of paths p1, p2, …, pk such that pi < pj for i < j

White Box Testing and Symbolic Execution 10 White Box Coverage Types Statement Coverage: Every statement is executed Branch Coverage: Every branch option is chosen Path Coverage: Every path is executed Basic Path Coverage: Every basic path is executed Loops?

White Box Testing and Symbolic Execution 11 Basic Path Coverage Basic path set is of size E – N + 2 (Linear Complexity) Each path is called “basic path” Example: p1 = start – 1,2 – 3 – end p2 = start – 1,2 – 3 – 4 – 6 – 3 – end p3 = start – 1,2 – 3 – 4 – 5 – 6 – 3 – end E – N + 2 = 8 – = 3 1, 2345endstart T T 6 F F

White Box Testing and Symbolic Execution 12 Path Function A function, when D is the working domain Represents the current values of the variables as function of their initial values Each variable X is represented by a projection function Function composition: For example:

White Box Testing and Symbolic Execution 13 Path Condition A condition that should be fulfilled for going along the path A constraint on the initial values of the variables For Example: p = start – 1,2 – 3 – end. 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4If (a^2 > c) 5c = c + a 6a = a - 2 } The path condition is B <= 1, when B is b’s value at start 1, end start T T 6 F F

White Box Testing and Symbolic Execution 14 Symbolic Execution A method for deriving test cases which satisfy a given path Performed as a simulation of the computation on the path Initial path function = Identity function, Initial path condition = true Each vertex on the path induce a symbolic composition on the path function and a logical constraint on the path condition: If an assignment was made: If a conditional decision was made: path condition path condition branch condition Output: path function and path condition for the given path

White Box Testing and Symbolic Execution 15 Example: Symbolic Composition x = x + y y = y + x end The final path function represents the values of X, Y, Z after both assignments as a function of their initial value

White Box Testing and Symbolic Execution 16 Concatenation and Associativity If is the path function of path and is the path function of path then is the path function of path The composition is associative: Symbolic Execution is a special case when V1 VnVk

White Box Testing and Symbolic Execution 17 Example: Symbolic Execution 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4If (a^2 > c) 5c = c + a 6a = a - 2 } Find test case for path: p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end 1, end start T T 6 F F input = b output = c

White Box Testing and Symbolic Execution 18 Example: Symbolic Execution 1, 2345endstart T T 6 F F 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4If (a^2 > c) 5c = c + a 6a = a - 2 } p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end vertexpath functionpath condition start: (A, B, C) true 1,2 (A, B, C) true 3 (B, B, 0) true 4 (B, B, 0) (true Λ B>1) ↔ B>1 5 (B, B, 0) (B>1 Λ B^2>0) ↔ B>1 input = b output = c

White Box Testing and Symbolic Execution 19 Example: Symbolic Execution 1, 2345endstart T T 6 F F 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4If (a^2 > c) 5c = c + a 6a = a – 2 } p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end vertex path function path condition 6(B, B, B) B>1 3(B-2, B, B) B>1 4(B-2, B, B)(B>1 Λ B-2>1) ↔ B>3 5(B-2, B, B) (B>3 Λ (B-2)^2>B) ↔ B>4 6(B-2, B, 2B-2) B>4 3(B-4, B, 2B-2) B>4 end (B-4, B, 2B-2) (B>4 Λ B-4<=1) ↔ B=5 input = b output = c

White Box Testing and Symbolic Execution 20 Example: Symbolic Execution 1, end start T T 6 F F 1 a = Read(b) 2 c = 0 3 while (a > 1) { 4if (a^2 > c) 5c = c + a 6a = a – 2 } p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end end (B-4, B, 2B-2) B=5 Hence the test case is B = 5 and the expected result is 2B-2 = 8. Is there a test case for p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end ? input = b output = c

White Box Testing and Symbolic Execution 21 Question (from exam) 1d = b + c; 2if (d > 20) 3a = 3 * a + d; 4if (b < a) { 5 a = 1; 6 if (d < 2 * b) 7 b = 2; 8} 1.Draw program’s Flow Graph 2.Find minimal number of test cases for the following coverage types: a)Statement Coverage b)Path Coverage c)Branch Coverage d)Basic Path Coverage

White Box Testing and Symbolic Execution 22 White Box Testing vs. Black Box Testing Given a function f(X1, X2, …, X10) with the following preconditions: 1.Every parameter is odd 2.Every parameter is less or equal to M 3.Some parameter is equal to M The function should report about every precondition that is not fulfilled f examines each parameter in turn using if statements (without else) and handles differently the following cases: a.Exactly one parameter is higher than M b.Two or more parameters are higher than M Check f’s correctness using White/Black Box Testing methods