# 1 T.Y. Chen Swinburne University of Technology, Australia T.H. Tse and Zhiquan Zhou The University of Hong Kong Semi-Proving: an Integrated Method Based.

## Presentation on theme: "1 T.Y. Chen Swinburne University of Technology, Australia T.H. Tse and Zhiquan Zhou The University of Hong Kong Semi-Proving: an Integrated Method Based."— Presentation transcript:

1 T.Y. Chen Swinburne University of Technology, Australia T.H. Tse and Zhiquan Zhou The University of Hong Kong Semi-Proving: an Integrated Method Based on Global Symbolic Evaluation and Metamorphic Testing (speaker)

2 Presentation Outline  Conventional Program Testing and Proving  Metamorphic Testing  Our method: Semi-Proving  Summary.

3  Conventional Program Testing and Proving  Metamorphic Testing  Our method: Semi-Proving  Summary. Presentation Outline

4 Conventional Program Testing and Proving Given a bijective function f ; A Program: F_Sort (a 1, a 2,..., a n ), n  2 Output: (a 1 ’, a 2 ’,..., a n ’), such that 1. (a 1 ’, a 2 ’,..., a n ’) is a permutation of (a 1, a 2,..., a n ) 2. f (a 1 ’)  f (a 2 ’) ...  f (a n ’). Given a bijective function f ; A Program: F_Sort (a 1, a 2,..., a n ), n  2 Output: (a 1 ’, a 2 ’,..., a n ’), such that 1. (a 1 ’, a 2 ’,..., a n ’) is a permutation of (a 1, a 2,..., a n ) 2. f (a 1 ’)  f (a 2 ’) ...  f (a n ’).

5 Conventional Program Testing and Proving  Testing 1. Design test cases: e.g. (2, 6, 3) for n=3 2. Run: F_Sort (2, 6, 3) = (6, 3, 2) 3. Check: f (6) < f (3) < f (2) ? 1. Design test cases: e.g. (2, 6, 3) for n=3 2. Run: F_Sort (2, 6, 3) = (6, 3, 2) 3. Check: f (6) < f (3) < f (2) ?

6 Conventional Program Testing and Proving  Proving correctness 1. F_Sort terminates for any valid input; 2. The output is correct. 1. F_Sort terminates for any valid input; 2. The output is correct.

7 Conventional Program Testing and Proving  Proving properties F_Sort (a 1, a 2,..., a n ) = (a 1 ’, a 2 ’,..., a n ’) Permutation.

8 Metamorphic Testing  Metamorphic Testing Employing relationships between different executions Fact: different permutations will produce same output F_Sort (a 1, a 2, a 3 ) Fact: different permutations will produce same output F_Sort (a 1, a 2, a 3 ) F_Sort (a 3, a 1, a 2 ) = “ Metamorphic Relation ” ·

9 Metamorphic Testing Metamorphic Test Cases: {(2, 6, 3), (3, 2, 6)} Metamorphic Testing: 1. F_Sort (2, 6, 3) = (6, 3, 2) Metamorphic Testing: 1. F_Sort (2, 6, 3) = (6, 3, 2) No matter whether an oracle is available or not; Very useful when the oracle cannot be found. 2. F_Sort (3, 2, 6) = (6, 3, 2) | || | PASS

10 Metamorphic Testing Metamorphic Test Cases: {(2, 6, 3), (3, 2, 6)} Metamorphic Testing: 1. F_Sort (2, 6, 3) = (6, 3, 2) Metamorphic Testing: 1. F_Sort (2, 6, 3) = (6, 3, 2) 2. F_Sort (3, 2, 6) = (3, 6, 2) Failure. | || |

11  Conventional Program Testing and Proving  Metamorphic Testing  Semi-Proving: Verifying Metamorphic Relations  Summary. Presentation Outline

12 Semi-Proving: Verifying Metamorphic Relations  Objective: If the program does not satisfy a metamorphic relation on some inputs, locate these inputs; Otherwise prove the satisfaction of the metamorphic relation over all inputs.

13  Why called “Semi”? Proving necessary properties, which may not be sufficient for program correctness  Characteristics of Semi-Proving Multiple symbolic executions Testing and proving. Semi-Proving: Verifying Metamorphic Relations

14 double GetMid (double x1, double x2, double x3) {double mid; mid = x3; if (x2 < x3) if (x1 < x2) mid = x2; else { if (x1 < x3) mid = x1; } else if (x1 > x2) mid = x2; else if (x1 > x3) mid = x1; return mid; } double GetMid (double x1, double x2, double x3) {double mid; mid = x3; if (x2 < x3) if (x1 < x2) mid = x2; else { if (x1 < x3) mid = x1; } else if (x1 > x2) mid = x2; else if (x1 > x3) mid = x1; return mid; } Semi-Proving: Verifying Metamorphic Relations

15 SpecificationSpecification “GetMid (X, Y, Z)” returns the median of (X, Y, Z) E.g. GetMid (3, 4, 1): “3”. Semi-Proving: Verifying Metamorphic Relations

16  Verifying “GetMid” by Semi-Proving Identify a Metamorphic Relation GetMid ( X, Y, Z ) = GetMid ( permute(X, Y, Z) ) Semi-Proving: Verifying Metamorphic Relations any numbersany permutation Purpose: to verify

17  Basic concepts Transposition simple permutation that exchanges two elements (1, 2, 3) (1, 2, 3).........  1 (1, 2, 3)  (1, 3, 2).........  2  (2, 1, 3) Semi-Proving: Verifying Metamorphic Relations

18 A tuple (1, 2, 3) A permutation (2, 3, 1) (1, 2, 3) A tuple (1, 2, 3) A permutation (2, 3, 1) (1, 2, 3) (2, 3, 1)  11 (2, 1, 3)  22  Basic concepts Composition of Transpositions Semi-Proving: Verifying Metamorphic Relations

19  Result from Group Theory Any permutation of (X, Y, Z) can be achieved by compositions of transpositions (X, Z, Y) and (Y, X, Z). Semi-Proving: Verifying Metamorphic Relations

20 Semi-Proving: Verifying Metamorphic Relations  Purpose GetMid ( X, Y, Z ) = GetMid ( permute(X, Y, Z) ) Only need to verify: Any permutation. GetMid (X, Y, Z) = GetMid (X, Z, Y) GetMid (X, Y, Z) = GetMid (Y, X, Z)

21 Semi-Proving: Verifying Metamorphic Relations  Purpose GetMid ( X, Y, Z ) = GetMid ( permute(X, Y, Z) ) Only need to verify: GetMid (X, Y, Z) = GetMid (X, Z, Y) GetMid (X, Y, Z) = GetMid (Y, X, Z)

22  Global Symbolic Evaluation on GetMid (X, Y, Z) Execute all the possible paths. Semi-Proving: Verifying Metamorphic Relations

23 double GetMid (double x1, double x2, double x3) {double mid; mid = x3; if (x2 < x3) if (x1 < x2) mid = x2; else { if (x1 < x3) mid = x1; } else if (x1 > x2) mid = x2; else if (x1 > x3) mid = x1; return mid; } double GetMid (double x1, double x2, double x3) {double mid; mid = x3; if (x2 < x3) if (x1 < x2) mid = x2; else { if (x1 < x3) mid = x1; } else if (x1 > x2) mid = x2; else if (x1 > x3) mid = x1; return mid; } Semi-Proving: Verifying Metamorphic Relations

24 C1: (Y  X < Z) OR (Z < X  Y) Path Conditions C2: (X < Y < Z) OR (Z  Y < X) C3: (Y < Z  X) OR (X  Z  Y) Semi-Proving: Verifying Metamorphic Relations X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true

25 Semi-Proving: Verifying Metamorphic Relations ? GetMid (X, Z, Y) ? X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true

26 C4: (Z  X < Y) OR (Y < X  Z) C5: (X < Z < Y) OR (Y  Z < X) C6: (Z < Y  X) OR (X  Y  Z) PASS Semi-Proving: Verifying Metamorphic Relations ? GetMid (X, Z, Y) ? X when C4 is true = Z when C5 is true Y when C6 is true X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true

27 ? ? X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true X when C4 is true = Z when C5 is true Y when C6 is true  Contradiction PASS C1: (Y  X < Z) OR (Z < X  Y) & Semi-Proving: Verifying Metamorphic Relations GetMid (X, Z, Y) ? C4: (Z  X < Y) OR (Y < X  Z) C5: (X < Z < Y) OR (Y  Z < X) C6: (Z < Y  X) OR (X  Y  Z)

28 ? ? C4: (Z  X < Y) OR (Y < X  Z) C5: (X < Z < Y) OR (Y  Z < X) C6: (Z < Y  X) OR (X  Y  Z) X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true X when C4 is true = Z when C5 is true Y when C6 is true C1: (Y <= X < Z) OR (Z < X <= Y) & X=Y { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/12/3388088/slides/slide_28.jpg", "name": "28 .", "description": "C4: (Z  X < Y) OR (Y < X  Z) C5: (X < Z < Y) OR (Y  Z < X) C6: (Z < Y  X) OR (X  Y  Z) X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true X when C4 is true = Z when C5 is true Y when C6 is true C1: (Y <= X < Z) OR (Z < X <= Y) & X=Y

29 ? ? ? C4: (Z  X < Y) OR (Y < X  Z) C5: (X < Z < Y) OR (Y  Z < X) C6: (Z < Y  X) OR (X  Y  Z) X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true X when C4 is true = Z when C5 is true Y when C6 is true C1: (Y <= X < Z) OR (Z < X <= Y) & Yes. X=Y PASS X=Y { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/12/3388088/slides/slide_29.jpg", "name": "29 .", "description": "C4: (Z  X < Y) OR (Y < X  Z) C5: (X < Z < Y) OR (Y  Z < X) C6: (Z < Y  X) OR (X  Y  Z) X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true X when C4 is true = Z when C5 is true Y when C6 is true C1: (Y <= X < Z) OR (Z < X <= Y) & Yes. X=Y PASS X=Y

30 ? X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true Semi-Proving: Verifying Metamorphic Relations GetMid (X, Z, Y) verified

31 ? X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true Semi-Proving: Verifying Metamorphic Relations ConclusionConclusion GetMid (X, Z, Y)

32 ? X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true Semi-Proving: Verifying Metamorphic Relations ConclusionConclusion GetMid (X, Z, Y)

33 X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true Semi-Proving: Verifying Metamorphic Relations ConclusionConclusion GetMid (X, Z, Y)

34 X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true Semi-Proving: Verifying Metamorphic Relations ConclusionConclusion GetMid (X, Z, Y) Composition of transpositions GetMid (X, Y, Z) = GetMid ( Permute(X, Y, Z) ) GetMid (Y, X, Z) AnyAny.

35  Detecting Program Faults · Semi-Proving: Detecting Program Faults

36 double GetMid (double x1, double x2, double x3) {double mid; mid = x3; if (x2 < x3) if (x1 < x2) mid = x2; else { if (x1 < x3) mid = x1; } else if (x1 > x2) mid = x2; else if (x1 > x3) mid = x1; return mid; } double GetMid (double x1, double x2, double x3) {double mid; mid = x3; if (x2 < x3) if (x1 < x2) mid = x2; else { if (x1 < x3) mid = x1; } else if (x1 > x2) mid = x2; else if (x1 > x3) mid = x1; return mid; }

37 Verify: GetMid (X, Y, Z) = GetMid (X, Z, Y) Semi-Proving: Detecting Program Faults | || | X when Y  X < Z ? | || | Y when (Z < Y  X ) OR (Y  Z AND X  Z) AND

38 Verify: GetMid (X, Y, Z) = GetMid (X, Z, Y) Semi-Proving: Detecting Program Faults | X when Y  X < Z ? | Y when (Z < Y  X ) OR (Y  Z AND X  Z) AND  (Y=X { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/12/3388088/slides/slide_38.jpg", "name": "38 Verify: GetMid (X, Y, Z) = GetMid (X, Z, Y) Semi-Proving: Detecting Program Faults | X when Y  X < Z .", "description": "| Y when (Z < Y  X ) OR (Y  Z AND X  Z) AND  (Y=X

39 Verify: GetMid (X, Y, Z) = GetMid (X, Z, Y) Semi-Proving: Detecting Program Faults | X when Y  X < Z ? | Y when (Z < Y  X ) OR (Y  Z AND X  Z) AND  (Y=X { "@context": "http://schema.org", "@type": "ImageObject", "contentUrl": "http://images.slideplayer.com/12/3388088/slides/slide_39.jpg", "name": "39 Verify: GetMid (X, Y, Z) = GetMid (X, Z, Y) Semi-Proving: Detecting Program Faults | X when Y  X < Z .", "description": "| Y when (Z < Y  X ) OR (Y  Z AND X  Z) AND  (Y=X

40 Summary  A proving technique: all the paths  A testing technique: failure-causing inputs selected path(s)  Characteristics Metamorphic relations Multiple symbolic executions Employing global symbolic evaluation and constraint solving.

41 Questions are welcome

Download ppt "1 T.Y. Chen Swinburne University of Technology, Australia T.H. Tse and Zhiquan Zhou The University of Hong Kong Semi-Proving: an Integrated Method Based."

Similar presentations