Download presentation

Presentation is loading. Please wait.

Published byJanessa Brent Modified about 1 year ago

1
1 T.Y. Chen Swinburne University of Technology, Australia T.H. Tse and Zhiquan Zhou The University of Hong Kong Semi-Proving: an Integrated Method Based on Global Symbolic Evaluation and Metamorphic Testing (speaker)

2
2 Presentation Outline Conventional Program Testing and Proving Metamorphic Testing Our method: Semi-Proving Summary.

3
3 Conventional Program Testing and Proving Metamorphic Testing Our method: Semi-Proving Summary. Presentation Outline

4
4 Conventional Program Testing and Proving Given a bijective function f ; A Program: F_Sort (a 1, a 2,..., a n ), n 2 Output: (a 1 ’, a 2 ’,..., a n ’), such that 1. (a 1 ’, a 2 ’,..., a n ’) is a permutation of (a 1, a 2,..., a n ) 2. f (a 1 ’) f (a 2 ’) ... f (a n ’). Given a bijective function f ; A Program: F_Sort (a 1, a 2,..., a n ), n 2 Output: (a 1 ’, a 2 ’,..., a n ’), such that 1. (a 1 ’, a 2 ’,..., a n ’) is a permutation of (a 1, a 2,..., a n ) 2. f (a 1 ’) f (a 2 ’) ... f (a n ’).

5
5 Conventional Program Testing and Proving Testing 1. Design test cases: e.g. (2, 6, 3) for n=3 2. Run: F_Sort (2, 6, 3) = (6, 3, 2) 3. Check: f (6) < f (3) < f (2) ? 1. Design test cases: e.g. (2, 6, 3) for n=3 2. Run: F_Sort (2, 6, 3) = (6, 3, 2) 3. Check: f (6) < f (3) < f (2) ?

6
6 Conventional Program Testing and Proving Proving correctness 1. F_Sort terminates for any valid input; 2. The output is correct. 1. F_Sort terminates for any valid input; 2. The output is correct.

7
7 Conventional Program Testing and Proving Proving properties F_Sort (a 1, a 2,..., a n ) = (a 1 ’, a 2 ’,..., a n ’) Permutation.

8
8 Metamorphic Testing Metamorphic Testing Employing relationships between different executions Fact: different permutations will produce same output F_Sort (a 1, a 2, a 3 ) Fact: different permutations will produce same output F_Sort (a 1, a 2, a 3 ) F_Sort (a 3, a 1, a 2 ) = “ Metamorphic Relation ” ·

9
9 Metamorphic Testing Metamorphic Test Cases: {(2, 6, 3), (3, 2, 6)} Metamorphic Testing: 1. F_Sort (2, 6, 3) = (6, 3, 2) Metamorphic Testing: 1. F_Sort (2, 6, 3) = (6, 3, 2) No matter whether an oracle is available or not; Very useful when the oracle cannot be found. 2. F_Sort (3, 2, 6) = (6, 3, 2) | || | PASS

10
10 Metamorphic Testing Metamorphic Test Cases: {(2, 6, 3), (3, 2, 6)} Metamorphic Testing: 1. F_Sort (2, 6, 3) = (6, 3, 2) Metamorphic Testing: 1. F_Sort (2, 6, 3) = (6, 3, 2) 2. F_Sort (3, 2, 6) = (3, 6, 2) Failure. | || |

11
11 Conventional Program Testing and Proving Metamorphic Testing Semi-Proving: Verifying Metamorphic Relations Summary. Presentation Outline

12
12 Semi-Proving: Verifying Metamorphic Relations Objective: If the program does not satisfy a metamorphic relation on some inputs, locate these inputs; Otherwise prove the satisfaction of the metamorphic relation over all inputs.

13
13 Why called “Semi”? Proving necessary properties, which may not be sufficient for program correctness Characteristics of Semi-Proving Multiple symbolic executions Testing and proving. Semi-Proving: Verifying Metamorphic Relations

14
14 double GetMid (double x1, double x2, double x3) {double mid; mid = x3; if (x2 < x3) if (x1 < x2) mid = x2; else { if (x1 < x3) mid = x1; } else if (x1 > x2) mid = x2; else if (x1 > x3) mid = x1; return mid; } double GetMid (double x1, double x2, double x3) {double mid; mid = x3; if (x2 < x3) if (x1 < x2) mid = x2; else { if (x1 < x3) mid = x1; } else if (x1 > x2) mid = x2; else if (x1 > x3) mid = x1; return mid; } Semi-Proving: Verifying Metamorphic Relations

15
15 SpecificationSpecification “GetMid (X, Y, Z)” returns the median of (X, Y, Z) E.g. GetMid (3, 4, 1): “3”. Semi-Proving: Verifying Metamorphic Relations

16
16 Verifying “GetMid” by Semi-Proving Identify a Metamorphic Relation GetMid ( X, Y, Z ) = GetMid ( permute(X, Y, Z) ) Semi-Proving: Verifying Metamorphic Relations any numbersany permutation Purpose: to verify

17
17 Basic concepts Transposition simple permutation that exchanges two elements (1, 2, 3) (1, 2, 3)......... 1 (1, 2, 3) (1, 3, 2)......... 2 (2, 1, 3) Semi-Proving: Verifying Metamorphic Relations

18
18 A tuple (1, 2, 3) A permutation (2, 3, 1) (1, 2, 3) A tuple (1, 2, 3) A permutation (2, 3, 1) (1, 2, 3) (2, 3, 1) 11 (2, 1, 3) 22 Basic concepts Composition of Transpositions Semi-Proving: Verifying Metamorphic Relations

19
19 Result from Group Theory Any permutation of (X, Y, Z) can be achieved by compositions of transpositions (X, Z, Y) and (Y, X, Z). Semi-Proving: Verifying Metamorphic Relations

20
20 Semi-Proving: Verifying Metamorphic Relations Purpose GetMid ( X, Y, Z ) = GetMid ( permute(X, Y, Z) ) Only need to verify: Any permutation. GetMid (X, Y, Z) = GetMid (X, Z, Y) GetMid (X, Y, Z) = GetMid (Y, X, Z)

21
21 Semi-Proving: Verifying Metamorphic Relations Purpose GetMid ( X, Y, Z ) = GetMid ( permute(X, Y, Z) ) Only need to verify: GetMid (X, Y, Z) = GetMid (X, Z, Y) GetMid (X, Y, Z) = GetMid (Y, X, Z)

22
22 Global Symbolic Evaluation on GetMid (X, Y, Z) Execute all the possible paths. Semi-Proving: Verifying Metamorphic Relations

23
23 double GetMid (double x1, double x2, double x3) {double mid; mid = x3; if (x2 < x3) if (x1 < x2) mid = x2; else { if (x1 < x3) mid = x1; } else if (x1 > x2) mid = x2; else if (x1 > x3) mid = x1; return mid; } double GetMid (double x1, double x2, double x3) {double mid; mid = x3; if (x2 < x3) if (x1 < x2) mid = x2; else { if (x1 < x3) mid = x1; } else if (x1 > x2) mid = x2; else if (x1 > x3) mid = x1; return mid; } Semi-Proving: Verifying Metamorphic Relations

24
24 C1: (Y X < Z) OR (Z < X Y) Path Conditions C2: (X < Y < Z) OR (Z Y < X) C3: (Y < Z X) OR (X Z Y) Semi-Proving: Verifying Metamorphic Relations X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true

25
25 Semi-Proving: Verifying Metamorphic Relations ? GetMid (X, Z, Y) ? X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true

26
26 C4: (Z X < Y) OR (Y < X Z) C5: (X < Z < Y) OR (Y Z < X) C6: (Z < Y X) OR (X Y Z) PASS Semi-Proving: Verifying Metamorphic Relations ? GetMid (X, Z, Y) ? X when C4 is true = Z when C5 is true Y when C6 is true X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true

27
27 ? ? X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true X when C4 is true = Z when C5 is true Y when C6 is true Contradiction PASS C1: (Y X < Z) OR (Z < X Y) & Semi-Proving: Verifying Metamorphic Relations GetMid (X, Z, Y) ? C4: (Z X < Y) OR (Y < X Z) C5: (X < Z < Y) OR (Y Z < X) C6: (Z < Y X) OR (X Y Z)

28
28 ? ? C4: (Z X < Y) OR (Y < X Z) C5: (X < Z < Y) OR (Y Z < X) C6: (Z < Y X) OR (X Y Z) X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true X when C4 is true = Z when C5 is true Y when C6 is true C1: (Y <= X < Z) OR (Z < X <= Y) & X=Y

29
29 ? ? ? C4: (Z X < Y) OR (Y < X Z) C5: (X < Z < Y) OR (Y Z < X) C6: (Z < Y X) OR (X Y Z) X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true X when C4 is true = Z when C5 is true Y when C6 is true C1: (Y <= X < Z) OR (Z < X <= Y) & Yes. X=Y PASS X=Y

30
30 ? X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true Semi-Proving: Verifying Metamorphic Relations GetMid (X, Z, Y) verified

31
31 ? X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true Semi-Proving: Verifying Metamorphic Relations ConclusionConclusion GetMid (X, Z, Y)

32
32 ? X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true Semi-Proving: Verifying Metamorphic Relations ConclusionConclusion GetMid (X, Z, Y)

33
33 X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true Semi-Proving: Verifying Metamorphic Relations ConclusionConclusion GetMid (X, Z, Y)

34
34 X when C1 is true GetMid (X, Y, Z) =Y when C2 is true Z when C3 is true Semi-Proving: Verifying Metamorphic Relations ConclusionConclusion GetMid (X, Z, Y) Composition of transpositions GetMid (X, Y, Z) = GetMid ( Permute(X, Y, Z) ) GetMid (Y, X, Z) AnyAny.

35
35 Detecting Program Faults · Semi-Proving: Detecting Program Faults

36
36 double GetMid (double x1, double x2, double x3) {double mid; mid = x3; if (x2 < x3) if (x1 < x2) mid = x2; else { if (x1 < x3) mid = x1; } else if (x1 > x2) mid = x2; else if (x1 > x3) mid = x1; return mid; } double GetMid (double x1, double x2, double x3) {double mid; mid = x3; if (x2 < x3) if (x1 < x2) mid = x2; else { if (x1 < x3) mid = x1; } else if (x1 > x2) mid = x2; else if (x1 > x3) mid = x1; return mid; }

37
37 Verify: GetMid (X, Y, Z) = GetMid (X, Z, Y) Semi-Proving: Detecting Program Faults | || | X when Y X < Z ? | || | Y when (Z < Y X ) OR (Y Z AND X Z) AND

38
38 Verify: GetMid (X, Y, Z) = GetMid (X, Z, Y) Semi-Proving: Detecting Program Faults | X when Y X < Z ? | Y when (Z < Y X ) OR (Y Z AND X Z) AND (Y=X

39
39 Verify: GetMid (X, Y, Z) = GetMid (X, Z, Y) Semi-Proving: Detecting Program Faults | X when Y X < Z ? | Y when (Z < Y X ) OR (Y Z AND X Z) AND (Y=X

40
40 Summary A proving technique: all the paths A testing technique: failure-causing inputs selected path(s) Characteristics Metamorphic relations Multiple symbolic executions Employing global symbolic evaluation and constraint solving.

41
41 Questions are welcome

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google