Download presentation

Presentation is loading. Please wait.

1
**Introduction to White-box Testing**

Frank Xu, Ph.D. Department of Computer and Information Science Gannon University

2
**Motivation Software problems and related faults: Airbus A380 (2006)**

Source: The top 10 IT disasters of all time: Software problems and related faults: Airbus A380 (2006) suffers from incompatible software issues At least two years behind schedule Ariane 5 rocket (1996) Data conversion (64-bit to 16-bit format) Cost $500M Therac 25 - Radiation therapy machine (1993) Overdosing patients during radiation Cost 6 lives

3
Testing One of the practical methods commonly used to detect the presence of errors (failures) in a computer program is to test it for a set of inputs. The output is correct? Our program I1, I2, I3, …, In, … Expected results = =? Actual results Black-box testing “Inputs” White-box testing

4
Exercise Implementing a method to solve Triangle problem

5
**What is Triangle Problem?**

6
**How to test it? String getTriangleType (int a, int b, int c){**

if((a<b+c) && (b<a+c) && (c<a+b)){ if (a==b && b==c) return “Equilateral ”; else if (a!=b && a!=c &&b!=c) return “Scalene ”; else return “Isosceles” ; } return “NotATriangle “;

7
**Steps for White-box Testing**

assertEquals(“equalateral”, triangle.getTriangleType(7,7,7)) Step 1 Step 2 assertEquals(“Isosceles ”, triangle.getTriangleType(6,6,8)) ….. Source Code Control Flow Diagram Paths (based on coverage) Junit Test cases

8
**if ... else Control Flow if (condition) statement1; else statement2;**

if (a==b && b==c) return “Equilateral ”; else if (a!=b && a!=c &&b!=c) return “Scalene ”; else return “Isosceles” ; if (condition) statement1; else statement2;

9
Control Flow Diagram

10
**Branch Coverage Method**

Identify all the decisions Count all the branches from each of the decisions Pick the minimum number of paths that will cover all the branches from the decisions.

11
**How many paths? For decision coverage McCabe’s Cyclomatic number**

# of binary decisions + 1 Input for path Equilateral: (7,7,7) assertEquals (“Equilateral”, triangle.getTriType(7,7,7))

12
**Related to Cyber Security**

Include Software security Network security Data security Common approach for software security Testing for security E.g., testing for web security Test cases are needed for all testing

13
**Auto-Generate Test Cases is Challenging**

Challenging 1:inputs generation Challenging 2: Oracle Problem assertEquals (“Equilateral”, triangle.getTriType(7,7,7)) ? assertEquals (“Isosceles ”, triangle.getTriType(6,6,8)) …..

14
**Mining Decision Trees as Test Oracles for Java Bytecode**

Xu, W., Ding, T., Xu, D., Rule-based Test Input Generation From Bytecode, Proc. of the 8th International Conference on Software Security and Reliability, pp , San Francisco, CA, June 2014. Xu, W., Ding, T., Wang, H., Xu. D., Mining Test Oracles for Test Inputs Generated from Java Bytecode, Proc. of the 37th Annual International Computer Software & Applications Conference, pp , Kyoto, Japan, July 2013

15
**Our Approach Rule-based search method to generate inputs**

Seed value adjust seed values based on rules (5,7,8) for Isosceles Adjust input values a==b (7,7,8) (5,5,8) Using heuristic model for test oracle A new data mining approach to build a heuristic behavioral model (in the form of decision tree)

16
**Revisit: Triangle Problem**

17
**Java is Complex Statement Condition contains comparison and expression**

a <b+c (Java) Condition (a<b+c) && (b<a+c) && (c<a+b)

18
**Jimple Java simpler version Simplify Statement Simplify condition**

Intermediate representation of bytecode Simplify Statement a <b+c (Java) [1] $d=b+c and [2] a <$d (Jimple) Simplify condition (a<b+c) && (b<a+c) && (c<a+b) (Java) Jimple if (a<b+c) { if (b<a+c) { if(c<a+b) … }}}

19
**Training data Validation data Input Generator Input Generator (7,7,7)**

Path 1 [1]→[2]→[3]→[4]→[5] f >[6]→[7] f >[8]→[9] f >[10] f >[11] f >[12] 2 [1]→[2]→[3]→[4]→[5] f >[6]→[7] f >[8]→[9] f >[10] t >[13] f >[14] f >[15] t >[17] 3 ,,, .. 8 [1]→[2]→[3]→[4]→[5] f >[6]→[7] f >[8]→[9] f >[10] t >[13] f >[14] f >[15] f >[16] 9 [1]→[2]→[3]→[4]→[5] f >[6]→[7] f >[8]→[9] f >[10] f >[11] t >[13] f >[14] f >[15] f >[16] 10 [1]→[2]→[3]→[4]→[5] f >[6]→[7] f >[8]→[9] t >[18] 11 [1]→[2]→[3]→[4]→[5] f >[6]→[7] t >[18] 12 [1]→[2]→[3]→[4]→[5] t >[18] CFG Generator Path generator Input Generator Input Generator ID a b c 1 7 2 11 3 8 19 4 5 22 9 … 13 33 45 14 52 30 15 31 47 16 27 28 Validation data (7,7,7) Model miner Equilateral Training data

20
Input Generator

21
**Path: [1]→[2]→[3]→[4]→[5] t >[18]**

Search an input that make predicate [5]:i0>=$i3 to true a>=b+c (NotATriangle)

22
**Apply Rules to a Predicate Tree for Generating Test Inputs**

10 4 7

23
**Expected Evaluation Outcomes**

ID Predicate Expected Evaluation Outcomes Advising Rules i0 > i1 (i0 > i1) = true (i0 ↑, i1) (i0, i1↓) (i0 > i1) = false (i0 ↓,b) (i0, i1↑) i0 == i1 (i0==i1) = true (i0 ↓D, i1)( i0, i1↑D) (i0== i1)= false (i0 ↑,i1) (i0, i1↑) (i0↓,i1) (i0, i1↓) i2 = i0 + i1 i2 ↑ (i0 ↑, i1) (i0, i1↑) i2 ↓ (i0 ↓, i1) (i0, i1↓) i2 = i0 - i1 (i0 ↑, i1) (i0, i1↓) (i0 ↓, i1)( i0, i1↑) i2 = i0 * i1 (i0>0, i1>0) (i0↑, i1) (i0, i1↑) (i0↓, i1) (i0, i1↓) i2 = i0 / i1 (i0>0, i0 > 0) (i0↑, i1) (i0, i1↓) (i0-↓ i1)( i0, i1↑) .. s0>s1 (s0 >s1) = true (s0[k] ↑, s1) (s0, s1[l] ↓) (s0 > s1) = false (s0[k] ↓,s2) (s0, s1[l] ↑)

24
Model Miner

25
**Jimple Predicates and Attributes of Triangle Program**

Source Code i0 > = $i3 a > = b + c i1 > = $i4 b > = a + c i2 > = $i5 c > = a + b i0 != i1 a != b i1 != i2 b != c i0 == i1 a == b i0 == i2 a == c i1 == i2 b == c Input (a=7,b=7,c=7) f t For a given input, predicates produce a set of T or F values

26
**Covert Inputs Using Attributes**

Test input ID a b c a>=b+c b>=a+c c>=a+b a!=b b!=c a==b a==c b==c Output 1 7 f t 2 11 3 4 8 19 … 13 33 45 14 52 30 15 31 47 16 27 28 22

27
**The key idea of the algorithm is to Tool**

Test input ID a b c a>=b+c b>=a+c c>=a+b a!=b b!=c a==b a==c b==c Output 1 7 f t 2 11 3 4 8 19 … 13 33 45 14 52 30 15 31 47 16 27 28 22 C4.5 mining algorithm The key idea of the algorithm is to calculate the highest normalized information gain of attributes and then build a decision node that splits on the attributes Tool Weka 3:

28
Empirical Study

29
**Three Study Subjects Unit Under Testing Line of Code**

Number of Predicates Java Jimple (Allow duplications) Attributes (No duplication) Triangle 22 27 3 8 Next Date 48 51 9 19 Vending Machine 82 68 21 41 10

30
**Goal of Empirical Studies**

Measure fault detection capability # mutants killed /#mutants *100%

31
**Measure fault detection capability: Process**

Mutation Operator Examples Category ID Type Original Replaced Arithmetic Operations 1 Arithmetic Operator Replacement a + b a - b 2 Arithmetic Operator Insertion b + c -b + c Relations 3 Relational Operator Replacement a != b a == b Conditions 4 Conditional Operator Replacement (a==b) && (b==c) (a==b) || (b==c) Constants 5 Constant Value Modification s = a s = b Return Values 6 Return Value Modification return s return s’ Step 1: Implant mutants Step 2: Build a decision tree model Step 3: Find mismatches Find possible causes Step 4: Calculate fault detectability Insert bug Create mutant and insert mutants Two possible causes Found bugs assertEquals(“Equilateral”, new Trianlge(7,7,7).getTriType()) Model is not correct assertEquals (“Isosceles”, new Trianlge(7,7,7).getTriType()) Faulty version Find mismatches

32
**Results of fault detection evaluation**

Mutation ID # of Mutants # of Tests Executed Oracle Results # Mutants Discovered Faults in Models S D U Triangle 51 123 48 3 NextDate 54 94 95 109 15 16 18 2 Vending 45 465 Total 150 886 142 8 statement coverage (S), decision coverage (D), and un-reduced decision coverage (U),

33
**Conclusions The first attempt to mine decision tree models**

from auto-generated test inputs based on static analysis of Java bytecode Our empirical study indicates that using the mined test oracles, average 94.67% mutants are killed by the generated test inputs. Future plan Empirical study

34
Thanks

35
Related Work Lo et al. (Lo, Cheng, Han, Khoo, & Sun, 2009), Milea et al. (Milea, Khoo, Lo, & Pop, 2012) mines a set of discriminative features capturing repetitive series of events from program execution traces. These features are then used to train a classier to detect failures. Bowring et al. (Bowring, Rehg, & Harrold, 2004) models program executions as Markov models, and a clustering method for Markov models that aggregates multiple program executions into effective behavior classifiers. (Pacheco & Ernst, 2005) Pacheco and Ernst build an operational model from observations of the software running properly. The operation model includes object invariants and properties. The object invariants are the conditions hold on entry and exit of all public methods. Our approach generates and classifies inputs based on the internal structure of the UUT. Briand (Briand, 2008) has proposed the use of machine learning techniques - including decision trees - for the test oracle problem. The decision tree model he has proposed is manually built from software requirements.

36
**Future Research Direction**

Requirements Engineering & Natural language Process Generating UML diagrams, e.g., Use case, Class diagram Validating SRS Deriving test cases from SRS Software Design & Social Networks Analysis Utilizing SSA for analyzing communication diagram, class diagram, and sequence diagram for improving the quality of the software Software Implementation & Big Data Mining repository for software quality assurance using Hadoop Software Testing & Mobile/Cloud Application Testing mobile applications and distributed applications

37
Test Oracle Overview

38
Control Flow Diagram

39
**Build Variable Dependency Tree (VDT)**

40
**Bio – Frank Xu Education Working Experience**

Ph.D. in Software Engineering North Dakota State University M.S. in Computer Science Towson University B.S. in Computer Science Southeast Missouri State University Working Experience GE Transportation, present, Consultant of Locomotive Remote Diagnostics Service Center Gannon University, Present, Assistant/Associate Professor of Software Engineering, Director of Keystone Software Development Institute University VA –Wise, , Assistant Professor of Software Engineering Swanson Health Products, 2005 ~ 2007, Sr. Programmer Analyst Volt Information Science Inc., 2004 ~ 2005, Software Engineer

41
**Teaching Software engineer in UML Requirement engineering**

Software Design Software Quality Assurance Software Maintenance Object-Oriented Modeling Personal Software Process Advance Programming in Java Mobile Application Programming Multi-Tiered Systems Dynamic Web Programming Operating System Discrete Math

42
**Source: Student Evaluation Report**

Teaching Evaluation Source: Student Evaluation Report

43
Research

44
**Publications and Citations**

Year 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 Total Conference 3 2 1 4 24 Journal 8 5 6 32 Source: Google scholar:

Similar presentations

OK

1 Phase Testing. Janice Regan, 2008 2 For each group of units Overview of Implementation phase Create Class Skeletons Define Implementation Plan (+ determine.

1 Phase Testing. Janice Regan, 2008 2 For each group of units Overview of Implementation phase Create Class Skeletons Define Implementation Plan (+ determine.

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Ppt on acid base indicators Ppt on pathophysiology of obesity Three dimensional viewing ppt on ipad Ppt on depth first search vs breadth Ppt on networking related topics on typhoons Ppt on two stroke petrol engine Ppt on measures to control environmental pollution Ppt on chromosomes and chromatids Ppt on nuclear family and joint family definition Ppt on water softening techniques definition