1. Frank Xu, Ph.D. Gannon University Automated Test-Input Generation Xu, W., Ding, T., Wang, H., Xu. D., Mining Test Oracles for Test Inputs Generated from Java Bytecode, Proc. of the 37th Annual International Computer Software & Applications Conference, pp. 27- 32, Kyoto, Japan, July 2013 Mining Decision Trees as Test Oracles for Java Bytecode (Extended version of conference paper), Accepted by Journal of Systems and Software

2. About Me – Frank Xu Education Ph.D. in Software Engineering, North Dakota State University M.S. in Computer Science, Towson University B.S. in Computer Science, Minor in Math, Southeast Missouri State University Working Experience GE Transportation, 2008- present, Consultant Gannon University, 2008- present, Assistant Professor of Software Engineering, Director of Keystone Software Development Institute University VA –Wise, 2007- 2008, Assistant Professor of Software Engineering Swanson Health Products, 2005 ~ 2007, Sr. Web Programmer Analyst Volt Information Science Inc., 2004 ~ 2005, Software Engineer (Web)

3. Teaching Source: Student Evaluation Report

4. Research Source: Google scholar: http://scholar.google.com/citations?user=9_I4ZUgAAAAJ&hl=en

5. A UTOMATED T EST -I NPUT G ENERATION Introduction Software testing Test automation Test Inputs How to Generate Test Inputs Simplifying Java Code Applying rules Empirical Study/Demo Conclusions

6. I NTRODUCTION

7. Exercise Implementing a method to solve Triangle problem

8. What is Triangle Problem?

9. What is Method? Is a function or a service to complete a task A method that determines the maximum of two numbers. A method that sorts a list of names A method that opens a file from the file system Method Invoked by a method call Returns a result to calling method (caller) Similar to a boss (caller) asking a worker (called method) to complete a task

10. Method square returns int that result stores Method square returns the square of y y is the parameter of method square  2003 Prentice Hall, Inc.

11. How to test Triangle? String getTriangleType (int a, int b, int c){ if((a<b+c) && (b<a+c) && (c<a+b)){ if (a==b && b==c) return “Equilateral ”; else if (a!=b && a!=c &&b!=c) return “Scalene ”; else return “Isosceles” ; } else return “NotATriangle “; } String getTriangleType (int a, int b, int c){ if((a<b+c) && (b<a+c) && (c<a+b)){ if (a==b && b==c) return “Equilateral ”; else if (a!=b && a!=c &&b!=c) return “Scalene ”; else return “Isosceles” ; } else return “NotATriangle “; }

12. if... else Control Flow if (condition) statement1; else statement2; if (a==b && b==c) return “Equilateral ”; else if (a!=b && a!=c &&b!=c) return “Scalene ”; else return “Isosceles” ;

13. Control Flow Diagram

14. S UMMARY : T EST T RIANGLE S TEPS Source Code Control Flow Diagram Paths (based on coverage) assertEquals(“Isosceles ”, getTriangleType(7,7,9)) Junit Test cases assertEquals(“Isosceles ”, getTriangleType(6,6,8)) ….. Step 1 Step 2 Step 3

15. H OW TO A UTO -G ENERATE T ESTING I NPUTS ? assertEquals (“Isosceles ”, getTriangleType(7,7,9) assertEquals (“Isosceles ”, getTriangleType(6,6,8) ….. ?

16. S OLUTION Randomly generate inputs (5,7,6) for Isosceles Adjust input values a=5, b=7, a==b

17. R EVISIT : T RIANGLE P ROBLEM

18. Java is Complex Statement contains comparison and expression a <b+c (Java) Condition (a<b+c) && (b<a+c) && (c<a+b)

19. J AVA S IMPLER V ERSION Simplify Statement a <b+c (Java) [1] $i3=i1+i2 and [2] i0>=$i3 (Jimple) Simplify condition (a<b+c) && (b<a+c) && (c<a+b) (Java) Jimple if (a<b+c) { if (b<a+c) { if(c<a+b) … }}} www.sable.mcgill.ca/soot/

20. IDPath 1 2 3,,,..,,, 8 9 10 11 12 IDabc 1777 21173 38 19 41173 522 9 ………… … 1333 45 14523052 153147 16272822 Generate inputs Path generation Generate CFG

21. H OW TO G ENERATE T EST I NPUTS IDabc 1777 21173 38 19 41173 522 9 6304330 72213 8 1620 … ………… … 1333 45 14523052 153147 16272822 IDPath 1 2 3,,, 4 5 6 7 8 9 10 11 12

22. Search an input that make predicate [5]:i0>=$i3 to true a>=b+c (NotATriangle) Challenge: backtracking $i3 to input variables Recall $i3=i1+i2 Solution:

23. Apply Rules for Generating Test Inputs 10 4 7

24. IDPredicateExpected Evaluation Outcomes Advising Rules 1. i0 > i1(i0 > i1) = true(i0 ↑, i1) (i0, i1 ↓ ) 1. (i0 > i1) = false(i0 ↓,b) (i0, i1 ↑ ) 1. i0 == i1(i0==i1) = true(i0 ↓ D, i1)( i0, i1 ↑ D) 1. (i0== i1)= false(i0 ↑,i1) (i0, i1 ↑ ) (i0 ↓,i1) (i0, i1 ↓ ) 1. i2 = i0 + i1i2 ↑ (i0 ↑, i1) (i0, i1 ↑ ) 1. i2 ↓ (i0 ↓, i1) (i0, i1 ↓ ) 1. i2 = i0 - i1i2 ↑ (i0 ↑, i1) (i0, i1 ↓ ) 1. i2 ↓ (i0 ↓, i1)( i0, i1 ↑ ) 1. i2 = i0 * i1 (i0>0, i1>0) i2 ↑ (i0 ↑, i1) (i0, i1 ↑ ) 1. i2 ↓ (i0 ↓, i1) (i0, i1 ↓ ) 1. i2 = i0 / i1 (i0>0, i0 > 0) i2 ↑ (i0 ↑, i1) (i0, i1 ↓ ) 1. i2 ↓ (i0- ↓ i1)( i0, i1 ↑ ).. 1. s0>s1(s0 >s1) = true(s0[k] ↑, s1) (s0, s1[l] ↓ ) 1. (s0 > s1) = false(s0[k] ↓,s2) (s0, s1[l] ↑ )

25. E MPIRICAL S TUDY

26. T HREE S TUDY S UBJECTS Line of CodeNumber of Predicates JavaJimpleJava Jimple (Allow duplications) Attributes (No duplication) Triangle2227388 Next Date4851919 Vending Machine8268214110

27. G OAL OF E MPIRICAL S TUDIES What is the performance of the proposed approach?

28. IDGoalExecution Time to Generate 100 Inputs (ms) Triangle 1 Equilateral255 2 Isosceles167 3 Scalene146 4 NotATriangle121 Next Date 1 Normal day && 1st month 96 2 Last day && 1st month 168 3 Normal day 167 4 Last day of a normal month 198 5 Normal day in Dec 114 6 Last day of a year 172 7 Normal day in Feb 153 8 leap year 738 9 Non- leap year 779 10 Incorrect days of a leap year 912 Vending Machine (Change, Dollar, Cents, Juice, Beer) 1(1,1,0,0.1)534 2(1,1,0,0,0)553 3(1,0,1,1,0)523 4(1,0,1,0,1)495 5(0,1,0,1,0)277 6(0,1,0,0,0)339 7(0,0,1,1,1)285 8(0,0,1,0,1)310

29. Demo http://perceval.gannon.edu/xu001/research/GannonJVM/ Path 1: Equilateral Path 5, 7, 8: Isosceles Path 6: Scalene Path 10, 11, 12: not a triangle

30. Key Points Understand requirements before implementation Test your code Auto test your code if possible Auto generate test inputs

31. Thanks

32. Future Research Directions Requirements Engineering & Natural language Process Generating UML diagrams, e.g., Use case, Class diagram Validating SRS Deriving test cases from SRS Software Design & Social Networks Analysis Utilizing SSA for analyzing communication diagram, class diagram, and sequence diagram for improving the quality of the software Software Implementation & Big Data Mining repository for software quality assurance using Hadoop Software Testing & Mobile/Cloud Application Testing mobile applications and distributed applications