1. Mutation Testing
Presented by
Sharath Kumar Garlapati
Vinesh Thummala.

2. Learning Objectives What is Mutation testing History of Mutation
Traditional Process of Mutation testing
Kinds of Mutations
Mutation Operators
An Example for Mutation Testing
Pros and Cons
Conclusion

3. What is Mutation testing?
Mutation Testing is a testing technique that focuses on measuring the adequacy of test cases
Mutation Testing is NOT a testing strategy like path or data- flow testing. It does not outline test data selection criteria.

4. History of Mutation Proposed by Richard Lipton as a student in 1971.
Developed and published by DeMillo, Lipton and Sayward in late 70’s.
Implemented as a testing tool by Timothy Budd in 1980 from Yale University.

5. Mutation Process
[2]

6. Mutation Process
The process, given program P and test suite T, is as follows:
Systematically apply mutations to program P to obtain a sequence,P1, P2,... Pn of mutants of P. Each mutant is derived by applying a single mutation operation to P.
Run the test suite T on each of the mutants T is said to kill
mutant Pj, if it detects an error.
If k out of n mutants killed, the adequacy of T is measured by the quotient k/n implies T is mutation adequate, if k = n.

7. Kinds of Mutations
Value Mutation - Involves changing the values of constants or parameters(by adding or subtracting values etc).
Decision Mutation- Involves in modifying conditions to reflect potential slips(replacing a > by a < in a comparison).
Statement Mutation - Deleting or Swapping the lines of codes.

8. Mutation Operators Statement deletion
Boolean expression changes (true and false)
Arithmetic operation changes (+ and *, - and /)
Boolean operator changes (> and >=, == and <=)
Declared in the same scope variables changes (variable types should be the same)

9. Class level mutation operators
Polymorphic
Method Overloading
Method Overriding/Hiding in Inheritance
Field Variable Hiding in Inheritance
Information Hiding (Access Control)
Static/Dynamic States of Objects
Exception Handling

10. Polymorphic mutations
class T { … }
interface K { … }
class S extends T implements K { … }
class U extends S { … }
The original code:
S s = new S();
Mutants:
T s = new S();
K s = new S();
S s = new T();
S s = new U();

11. Method overloading mutations
public LogMessage(int level, String logKey, Object[]inserts){…}
public LogMessage(int level, String logKey, Object insert) {…}
Mutant:
public LogMessage(String logKey, int level, Object[]inserts{…}

12. Method Overriding/Hiding in Inheritance
public class Animal { public void speak(){..} public void bite(){..} } public class Cat extends Animal { public void speak() {..} Mutant: //public void speak() {..}

13. Field Variable Hiding in Inheritance
public class Animal { public String kind; public int legCount; } public class Cat extends Animal { Mutant: //public int legCount;

14. Information Hiding (Access Control)
The original code:
protected Address address;
Mutants:
public Address address;
private Address address;
Address address; //default (no access modifier defined)

15. Static/Dynamic States of Objects
The original code:
public static int VALUE = 100;
private String s;
Mutants:
public int VALUE = 100; //static is removed.
private static String s; //static is added.

16. Exception Handling
The original code: String formatMsg(LogMessage msg){ … try { … } catch(MissingResourceException mre){… } … Mutant: String formatMsg(LogMessage msg) throws MissingResourceException { … //try-catch block removed

17. Weak Mutation
Compares the internal states of the mutant and original program immediately after execution of the mutated portion of the program.
Requires much less computing power to ensure that the test suite satisfies weak mutation testing.

18. Strong Mutation
When strong mutation testing is performed, the values produced by mutated code and changes (or not) the output of the module or subroutine.
Requires much more computing power.

19. Automation of Mutation Testing
List of tools available –
Ninja Turtles- .NET mutation testing tool
Mutagenesis- PHP mutation testing framework
Heckle- Ruby Mutation Testing Tool
Jester- Mutation Testing Tool for Java

20. Advantages
It is a powerful approach to attain high testing coverage of the source program.
This testing is capable of comprehensively testing the mutant program.
Mutation testing brings a good level of error detection to the software developer.
Customers are benefited from this testing by getting a more reliable and stable system.

21. DisAdvantages Costly and Time consuming. Many test cases are required.
Requires a lot of testing before a dependable data is obtained.
Needs an automated tool to reduce testing time.
Not an applicable method for black box testing.
Very complicated to use without an automated tool.

22. Conclusion Not just a testing method, more of an analytical method.
An effective program for improving the quality of testing software but its widespread use is prevented by the difficulties encountered in using them.
It would be very helpful to the developers of complex software, if testing methods could be improved

23. Reference
[1] Anguswamy,R., Frakes,W., A Comparison of Database Fault Detection Capabilities Using
Mutation Testing, Proceedings of ESEM,12,(Sept,2012,NY,USA),
[2] Praphamontripong,U., Offutt,J., Applying Mutation Testing to Web Applications, Proceeding
of ICSTW ,10 ,(Apr,2010, Paris,
[3] Schuler,D., Zeller,A., Efficient Mutation Testing for Java, Proceedings of ESEC/FSE, 06
(Aug , NY, USA),
[4] Madeyski, L., Nguyen, Q., Problems of Mutation Testing and Higher Order Mutation Testing ,
Proceedings of ICCSAMA, 12,(2014, Switzerland),
[5] Khurshid, S., Marinov, D., Zhang, L., Zhang,Lu., Regression Mutation Testing, Proceeding
of ISSTA ,12,(July.2012, MN, USA),

24. Thank You