MUTATION TESTING

Mutation testing is a technique that focuses on measuring the adequacy (quality) of test data (or test cases). Modify a program by introducing a single small change to the code A modified program is called mutant A mutant is said to be killed when the execution of test case cause it to fail. The mutant is considered to be dead A mutant is an equivalent to the given program if it always produce the same output as the original program A mutant is called killable or stubborn, if the existing set of test cases is insufficient to kill it A mutation score for a set of test cases is the percentage of non- equivalent mutants killed by the test suite The test suite is said to be mutation-adequate if its mutation score is 100%

the following program P that finds rank corresponding to the first time the maximum value appears in the array 1. main(argc,argv) 2. int argc, r, i; 3. char *argv[]; 4. { r = 1; 5. for i = 2 to 3 do 6. if (atoi(argv[i]) > atoi(argv[r])) r = i; 7. printf("Value of the rank is %d \n", r); 8. exit(0); }

TestCases Test case 1: Input: Output: Value of the rank is 3 Test case 2: Input: Output: Values of the rank is 2 Test case 3: Input: Output: Value of the rank is 1

1. Mutant 1: Change line 5 to for i = 1 to 3 do 2. Mutant 2: Change line 6 to if (i > atoi(argv[r])) r = i; 3. Mutant 3: Change line 6 to if (atoi(argv[i]) >= atoi(argv[r])) r = i; 4. Mutant 4: Change line 6 to if (atoi(argv[r]) > atoi(argv[r])) r = i; let us mutate the program P. We can start with the following changes:

Mutants 1 and 3: The programs will completely pass the test suite. In other words, mutants 1 and 3 are not killed. Mutant 2: The program will fail test case 2. Mutant 4: The program will fail test case 1 and test case 2.

calculate the mutation score Mutation score = 100×D/(N −E), where ▫D: is the dead mutants, ▫N: the total number of mutants, ▫E: the number of equivalent mutants. Assuming that mutants 1 and 3 are nonequivalent. mutation score is 50%. It is low because we assumed that mutants 1 and 3 are nonequivalent to the original program we need to add new test cases to kill these two mutants

1 and 3 are equivalent mutants or those are killable??!! analyze mutant is an equivalent mutant. add a fourth test case as follows: Test case 4: Input: P : the output “Value of the rank is 1” mutant 3 : the output “Value of the rank is 2.” Thus, this test data kills mutant 3, which give us a mutation score of 100%.

Steps to build a robust test suite Step 1: Begin with a program P and a set of test cases T known to be correct. Step 2: Run each test case in T against the program P. ▫If it fails (o/p incorrect) P must be modified and restarted. Else, go to step 3 Step 3: Create a set of mutants {Pi }, each differing from P by a simple, syntactically correct modification of P.

Step 4: Execute each test case in T against each mutant Pi. If the o/p is differ  the mutant Pi is considered incorrect and is said to be killed by the test case If Pi produces exactly the same results: ▫P and Pi are equivalent ▫Pi is killable (new test cases must be created to kill it)

Step 5: Calculate the mutation score for the set of test cases T. Mutation score = 100×D/(N −E), Step 6: If the estimated mutation adequacy of T in step 5 is not sufficiently high, then design a new test case that distinguishes Pi from P, add the new test case to T, and go to step 2.