1. © 2006 Fraunhofer CESE1 MC/DC in a nutshell Christopher Ackermann

2. © 2006 Fraunhofer CESE2 What is MC/DC? Unrelated to AC/DC! MC/DC stands for Modified Condition / Decision Coverage Some kind of Predicate Coverage technique Some terms: – Condition: Leaf level Boolean expression. – Decision: Controls the program flow. Main idea: Each condition must be shown to independently affect the outcome of a decision, i.e. the outcome of a decision changes as a result of changing a single condition.

3. © 2006 Fraunhofer CESE3 1. Requirement The decision has taken all possible outcomes at least once. We could also say we cover both the true and the false branch (like we do in Branch Testing). If ((a) & (b) | (c)) then… truefalse

4. © 2006 Fraunhofer CESE4 2. Requirement Every condition in the decision has taken all possible outcomes at least once. Aims to cover compound conditions (like Condition/Decision Coverage) If ((a) & (b) | (c)) then… truefalsetruefalsetruefalse

5. © 2006 Fraunhofer CESE5 3. Requirement Every condition in the decision independently affects the decision’s outcome. Not like anything we have talked about, so let’s elaborate… If ((a) & (b) | (c)) then… truefalse truefalse Change the value of each condition individually while keeping all other conditions constant.

6. © 2006 Fraunhofer CESE6 Creating MC/DC test cases If (A and B) then… (1) create truth table for conditions. (2) Extend truth table so that it indicated which test cases can be used to show the independence of each condition.

7. © 2006 Fraunhofer CESE7 Creating test cases cont’d Show independence of A: – Take 1 + 3 Show independence of B: – Take 1 + 2 Resulting test cases are – 1 + 2 + 3 – (T, T) + (T, F) + (F, T)

8. © 2006 Fraunhofer CESE8 More advanced example If (A and (B or C)) then… Note: We want to determine the MINIMAL set of test cases Here: {2,3,4,6} {2,3,4,7} Non-minimal set is: {1,2,3,4,5}

9. © 2006 Fraunhofer CESE9 Where does it fit in? The MC/DC criterion is much stronger than the condition/decision coverage criterion, but the number of test cases to achieve the MC/DC criterions still varies linearly with the number of conditions n in the decisions. – Much more complete coverage than condition/decision coverage, but – at the same time it is not terribly costly in terms of number of test cases.

10. © 2006 Fraunhofer CESE10 Object Code Coverage In some situations the compiler performs a translation of each conditions into a single decision (conditional jump) in the object code. The minimal test set for condition/decision coverage does not provide decision coverage of the generated object code. ({1,8},{2,7},{3,6}) MC/DC ensures decision coverage of the underlying object code. ({2,3,4,6} or {2,3,4,7})

11. © 2006 Fraunhofer CESE11 Advantages of MC/DC Linear growth in required tests: – For an expression with N uncoupled conditions, the modified condition/decision criterion can be met with a minimum of N + 1 tests by varying exactly one condition of each of the first N tests. Object code coverage: – The MC/DC criterion ensures a much higher level of decision coverage of the object code than either decision or condition/decision coverage. Operand sensitivity: – Extraneous operands that cannot independently affect the outcome are found during construction of the test set Sensitivity to non-equivalent functions:

12. © 2006 Fraunhofer CESE12 Efficiency For a given function of N operands and any M distinct tests, the probability P (N,M) of detecting an error in an incorrect implementation of a Boolean expression with N conditions is given by Probability of finding an error increases as N increases!

13. © 2006 Fraunhofer CESE13 Efficiency cont’d This suggests that breaking down compound conditions has a negative effect on the results of MC/DC based testing.

14. © 2006 Fraunhofer CESE14 Argument in the software industry Federal Aviation Administration’s requirement that test suites be MC/DC adequate (DO-178B). In the avionics domain, complex Boolean expressions are much more common.

15. © 2006 Fraunhofer CESE15 Argument in the software industry Federal Aviation Administration’s requirement that test suites be MC/DC adequate. Argument – Too expensive – “For example, one of our industrial partners reports that for one of its products of about 20,000 lines of code, the MC/DC- adequate test suite requires seven weeks to run.” Counter Argument – Significantly more effective – A recent empirical study performed during the real testing of the attitude-control software for the HETE-2 (High Energy Transient Explorer) found that the test cases generated to satisfy the MC/DC coverage requirement detected important errors not detectable by functional testing.

16. © 2006 Fraunhofer CESE16 Conclusion Critics lead to research work that tries to reduce testing effort through – Test suite reduction – Test suite prioritization Question: What is the difference between MC/DC and the technique discussed in the Predicat3e Paper? Paper: “Much of the benefit of the MC/DC criterion lies not so much in the specific tests selected as in the requirement that at least N + 1 tests are executed” – Question: Why not just randomly pick N+1 test cases then?

17. © 2006 Fraunhofer CESE17 References J.J. Chilenski and S.P. Miller, “Applicability of Modified Condition/Decision Coverage to Software Testing,” Software Eng. J., vol. 9, no. 5, pp. 193-200, 1994. D. Richard Kuhn. “Fault classes and error detection capability of specification-based testing” ACM Transactions on Software Engineering and Methodology, 8(4):411--424, October 1999. Jones, J. and Harrold, M. “Test-Suite Reduction and Prioritization for Modified Condition/Decision Coverage”, Proceedings of the IEEE International Conference on Software Maintenance ( ICSM'01), Florence, Italy, 7-9 November 2001, pp. 92--101. Dupuy, A. and Leveson, N. “An empirical evaluation of the MC/DC coverage criterion on the HETE-2 satellite software”, Proceedings of the Digital Aviation Systems Conference (DASC), Philadelphia, USA, October 2000.