1. White Box Testing

2. Agenda White-box vs Black-box Program Flow Controls White-box Test Methods Exercises Complexity Q&A

3. What is White-box Testing? Looking at the internal structure of a program and deriving test cases based on the logic or control flow. Test cases can be designed to reach every branch in the code and to exercise each condition Typically done during unit testing Also known as: –Structural Testing –Glass-Box Testing

4. What is Black-box Testing? Looking at the program from an external point of view and deriving test cases based on the specification. The only criteria upon which the program is judged is if it produces the correct output for a given input.

5. Why Do Both? Black-box Impossible to write a test case for every possible set of inputs and outputs Some of the code may not be reachable without extraordinary measures Specifications are not always complete White-box Does not address the question of whether or not the program matches the specification Does not tell you if all of the functionality has been implemented Does not discover missing program logic

6. Basic Program Flow Controls IF IF-Then-Else FOR While Do-While Case

7. IF Diagram

8. IF-THEN-ELSE Diagram

9. FOR or WHILE Diagram

10. DO-WHILE Diagram

11. CASE Diagram

12. Example Code Fragment

13. Example Control Flow Graph Source: The Art of Software Testing – Glenford Myers

14. Exercise #1 int main (int argc, char *argv[]) { /* Process CTRL-C Interrupts */ signal(SIGINT,catcher); if (validate_command_line(argc)) return(1); if (job_initialize(argv)) return(1); processTestList(); /* Process all testCases in TestList */ displayResults(); fprintf(stdout,"Test Complete\n"); job_termination(); return(0); } /* main */ Can you diagram this code?

15. Did You Get Something Like This?

16. Exercise #2 /* Attempt Statusreadback -log SRB data to logFile */ int process_srb(void) { int srb_count = 0; do { srb_count = read_printer(srb_in); } while (!srb_count); fprintf(logFile,"%s\n",srb_in); return(srb_count); } /* process_srb */ /* Write String to Printer via Parallel or Serial port */ void write_printer (char *outputline) { if (strstr(printertype,"PAR") != NULL) { bwrite_parST(printerport,outputline); } else if (strstr(printertype,"SER") != NULL) { bwwrite_serial(printerport,outputline,strlen(outputline)); } else if (strstr(printertype, "FILE") !=NULL) { fprintf(printerFile,"%s",outputline); } } /* write_printer */ Can you diagram this code?

17. Did You Get Something Like This?

18. White-box Test Methods Statement Coverage Decision/Branch Coverage Condition Coverage Decision/Condition Coverage Path Coverage

19. Example Code Fragment If ((A>1) & (B=0)) then Do; X=X/A; END; If ((A==2) | (X>1)) then Do; X=X+1; END; Source: The Art of Software Testing – Glenford Myers

20. Statement Coverage Exercise all statements at least once How many test cases?  A=2 and B=0 (ace) Source: The Art of Software Testing – Glenford Myers

21. Decision/Branch Coverage Each decision has a true and a false outcome at least once How many test cases?  A=2 and B=0 (ace)  A=1 and X=1 (abd) Source: The Art of Software Testing – Glenford Myers

22. Condition Coverage Each condition in a decision takes on all possible outcomes at least once Conditions: A>1, B=0, A=2, X>1 How many test cases?  A=2, B=0, and X=4 (ace)  A=1, B=1, and X=1 (abd) Source: The Art of Software Testing – Glenford Myers

23. Decision/Condition Coverage Each condition in a decision takes on all possible outcomes at least once, and each decision takes on all possible outcomes at least once How many test cases?  A=2, B=0, and X=4 (ace)  A=1, B=1, and X=1 (abd) What about these?  A=1, B=0, and X=3  A=2, B=1, and X=1 (abe)

24. Multiple Condition Coverage Exercise all possible combinations of condition outcomes in each decision Conditions: Source: The Art of Software Testing – Glenford Myers A>1, B=0 A>1, B<>0 A<=1, B=0 A 0 A=2, X>1 A=2, X<=1 A<>2, X>1 A<>2, X<=1

25. Multiple Condition Coverage How many test cases?  A=2, B=0, X=4  A=2, B=1, X=1  A=1, B=0, X=2  A=1, B=1, X=1 Source: The Art of Software Testing – Glenford Myers (ace) (abe) (abd)

26. Path Coverage Every unique path through the program is executed at least once How many test cases?  A=2, B=0, X=4 (ace)  A=2, B=1, X=1 (abe)  A=3, B=0, X=1 (acd)  A=1, B=1, X=1 (abd) Source: The Art of Software Testing – Glenford Myers

27. McCabe’s Cyclomatic Complexity Software metric Developed by Tom McCabe (circa 1976) Directly measures the number of linearly independent paths through a program’s source code, taking into account the various decision points Independent of implementation language Source: Wikipedia

28. Calculating Complexity

30. How Complex Should Code Be? <10: Simple module, not much risk 10-20: More Complex; moderate risk 20-50: Complex; high risk >50: Untestable; extremely high risk Source: Carnegie Mellon Software Engineering Institute

31. Complexity Caveats As code is broken into smaller modules to decrease cyclomatic complexity, structural complexity increases Some modules may have high complexity but are very easy to comprehend and easy to test High complexity numbers are only an indicator of something to investigate