1. Test Case Design: Strategies, Techniques & Models Robin Brennan, Senior Consultant Quality Assurance Institute

2. 2 Objectives Present test strategies, techniques and models to support test case design Particular focus on Equivalence Partitioning, Boundary Analysis and Decision Tables

3. Quality Assurance Institute3 Why Test Case Design? We cannot test everything We need test strategies to design test cases for today’s complex systems We need systematic techniques to design input/output, combinations, states, relationships and special testing The design of the test models and test cases often reveals defects from earlier phases Models support test design at any level – method, unit/class, integration, subsystem, system, user acceptance

4. Quality Assurance Institute4 Designing Test Cases Has a high probability of detecting errors Reduces the number of necessary test cases by more than one Covers additional test case possibilities A GOOD TEST CASE…

5. Quality Assurance Institute5 Designing Test Cases Three components of a test case: –Condition to be tested –Expected result –Procedure

6. Quality Assurance Institute6 Test Case Design – Strategies, Techniques & Models Equivalence Partitioning –Represents the equivalent classes of input & tests a representative of each class Boundary Analysis –Determines the boundary conditions on, above and below the edges of inputs and outputs Decision Tables –Represents the relationship between input conditions and output actions

7. Quality Assurance Institute7

8. 8 What Is Equivalence Partitioning A black box testing technique –Originally from “The Art of Software Testing” by Glenford Myers (1979) Determines categories of inputs Used for both valid and invalid variables Can be utilized at any level of testing Can be utilized early in the life cycle

9. Quality Assurance Institute9 What is Equivalence Partitioning Incorporates, through inference, additional test cases –Reveals information beyond the set of input values –Can reasonably assume that if the test uncovers an error, the other items in the same equivalence class would result in the same error

10. Quality Assurance Institute10 How to Perform Equivalence Partitioning Develop a set of conditions to be tested –Identify each input condition from requirement or design specifications –Partition each into two or more groups (equivalence classes) One Valid One or more Invalid

11. Quality Assurance Institute11 How to Perform Equivalence Partitioning (continued) Review all equivalence classes. If any components are not processed in the same manner, split the class into smaller classes Develop a minimal set of test cases –Test a representative of each category

12. Quality Assurance Institute12 Create an Equivalence Class Table External input condition on the left A column each for valid and invalid classes Unique number assigned to each equivalence class

13. Quality Assurance Institute13 Defining the Equivalence Classes 1.Input descriptions that specify a range of values: Identify one valid equivalence class Identify two invalid equivalence classes, one that lies above the range and one that lies below the range

14. Quality Assurance Institute14 Defining the Equivalence Classes 2.Input descriptions that define a number of values: Identify one valid equivalence class Identify two invalid equivalence classes outside each of the ends of the allowed number of values

15. Quality Assurance Institute15 Defining the Equivalence Classes 3.Input descriptions that define a set of values and each is handled differently (discrete input): Identify one valid equivalence class for each discrete input in the set Identify one additional invalid equivalence class for a value not included in the set

16. Quality Assurance Institute16 Defining the Equivalence Classes 4.Input descriptions that are “data typed”: Identify one valid equivalence class for the correct data type Identify at least one invalid equivalence class for a different data type

17. Quality Assurance Institute17 Defining the Equivalence Classes 5.Input descriptions with specific mandatory conditions: Identify one valid equivalence class for the specific mandatory condition Identify one invalid equivalence class for a condition that is not mandatory

18. Quality Assurance Institute18

19. Quality Assurance Institute19 Construct the Test Cases Construct as few as possible while still maintaining a reasonable level of confidence Avoid “error masking” by not combining invalid values in one test case –If more than one invalid condition is in a test case, an earlier value may result in never processing a subsequent value

20. Quality Assurance Institute20 Guidelines Write a new test case including as many valid equivalence classes as possible, until all valid conditions have been covered Write a separate test case for each invalid equivalence class

21. Quality Assurance Institute21

22. Quality Assurance Institute22

23. Quality Assurance Institute23 What Is Boundary Analysis Another black box testing technique Complimentary with Equivalence Partitioning Higher return-on-investment than random selection Can be used at any level, early in the life cycle Sounds simple but is often used ineffectively Includes both input and output domains Includes values on, above and below the edges

24. Quality Assurance Institute24 Guidelines for Input Values One or more elements are selected from the edges For continuous input, select the highest and lowest value within the range, and select the values just above and below the edges For discrete ordered sets of input, select the first and last elements in the set For a specific number of input values, select the minimum and maximum number of values, and select a number of values one below and one beyond the valid number of values

25. Quality Assurance Institute25 Guidelines for Output Values One or more elements are generated from the edges For continuous output, generate the highest and lowest value within the range, and generate the values just above and below the edges For a discrete ordered set of output, ensure the first and last elements in the set are processed For a specific number of output values, generate the minimum and maximum number of values, and generate a number of values, one below and one beyond the valid number of values

26. Quality Assurance Institute26 Boundary Analysis Examples

27. Quality Assurance Institute27

28. Quality Assurance Institute28 Decision Tables Graphical representation of conditional logic More effective than narratives for communicating information Can be used at any level Used in many software engineering methodologies as a design tool Constructed from specifications, use cases, class diagrams, behavior diagrams, source code Handles combinations of Equivalence Classes

29. Quality Assurance Institute29 Decision Tables for Testing Forces consideration of all combinations of conditions and their resultant actions Helps develop all variations avoiding omission Helps determine irrelevant test conditions Results in fewer test cases while retaining good test coverage

30. Quality Assurance Institute30 Use a Decision Table when… Distinct input conditions result in a distinct output action Conditions can be modeled by: –Mutually exclusive Boolean expressions –Binary response (Yes/No; True/False) Results do not depend on the order of input conditions Results do not depend on prior input or output

31. Quality Assurance Institute31 Decision Table Format

32. Quality Assurance Institute32 Decision Table Format - Vertical TITLE: Unique Name or Number to Identify the Table RULES: Each Unique Combination of Conditions & Actions (single column for each Rule) CONDITIONS: All Conditions to be Examined CONDITION ENTRIES: Decision Entry for the Condition (Yes/No; True/False) ACTIONS: All Actions that Can Occur for the Conditions ACTION ENTRIES: Resultant Action Based on the Conditions

33. Quality Assurance Institute33 Steps to Create Decision Tables 1.Identify the conditions and their entries 2.Identify the actions (results) 3.Determine the maximum number of Rules using the following formula:  2 n …where n is the number of conditions e.g., 3 conditions: 2 3 = 8 rules 4 conditions: 2 4 = 16 rules

34. Quality Assurance Institute34 Steps to Create Decision Tables 4.Divide the “Entries” side into the # of Rules Title 12345678 Conditions Actions

35. Quality Assurance Institute35 Steps to Create Decision Tables 5.For maximum number of rules: –The entries for condition 1 will be 50% “Yes” and 50% “No” –For condition 2, the entries will be a pattern of 25% “Yes”and 25% “No”, repeated across the table –This pattern continues for each subsequent condition…for example, condition 3 would have a pattern of 12.5% “Yes” and 12.5% “No” 6.Determine the action entry for each column

36. Quality Assurance Institute36 Steps to Create Decision Tables Title 12345678 Condition1YYYYNNNN Condition2YYNNYYNN Condition3YNYNYNYN Action1x Action2xxxxxxx

37. Quality Assurance Institute37 Steps to Create Decision Tables 7.Collapse the table by eliminating conditions that don’t need to be tested, as follows: –When two rules have the same action entry and their condition entries only vary by one answer, eliminate the condition where it varies –Next, analyze the table for redundancies and combine any rules that are now the same –Repeat this process until all unnecessary tests have been eliminated

38. Quality Assurance Institute38 Steps to Create Decision Tables 8.Confirm the accuracy of the table with the following calculation: Number of Rules = n + 1 where n is the number of conditions

39. Quality Assurance Institute39 Sample Decision Table Automobile Product Reorder Criteria: “Reorder the model if the current inventory of the model is less than 20 automobiles, the average sales is four automobiles a week or more, and the average sales profit on those sales is 7% or more”

40. Quality Assurance Institute40 Sample Decision Table Automobile Product Reorder: 12345678 Have  20 Units YYYYNNNN Sold Average of  4/week YYNNYYNN Profit  7% YNYNYNYN Reorder AutoX No ReorderXXXXXXX

41. Quality Assurance Institute41 Sample Decision Table Automobile Product Reorder: 12345678 Have  20 Units YYYYNNNN Sold Average of  4/week YYNNYYNN Profit  7% YNYNYNYN Reorder AutoX No ReorderXXXXXXX

42. Quality Assurance Institute42 Sample Decision Table Automobile Product Reorder: 12357 Have  20 Units YYYNN Sold Average of  4/week YYNYN Profit  7% YNYYY Reorder AutoX No ReorderXXXX

43. Quality Assurance Institute43 Sample Decision Table Automobile Product Reorder: 1235 Have  20 Units YYYN Sold Average of  4/week YYNY Profit  7% YNYY Reorder AutoX No ReorderXXX

44. Quality Assurance Institute44 Decision Tables to Test Cases… After creating the decision table… Verify the decision table Create one test case for each unique Rule Cross-reference with Equivalence Partitioning test cases and eliminate redundancy Use boundary-analysis to determine values for the condition entries Vary the order of the input variables Mix the sequence of test orders on different test executions

45. Quality Assurance Institute45 Test Case Design – Wrap Up Evaluate risk, complexity, frequency of use, inheritance, dependencies, variables, combinations, test time and budget Prioritize critical & complex components Use equivalence partitioning, boundary analysis, decision tables and other test design strategies and models to determine test cases Combine several test design strategies to ensure acceptable test suite coverage

46. Quality Assurance Institute46 Test Case Design Strategies Risk-based –Results of risk analysis State-based –State of the objects –Transition paths –State of the environment

47. Quality Assurance Institute47 Test Case Design Strategies Interaction-based –Interactions between components Specification-based –Functional; requirements; black box Implementation-based –Structural; code-based; white box

48. Quality Assurance Institute48 Test Case Design Strategies Use Case Scenarios –Primary flow –Alternate flows –Exceptions –Extensions CRUD –Create, Read, Update, Delete

49. Quality Assurance Institute49 In Summary Designing good test cases is hard work With a systematic approach, we can design fewer test cases while still ensuring a high probability of detecting errors A combination of strategies, techniques and models must be utilized Utilize checklists to support various types of testing

50. Quality Assurance Institute50 In Summary Use Equivalence Partitioning and Boundary Analysis for input and output variations Use Decision Tables for combinations of conditions Use State Charts for testing combinations of message sequencing & instance variables Use UML diagrams to identify relationships Use Cases are an excellent source of information for functional/system testing

51. Thank You