1. Black Box Unit Testing 1209

2. What is black-box testing? Unit (code, module) seen as a black box No access to the internal or logical structure Determine if given input produces expected output. Input Output

3. Black Box Testing Test set is derived from requirements Goal is to cover the input space Lots of approaches to describing input space: –Equivalence Classes –Boundary Value Analysis –Decision Tables –State Transitions –Use Cases –...

4. Advantage and Disadvantage Advantage: Disadvantage: Need to define an efficient strategy to limit number of test cases

5. General Black Box Process Analyze requirements Select valid and invalid inputs Determine expected outputs Construct tests Run tests Compare actual outputs to expected outputs

6. Equivalence Relations: Reflexive Symmetric Transitive

7. Equivalence classes: Basic Strategy Partition the input into equivalence classes Test a subset from each class

8. Basic strategy: Example:fact(n): if n =200 error if 0<=n<=20, exact value if 20<n<200, approximate value within.1%

9. Simple Example: Suppose you are building an airline reservation system. A traveler can be a child, an adult, or a senior. The price depends on the type of traveler. The seat reservation does not depend on the type of traveler. How many test cases are required for the reservation component and the billing component?

10. Finding equivalence classes: Identify restrictions for inputs and outputs in the specification If there is a continuous numerical domain, create one valid and two or three invalid classes (above, below, and NaN) If a number of values is required, create one valid and two invalid classes

11. Boundary Values: Programs that fail at interior elements of a class usually fail at the boundaries too. Test the boundaries.

12. Hints If a domain is a restricted set, check the boundaries. e.g., D=[1,10], test 0, 1, 10, 11 –It may be possible to test the boundaries of outputs, also. For ordered sets, check the first and last elements For complex data structures, the empty list, full lists, the zero array, and the null pointer should be tested

13. Boundary Value: in class Determine the boundary values for US Postal Service ZIP codes Determine the boundary values for a 15- character last name entry.

14. Decision Tables Construct a table (to help organize the testing) Identify each rule or condition in the system that depends on some input For each input to one of these rules, list the combinations of inputs and the expected results

15. Decision Table Example Test CaseC1 Student C2 Senior Result Discount? 1 2 3 4 Theater ticket prices are discounted for senior citizens and students.

16. Pairwise Testing

17. Problem 1 From Lee Copeland, A Practitioner’s Guide to Software Test Design, Artech House Publishers, 2004. A web site must operate correctly with different browsers: IE 5, IE 6, and IE 7; Mozilla 1.1; Opera 7; and FireFox 2, 3, and 4. It must work using RealPlayer, MediaPlayer, or no plugin. It needs to run under Windows ME, NT, 2000, XP, and Vista, and 7.0 It needs to accept pages from IIS, Apache and WebLogic running on Windows NT, 2000, and Linux servers

18. Problem 2 A bank is ready to test a data processing system Customer types –Gold (i.e., normal) –Platinum (i.e., important) –Business –Non profits Account types –Checking –Savings –Mortgages –Consumer loans –Commercial loans States (with different rules): CA, NV, UT, ID, AZ, NM From Lee Copeland, A Practitioner’s Guide to Software Test Design, Artech House Publishers, 2004.

19. When given a large number of combinations: (options improve …) Give up and don’t test Test all combinations … miss targets, delay product launch, and go out of business Choose one or two cases

20. Orthogonal Arrays A 2-D array with the property –All pairwise combinations occur in every pair of columns Example: consider 3 variables (columns) with {a,b}, {1,2}, and (α,β) 123 11Aα 21Bβ 32Aβ 42Bα

21. Orthogonal Array Example 123 11Aα 21Bβ 32Aβ 42Bα Look at each pair of columns (1 and 2),( 1 and 3), and (2 and 3) Does each of the 4 pairings appear in each? (Yes, of course!)

22. Pairwise Testing Algorithm 1.Identify variables 2.Determine choices for each variable 3.Locate an orthogonal array 4.Map test cases to the orthogonal array 5.Construct tests

23. Example using Bank 1.Identify variables 2.Determine choices for each variable 3.Locate an orthogonal array 4.Map test cases to the orthogonal array 5.Construct tests

24. Locate an orthogonal array (look up on web) 111 122 133 144 212 221 234 243 313 324 331 342 414 423 432 441 551 512 523 534 652 611 624 633 153 254 35 1 45 2 54 1 64 2

25. Map Test Cases (30) CACheckGold CASavePlat CAMortBusi CAConsNonP NVCheckPlat NVSaveGold NVMortNonP NVConsBusi UTCheckBusi UTSaveNonP UTMortGold UTConsPlat IDCheckNonP IDSaveBusi IDMortPlat IDConsGold AZCommGold AZCheckPlat AZSaveBusi AZMortNonP NMCommPlat NMCheckGold NMSaveNonP NMMortBusi CACommBusi NVCommNonP UTCommGold IDCommPlat AZConsGold NMConsPlat

26. Pairwise Summary When the combination is large, test all pairs, not all combinations Studies indicate that most defects are single or double-mode effects –Can be found in pairing –Few defects require more than two modes Orthogonal arrays have the pairwise property and can be found or generated

27. State Transition Testing

28. Build STD of system or component Cover the STD

29. Example Res Made Paid Cancelled Non Pay Ticketed Cancelled Cust Used Cust Order/start timer Payment made Ticket Printed Time Expires Ticket Delivered Cancel/ refund Cancel

30. Use Case Testing