1. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 1 Informatics 43 Introduction to Software Engineering Lecture 9-1 May 26, 2015 Emily Navarro Duplication of course material for any commercial purpose without the explicit written permission of the professor is prohibited.

2. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 2 Announcement No discussion this week

3. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 3 Today’s Lecture Black-box (Specification-based) Testing Homework 3 Quiz 6 study guide

4. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 4 Today’s Lecture Black-box (Specification-based) Testing Homework 3 Quiz 6 study guide

5. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 5 Two Approaches Black box testing – Specification-based testing – Test cases designed, selected, and ran based on specifications – Scale: tests the higher-level system behavior – Drawback: less systematic White box testing – Structural testing – Test cases designed, selected, and ran based on structure of the code – Scale: tests the nitty-gritty – Drawbacks: need access to source

6. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 6 Black-box/Specification-Based Testing

7. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 7 Black-box/Specification-Based Testing Use specifications to derive test cases – Requirements – Design – Function signature Assume no access to source code Based on some kind of input domain Choose test cases that guarantee a wide range of coverage – Typical values – Boundary values – Special cases – Invalid input values

8. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 8 Equivalence Class Partitioning – A Systematic Approach 1.Identify the set of all possible inputs (to what is being tested) 2.Identify a basis for subdividing the set of inputs – Possible bases Size Order Structure Correctness Your creative thinking 3.Use this basis to divide the set of all possible inputs into subsets/subdomains – Subdomains may overlap slightly

9. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 9 Equivalence Class Partitioning – A Systematic Approach (Cont.) 4.From each subdomain, select [a] representative(s) to be [a] test case input(s) – One test case may suffice 5.Test for each partition – “Normal” values – Boundary or edge input values (Boundary Value Analysis) Extreme values for a partition, beyond which new partitions begin Experience has shown that many errors are made at the boundaries rather than under normal conditions

10. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 10 Example: quizAverage Input: A list of numbers – Scores must be between 0 and 1000 (inclusive) Output: a single number which is the average of the numbers on the input list, not counting the lowest number on the list.

11. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 11 Equivalence Class Partitioning with quizAverage 1.Identify the set of all possible inputs (to what is being tested) 2.Identify a basis for subdividing the set of inputs 3.Use this basis to divide the set of all possible inputs into subsets/subdomains 4.From each subdomain, select [a] representative(s) to be [a] test case input(s)

12. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 12 Equivalence Class Partitioning with quizAverage 1.Identify the set of all possible inputs (to what is being tested) The set of all lists of numbers 2.Identify a basis for subdividing the set of inputs 3.Use this basis to divide the set of all possible inputs into subsets/subdomains 4.From each subdomain, select [a] representative(s) to be [a] test case input(s)

13. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 13 1.Identify the set of all possible inputs (to what is being tested) The set of all lists of numbers 2.Identify a basis for subdividing the set of inputs length of the list, position of minimum score, number of minima, magnitude of numbers 3.Use this basis to divide the set of all possible inputs into subsets/subdomains 4.From each subdomain, select [a] representative(s) to be [a] test case input(s) Equivalence Class Partitioning with quizAverage

14. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 14 Equivalence Class Partitioning with quizAverage 1.Identify the set of all possible inputs (to what is being tested) The set of all lists of numbers 2.Identify a basis for subdividing the set of inputs length of the list, position of minimum score, number of minima, magnitude of numbers 3.Use this basis to divide the set of all possible inputs into subsets/subdomains 0 elements, 1, 2-10, 11+, first, middle, last, 1, a few, all, 0-10, 11- 100, 101-1000 4.From each subdomain, select [a] representative(s) to be [a] test case input(s)

15. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 15 Equivalence Class Partitioning with quizAverage 1.Identify the set of all possible inputs (to what is being tested) The set of all lists of numbers 2.Identify a basis for subdividing the set of inputs length of the list, position of minimum score, number of minima, magnitude of numbers 3.Use this basis to divide the set of all possible inputs into subsets/subdomains 0 elements, 1, 2-10, 11+, first, middle, last, 1, a few, all, 0-10, 11-100, 101- 1000 4.From each subdomain, select [a] representative(s) to be [a] test case input(s) [], [87.3], [90,95,85], [80,81,82,83,84,85,86,87,88,89,90,91], [80,87,88,89], [87,88,80,89], [87,88,89,80], [80,87,88,89], [87,86,86,88], [88,88,88,88], [0,4,6,5,8], [17, 43, 98, 10, 100], [101, 1000]

16. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 16 Possible Bases List length – Empty list – One element – Two or three elements – Lots of elements Input domain: float[] Basis: list length one small empty large

17. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 17 Possible Bases Position of minimum score – Smallest element first – Smallest element in middle – Smallest element last Input domain: float[] Basis: position of minima somewhere in middle first last

18. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 18 Possible Bases Number of minima – Unique minimum – A few minima – All minima Input domain: float[] Basis: number of minima all data equal 1 minimum 2 minima

19. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 19 Possible Bases Magnitude of numbers – 0-10 – 11-100 – 101-1000 Input domain: float[] Basis: magnitude of numbers 11-100 0-10 101-1000

20. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 20 Testing Matrix Test case (input) Basis (subdomain) Expected output Notes

21. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 21 quizAverage 1 Test case (input) Basis: List lengthExpected output Notes EmptyOneSmallLarge ()x0.099999! (87.3)x87.3 crashes! (90,95,85)x92.5 (80,81,82,83, 84,85,86,87, 88,89,90,91) x86.0

22. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 22 quizAverage 2 Test case (input) Position of minimumExpected output Notes FirstMiddleLast (80,87,88,89)x88.0 (87,88,80,89)x88.0 (99,98,0,97,96)x97.5 (87,88,89,80)x88.0

23. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 23 quizAverage 3 Test case (input) Number of minimaExpected output Notes OneSeveralAll (80,87,88,89)x88.0 (87,86,86,88)x87.0 (99,98,0,97,0)x73.5 (88,88,88,88)x88.0

24. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 24 quizAverage 4 Test case (input) Magnitude of NumbersExpected output Notes 0-1011-100101-1000 (0,4,6,5,8)x5.75 (17,43,98,11,100)x64.5 (99,98,11,97,50)x86 (101,1000)x1000

25. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 25 Example: Hotel Management System Consider a hotel management system that takes phone numbers as input while gathering data about the guest Imagine we want to test the “input phone number” function of the system Specification: Should give a descriptive error message if – input is less than 10 digits – input is more than 20 digits – input contains non-numeric characters What are the properties about phone numbers that we can exploit to create “valuable” partitions? – length, content (types of characters, position of invalid characters)

26. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 26 Input Phone Number 1 SubdomainsTest Case Input DataExpected Output 0{}Error 1-98987 7890987 Error 10-2078394837289 728938473829387 Success 20 +7839483728978394837289 72893847382938772893847 3829387 Error Basis: Length

27. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 27 Input Phone Number 2 SubdomainsTest Case Input DataExpected Output all numbers2345789success contains -234-5789error contains ((123456-7890error contains )123)456-7890error contains other invalid characters 1234567890#76!abcerror Basis: Types of characters

28. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 28 Input Phone Number 3 SubdomainsTest Case Input DataExpected Output beginning&1234567error middle123$4569error end1234567#error Basis: Position of invalid characters

29. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 29 Example: Gmail Imagine we are testing the login functionality of Gmail – Input: username, password – Output: login successful or error message Two users: – Mary; maryspassword – Joe; joespassword What possible bases can we use to divide our testing into partitions? – whether the username is valid (no user matching in the system, invalid characters, length) – whether the password is valid (no no user matching in the system, invalid characters, length)

30. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 30 Login SubdomainsTest Case Input DataExpected Output matches the user Mary, maryspassword Joe, joespassword success doesn’t match any user Mary, newpassword Joe, anothernewpassword error matches another user Mary, joespassword Joe, maryspassword error Basis: whether the password matches the user

31. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 31 Example: Room Scheduler System Imagine we are testing a classroom scheduler program that handles M-F scheduling for five classrooms Room capacities – Room A: 500 – Room B: 300 – Room C: 100 – Room D: 50 – Room E: 20 All classes are 1-hour long, once per week, and can be scheduled between 8am-10pm

32. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 32 Example: Room Scheduler System Input – The current schedule – The number of students in the class to be scheduled – The desired time of the class to be scheduled Output – A list of available rooms that can hold the number of students, ordered from most appropriate (number of students is as close as possible to room capacity without going over) to least appropriate

33. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 33 Example: Room Scheduler System Example – Input: {Current schedule: Room A: M-F: 8-11am, 2-4pm Room B: T-F: 9-10am, 5-8pm Room C: F: 10am-3pm Room D: M-F: 8am-10pm Room E: M-F: 10am-5pm, 8pm-10pm; Num students: 73 Desired time: W 5-6pm} – Expected output: {Room C, Room A} Room capacities Room A: 500 Room B: 300 Room C: 100 Room D: 50 Room E: 20

34. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 34 Example: Room Scheduler System What possible bases can we use to divide our testing into partitions?

35. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 35 Schedule Room 1 SubdomainsTest Case Input DataExpected Output Basis:

36. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 36 Schedule Room 2 SubdomainsTest Case Input DataExpected Output Basis:

37. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 37 Today’s Lecture Black-box (Specification-based) Testing Homework 3 Quiz 6 study guide

38. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 38 Homework 3 You will be designing test cases for ZotMyHealth using a black-box/specification-based approach – You will be provided with a specification upon which to base your testing – Your document will mainly consist of testing matrices Homework 3 is posted Deadline is Tuesday, June 2, 11:55pm to EEE

39. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 39 Today’s Lecture Black-box (Specification-based) Testing Homework 3 Quiz 6 study guide

40. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 40 Quiz 6 Study Guide Testing (Lectures 8-2 and 9-1) – Validation/verification – How do we know when we are done? – Test-driven development – Error, fault, failure – Testing process model – Testing goals – Difference between white-box and black-box testing – Levels of testing (unit, functional/integration, system/acceptance) – Oracles – Test drivers/stubs

41. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 41 Next Time White-box (Structural) Testing