1. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 1 Informatics 43 Introduction to Software Engineering Lecture 7-1 November 10, 2015 Emily Navarro Duplication of course material for any commercial purpose without the explicit written permission of the professor is prohibited.

2. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 2 Announcement No discussion this week

3. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 3 Today’s Lecture Black-box (Specification-based) Testing Homework 2

4. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 4 Today’s Lecture Black-box (Specification-based) Testing Homework 2

5. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 5 Two Approaches Black box testing – Specification-based testing – Test cases designed, selected, and ran based on specifications – Scale: tests the higher-level system behavior – Drawback: less systematic White box testing – Structural testing – Test cases designed, selected, and ran based on structure of the code – Scale: tests the nitty-gritty – Drawbacks: need access to source

6. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 6 Black-box/Specification-Based Testing

7. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 7 Black-box/Specification-Based Testing Use specifications to derive test cases – Requirements – Design – Function signature Assume no access to source code Based on some kind of input domain Choose test cases that guarantee a wide range of coverage – Typical values – Boundary values – Special cases – Invalid input values

8. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 8 Equivalence Class Partitioning – A Systematic Approach 1.Identify the set of all possible inputs (to what is being tested) 2.Identify a basis for subdividing the set of inputs – Possible bases Size Structure Correctness Your creative thinking 3.Use this basis to divide the set of all possible inputs into subsets/subdomains

9. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 9 Equivalence Class Partitioning – A Systematic Approach (Cont.) 4.From each subdomain, select [a] representative(s) to be [a] test case input(s) – One test case may suffice 5.Test for each partition – “Normal” values – Boundary or edge input values (Boundary Value Analysis) Extreme values for a partition, beyond which new partitions begin Experience has shown that many errors are made at the boundaries rather than under normal conditions

10. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 10 Example: quizAverage Input: A list of numbers – Scores must be between 0 and 1000 (inclusive) Output: a single number which is the average of the numbers on the input list, not counting the lowest number on the list.

11. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 11 Equivalence Class Partitioning with quizAverage 1.Identify the set of all possible inputs (to what is being tested) 2.Identify a basis for subdividing the set of inputs 3.Use this basis to divide the set of all possible inputs into subsets/subdomains 4.From each subdomain, select [a] representative(s) to be [a] test case input(s)

12. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 12 Equivalence Class Partitioning with quizAverage 1.Identify the set of all possible inputs (to what is being tested) The set of all lists of numbers 2.Identify a basis for subdividing the set of inputs 3.Use this basis to divide the set of all possible inputs into subsets/subdomains 4.From each subdomain, select [a] representative(s) to be [a] test case input(s)

13. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 13 1.Identify the set of all possible inputs (to what is being tested) The set of all lists of numbers 2.Identify a basis for subdividing the set of inputs length of the list, position of minimum score, number of minima, magnitude of numbers 3.Use this basis to divide the set of all possible inputs into subsets/subdomains 4.From each subdomain, select [a] representative(s) to be [a] test case input(s) Equivalence Class Partitioning with quizAverage

14. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 14 Equivalence Class Partitioning with quizAverage 1.Identify the set of all possible inputs (to what is being tested) The set of all lists of numbers 2.Identify a basis for subdividing the set of inputs length of the list, position of minimum score, number of minima, magnitude of numbers 3.Use this basis to divide the set of all possible inputs into subsets/subdomains 0 elements, 1, 2-10, 11+, first, middle, last, 1, a few, all, 0-10, 11- 100, 101-1000 4.From each subdomain, select [a] representative(s) to be [a] test case input(s)

15. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 15 Equivalence Class Partitioning with quizAverage 1.Identify the set of all possible inputs (to what is being tested) The set of all lists of numbers 2.Identify a basis for subdividing the set of inputs length of the list, position of minimum score, number of minima, magnitude of numbers 3.Use this basis to divide the set of all possible inputs into subsets/subdomains 0 elements, 1, 2-10, 11+, first, middle, last, 1, a few, all, 0-10, 11-100, 101- 1000 4.From each subdomain, select [a] representative(s) to be [a] test case input(s) [], [87.3], [90,95,85], [80,81,82,83,84,85,86,87,88,89,90,91], [80,87,88,89], [87,88,80,89], [87,88,89,80], [80,87,88,89], [87,86,86,88], [88,88,88,88], [0,4,6,5,8], [17, 43, 98, 10, 100], [101, 1000]

16. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 16 Possible Bases List length – Empty list – One element – Small (two to ten elements) – Large (eleven+ elements) Input domain: float[] Basis: list length one large empty small

17. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 17 Possible Bases Position of minimum score – Smallest element first – Smallest element in middle – Smallest element last Input domain: float[] Basis: position of minima somewhere in middle first last

18. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 18 Possible Bases Number of minima – One minimum – A few minima – All minima Input domain: float[] Basis: number of minima all data equal 1 minimum A few minima

19. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 19 Possible Bases Magnitude of numbers – Small (0-10) – Medium (11-100) – Large (101-1000) Input domain: float[] Basis: magnitude of numbers 11-100 0-10 101-1000

20. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 20 Testing Matrix Test case (input) SubdomainExpected output Notes Basis

21. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 21 quizAverage 1 Test case (input) SubdomainsExpected output Notes EmptyOneSmallLarge ()x0.099999! (87.3)x87.3 crashes! (90,95,85)x92.5 (80,81,82,83, 84,85,86,87, 88,89,90,91) x86.0 Basis: List length

22. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 22 quizAverage 2 Test case (input) SubdomainsExpected output Notes FirstMiddleLast (80,87,88,89)x88.0 (87,88,80,89)x88.0 (99,98,0,97,96)x97.5 (87,88,89,80)x88.0 Basis: Position of minimum

23. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 23 quizAverage 3 Test case (input) SubdomainsExpected output Notes OneSeveralAll (80,87,88,89)x88.0 (87,86,86,88)x87.0 (99,98,0,97,0)x73.5 (88,88,88,88)x88.0 Basis: Number of minima

24. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 24 quizAverage 4 Test case (input) SubdomainsExpected output Notes 0-1011-100101-1000 (0,4,6,5,8)x5.75 (17,43,98,11,100)x64.5 (99,98,11,97,50)x86 (101,1000)x1000 Basis: Magnitude of numbers

25. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 25 Example: Hotel Management System Consider a hotel management system that takes phone numbers as input while gathering data about the guest Imagine we want to test the “input phone number” function of the system Specification: Should give a descriptive error message if – input is less than 10 digits – input is more than 20 digits – input contains non-numeric characters What are the properties about phone numbers that we can exploit to create “valuable” partitions? – Length, content (types of characters, position of invalid characters)

26. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 26 Input Phone Number 1 SubdomainsTest Case Input DataExpected Output 0emptyerror 1-9143 4839274 483-9274 Error 10-203829384958 483958473059481 (788) 835-9999 Success Error 21+38493820349584940598Error Basis: Length

27. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 27 Input Phone Number 2 SubdomainsTest Case Input DataExpected Output All numbers7483928374 333 Success Error Contains () as the only special characters (748)3928374Error Contains – as the only special characters 748-392-8374Error Contains other special characters (but not () or -) 8293847389A839BError Contains both () and - (748)392-8374Error Basis: Types of characters

28. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 28 Input Phone Number 3 SubdomainsTest Case Input DataExpected Output beginning(8888987483847error middle8888*89878error end8389473847+error Basis:position of invalid characters

29. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 29 Example: Email Imagine we are testing the login functionality of an email program – Input: username, password – Output: login successful or error message Two users: – Mary; maryspassword – Joe; joespassword What possible bases can we use to divide our testing into partitions?

30. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 30 Login SubdomainsTest Case Input DataExpected Output Basis:

31. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 31 Example: Room Scheduler System Imagine we are testing a classroom scheduler program that handles M-F scheduling for five classrooms Room capacities – Room A: 500 – Room B: 300 – Room C: 100 – Room D: 50 – Room E: 20 All classes are 1-hour long, once per week, and can be scheduled between 8am-10pm

32. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 32 Example: Room Scheduler System Input – The current schedule – The number of students in the class to be scheduled – The desired time of the class to be scheduled Output – A list of available rooms that can hold the number of students, ordered from most appropriate (number of students is as close as possible to room capacity without going over) to least appropriate

33. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 33 Example: Room Scheduler System Example – Input: {Current schedule: Room A: M-F: 8-11am, 2-4pm Room B: T-F: 9-10am, 5-8pm Room C: F: 10am-3pm Room D: M-F: 8am-10pm Room E: M-F: 10am-5pm, 8pm-10pm; Num students: 73 Desired time: W 5-6pm} – Expected output: {Room C, Room A} Room capacities Room A: 500 Room B: 300 Room C: 100 Room D: 50 Room E: 20

34. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 34 Example: Room Scheduler System What possible bases can we use to divide our testing into partitions?

35. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 35 Schedule Room 1 SubdomainsTest Case Input DataExpected Output Basis:

36. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 36 Schedule Room 2 SubdomainsTest Case Input DataExpected Output Basis:

37. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 37 Today’s Lecture Black-box (Specification-based) Testing Homework 2

38. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 38 Homework 2 You will be designing test cases for PrivatePractice using a black-box/specification-based approach – You will be provided with a specification upon which to base your testing – Your document will mainly consist of testing matrices Homework 2 is posted Deadline is Wednesday, November 18, 11:55pm to EEE

39. Department of Informatics, UC Irvine SDCL Collaboration Laboratory Software Design and sdcl.ics.uci.edu 39 Next Time White-box (Structural) Testing