Dynamic Black-Box Testing Part 1 What is dynamic black-box testing? How to reduce the number of test cases using: Equivalence partitioning Boundary value analysis

Test Design Technique Exhaustive testing is impossible So.. select a subset of all possible tests Improper selection can result in testing too much, testing too little, or testing the wrong things We need some intelligent thought process to guide our selection called test design techniques Test design technique: Procedure used to derive and/or select test cases

Test Design Technique A test design technique basically helps us to select a good set of tests from the total number of all possible tests for a given system Each individual technique is good at finding particular types of defect

Categories of Test Design Techniques Static technique Dynamic technique – Specification-based (black-box, also known as behavioral techniques) – Structure-based (white-box or structural techniques) – Experience- based

Dynamic Testing The software is tested by executing it on computer

Test design Test design is the act of creating and writing test suites for testing a software. Test suites: A set of several test cases for a component or system under test, where the post condition of one test is often used as the precondition for the next one. Test cases can be documented as described in the IEEE 829 Standard for Test Documentation 1.First, identify exact and detailed specific input. 2.Then, determine what the expected result of entering that input would be and document it as part of the test case. – Expected results include information displayed on a screen in response to an input or changes in data and status or other consequences (e.g. print)

Black-box Testing Specification-based techniques or behavioral testing In black-box testing the tester is concentrating on what the software does, not how it does it View the software as a black-box with inputs and outputs. Used for both: – Functional testing: concerned with whether the system does its features or functions – Non-functional testing: concerned with examining how well the system does its features (e.g. performance, usability, portability, maintainability)

Black-box Testing Appropriate at all levels of testing (component testing through to acceptance testing) There are four specification-based or black- box technique: – Equivalence partitioning – Boundary value analysis – Decision tables – State transition testing

Test-to-Pass & Test-to-Fail There are two fundamental approaches to testing software: – Test-to-pass: assure only that the software minimally works Apply simple and straightforward test cases Always run the test-to-pass cases first – Test-to-fail: purpose is breaking the software Try to force the error messages

Equivalence Partitioning (EP) It can be applied at any level of testing Divide (i.e. to partition) a set of test conditions into groups or sets that can be considered the same (i.e. the system should handle them equivalently) We need to test only one condition from each partition Assuming that all the conditions in one partition will be treated in the same way by the software

Equivalence Partitioning (EP) When looking for equivalence partitions, think about ways to group: – similar inputs – similar outputs – similar operation of the software. These groups are your equivalence partitions.

Equivalence Partitioning (EP) Example 1 A savings account in a bank has a different rate of interest depending on the balance in the account. 3% rate of interest is given if the balance in the account is in the range of $0 to $100, 5% rate of interest is given if the balance in the account is in the range of $100 to $1000, and 7% rate of interest is given if the balance in the account is $1000 and above.

Equivalence Partitioning (EP) Example 1 In order to test the software that calculates the interest due, we can identify the ranges of balance values that earn the different rates of interest.

Equivalence Partitioning (EP) Example 1 We would initially identify three valid equivalence partitions and one invalid Tester should not only test what is in the specification, but should also think about things that haven’t been specified – Non-numeric input is also an invalid partition (e.g. the letter ‘a’)

Equivalence Partitioning (EP) Example 1: So, calculate the interest on balances of: – $ – $50.00 – $ – $ Note: using techniques such as this makes testing both more effective and more efficient

Equivalence Partitioning (EP) Example 1: Now, let’s design the test cases Test cases are not ready yet, we need to compute the expected output Go back to the specification to find out how to calculate the interest rate Test Case IDInput

Equivalence Partitioning (EP) Example 1: interest due = balance * interest rate Test cases are ready Test Case IDInputExpected output 1-10Error message

Boundary Value Analysis Testing at the boundaries between partitions Think about the following characteristics – First/Last – Min/Max – Empty/Full Test: – First–1/Last+1 – Less than Empty/More than Full – Min–1/Max+1

Boundary Value Analysis Boundary value: An input value or output value which is on the edge of an equivalence partition or at the smallest incremental distance on either side of an edge

Boundary Value Analysis Example 1, interest rates for banks Six tests for boundary values – -$0.01 (an invalid boundary value) – $0.00 – $ – $ – $ – $

Boundary Value Analysis We have open boundary in the last partition Go back to specification or experience-based approach – field that holds the account balance figure may be only six figures plus two decimal figures. This would give a maximum account balance of $

Boundary Value Analysis Example 2: A printer that has an input option of the number of copies to be made, from 1 to 99. – To apply boundary value analysis: Take the minimum and maximum (boundary) values from the valid partition (1 and 99 in this case) Together with the first or last value respectively in each of the invalid partitions adjacent to the valid partition (0 and 100 in this case) – Four boundary value tests – Three equivalence partitioning tests

Other Test Cases Based of user experience – Always consider creating an equivalence partition that handles the default, empty, blank, null, zero, or none conditions.

Other Test Cases A percentage of the users will use the software incorrectly Test for Invalid, Wrong, and Garbage Data Have fun: – If the software wants numbers, give it letters. – If it accepts only positive numbers, enter negative numbers – If it's date, will it work correctly on the year 3000? – press multiple keys at a time

Quiz Assume that a software has a data-entry field for a 9-digit ZIP code, such as the one shown in the Figure. What equivalence partitions would you create for this text box? What boundary value test cases would you create?