1

Black Box Testing  Black box testing is also called functional testing  Black box testing ignores the internal mechanism of a system or component  Black box testingfocuses solely on the outputs  The outputs are generated in response to selected inputs and execution conditions. 2

Test Planning  Test planning should be done throughout the development cycle  especially early in the development cycle  As soon as customer requirements analysis has completed, the test team should start writing black box test cases against that requirements document. 3

Performing Black Box Testing  Black box testing finds errors in the external behavior of the code  incorrect or missing functionality  interface errors  errors in data structures used by interfaces  behavior or performance errors  initialization and termination errors.  Through this testing, we can determine if the functions appear to work according to specifications. 4

Performing Black Box Testing  It is best if the person who plans and executes black box tests is not the programmer of the code  It is best if the person does not know anything about the structure of the code. 5

Poor Specification of a Test Case 6

Preferred Specification of a Test Case 7

Requirements  When a user lands on the “Go to Jail” cell, the player goes directly to jail, does not pass go, does not collect $200.  On the next turn, the player must pay $50 to get out of jail and does not roll the dice or advance.  If the player does not have enough money, he or she is out of the game. 8

Things to Test There are many things to test in this short requirement above  Does the player get sent to jail after landing on “Go to Jail”?  Does the player receive $200 if “Go” is between the current space and jail?  Is $50 correctly decremented if the player has more than $50?  Is the player out of the game if he or she has less than $50? 9

 At first it is good to start out by testing some input that you know should definitely pass or definitely fail.  If these kinds of tests don’t work properly, you know you should just quit testing and put the code back into development.  We can start with a two obvious passing test case 10

11 Test Plan #1 for the Jail Requirement

Equivalence Partitioning  To keep down the testing costs, we don’t want to write several test cases that test the same aspect of our program.  A good test case uncovers a different class of errors  These are incorrect processing of all data that has been uncovered by prior test cases.  Equivalence partitioning is a strategy that can be used to reduce the number of test cases that need to be developed.  Equivalence partitioning divides the input domain of a program into classes 12

Equivalence Classes for Player Money  Less than $50  $50 or more 13

14 Test Plan #2 for the Jail Requirement

 If input conditions specify a range of values, create one valid and one or two invalid equivalence classes.  In the above example, this is (1) less than 50/invalid; 50 or more/valid.  If input conditions specify a member of a set, create one valid and one invalid equivalence class.  If an input condition is a Boolean, define one valid and one invalid class. 15

Boundary Value Analysis  According to Boris Beizer, “Bugs lurk in corners and congregate at boundaries.”  We need to focus testing at these boundaries.  This is called Boundary Value Analysis (BVA) and guides you to create test cases at the “edge” of the equivalence classes. 16

Boundary Value Analysis  Boundary value is defined as a data value  It corresponds to a minimum or maximum input, internal, or output value specified for a system or component  In the example, the boundary of the class is at 50  We should create test cases for the Player 1 having $49, $50, and $51.  These test cases will help to find common off-by-one errors  caused by errors like using >= when you mean to use >. 17

Boundary Value Analysis Test cases should be created for the boundaries (arrows) between equivalence classes  Less than $50  $50 or more 18

Creating BVA test case  If input conditions have a range from a to b (such as a=100 to b=300), create test cases: immediately below a (99) at a (100) immediately above a (101) immediately below b (299) at b (300) immediately above b (301) 19

COVERAGE CRITERIA FOR TESTING 20

ill-defined terms in Testing  complete testing  exhaustive testing  full coverage Are poorly defined terms because of theoretical limitation of software.  The number of potential inputs for most programs is so large as to be effectively infinite.  The only limitation is the size of the file that can be read by the parser.  the number of inputs is effectively infinite 21

Formal Coverage Criteria  We cannot test with all input  Coverage criteria are used to decide which test inputs to use.  Effective use of coverage criteria makes it more likely  Test engineers will find faults in a program This provides informal assurance Result : High quality and Reliability of the Software 22

Advantage of Coverage Criteria According to the practical perspective Coverage criteria provide rules for when to stop testing. 23

Graph Coverage Criteria Structural Coverage Criteria : Defined on a graph just in terms of nodes and edges Data Flow Coverage Criteria : Requires a graph to be annotated with references to variables 24

Node Coverage (Formal Definition): For each node n ∈ reach G (N 0 ), TR contains the predicate “visit n.” Node Coverage (NC): TR contains each reachable node in G. Structural Coverage Criteria -I Node Coverage (NC) (Standard Definition) Test set T satisfies node coverage on graph G if and only if for every syntactically reachable node n in N, there is some path p in path(T) such that p visits n. TR = { 0, 1, 2 } path(t 1 ) = [0, 1, 2] T 1 ={t 1 } T 1 satisfies node coverage on the graph

Structural Coverage Criteria - II Edge coverage is slightly stronger than node coverage  Edge Coverage (EC) : TR contains each reachable path of length up to 1, inclusive, in G.  The “length up to 1” allows for graphs with one node and no edges Edge Coverage : TR = { (0,1), (0, 2), (1, 2) } path (t 1 ) = [ 0,1,2] path (t 2 ) = [ 0, 2 ] T 2 ={t 1, t 2 } Test Paths = [ 0, 1, 2 ] [ 0, 2 ] T 2 satisfies edge coverage on the graph

Structural Coverage Example Node Coverage TR = { 0, 1, 2, 3, 4, 5, 6 } Test Paths: [ 0, 1, 2, 3, 6 ] [ 0, 1, 2, 4, 5, 4, 6 ] Edge Coverage TR = { (0,1), (0,2), (1,2), (2,3), (2,4), (3,6), (4,5), (4,6), (5,4) } Test Paths: [ 0, 1, 2, 3, 6 ] [ 0, 2, 4, 5, 4, 6 ] path (t 1 )= [ 0, 1, 2, 3, 6 ] path (t 2 )=[ 0, 2, 4, 5, 4, 6 ] T={t 1, t 2 }