1 ECE 453 – CS 447 – SE 465 Software Testing & Quality Assurance Instructor Kostas Kontogiannis

2 Overview  Integration Testing  Decomposition Based Integration  Call Graph Based Integration  Path Based Integration  Discussion Ref: “Software Testing A Craftsman's Approach” 2nd edition, Paul C. Jorgensen

3 Path Based Integration Testing The basic motivation is to combine structural and behavioral type of testing for integration testing as we did for unit testing The basic idea is to focus on interactions among system units rather than merely to test interfaces among separately developed and tested units In this respect, interface-based testing is structural while interaction-based is behavioral Overall we want to express integration testing in terms behavioral threads

4 Source node: –a program statement fragment at which program execution begins or resumes. –for example the first “begin” statement in a program. –also, immediately after nodes that transfer control to other units. Sink node: –a statement fragment at which program execution terminates. –the final “end” in a program as well as statements that transfer control to other units. Extended Concepts (1)

5 Module execution path: –a sequence of statements that begins with a source node and ends with a sink node with no intervening sink nodes. Message: –a programming language mechanism by which one unit transfers control to another unit. –can be interpreted as subroutine invocations, procedure calls and function references. –convention: the unit which receives the message always eventually returns control to the message source. –messages can pass data to other units. Extended Concepts (2)

6 MM-Path: –an interleaved sequence of module execution paths and messages. –we can describe sequences of module execution paths that include transfers of control among separate units. – MM-paths always represent feasible execution paths, and these paths cross unit boundaries. MM-Paths

A B C The Figure 1 above illustrates an MM-Path (the heavy line) for three modules. For example, in module A nodes 1 and 5 are source nodes while nodes 4 and 6 are sink nodes. MM-Path Example Figure 1 Ref: “Software Testing A Craftsman's Approach” 2nd edition, Paul C. Jorgensen

8 In addition, the following are module execution paths: MEP(A,1) = MEP(B,1) = MEP(A,2) = MEP(B,2) = MEP(A,3) = MEP(C,1) = MEP(C,2) = Execution Paths Example

9 Given a set of units, – their MM-Path graph is the directed graph in which nodes are module execution paths and, –edges correspond to messages and returns from one unit to another The definition is with respect to a set of units. –It directly supports composition of units and composition based integration testing. –It is possible to compose down to level of individual module execution paths but it would be more detailed than necessary. MM-Path Graph

10 MEP(A,2) MEP(B,1) MEP(C,1) MEP(B,2) MEP(A,3) MEP(A,1) MEP(C,2) Figure 2 The Figure 2 above illustrates the MM-Path graph for the example in Figure 1. The solid arrows indicate messages and the corresponding returns are indicated by dotted arrows. MM-Path Graph Example Ref: “Software Testing A Craftsman's Approach” 2nd edition, Paul C. Jorgensen

11 MM-Paths implement a function that transcends unit boundaries, thus we have the following relationship: –Consider the “intersection” of an MM-Path with a unit. The module execution paths in this intersection are an analog of a slice with respect to the (MM-Path) function. –Hence the module execution paths in such an intersection are the restriction of the function to the unit in which they occur. MM-Path Analogy

12 There are three observable behavioral criteria that put endpoints on MM-Paths: –event quiescence: occurs when a system is nearly idle, waiting for a port input event to trigger further processing. This is a system level property with an analog at the integration level: message quiescence. –message quiescence: occurs when a unit that sends no messages is reached (i.e. module C in Figure 1). –data quiescence: occurs when a sequence of processing culminates in the creation of stored data that is not immediately used. MM-Path Endpoints

13 Data quiescence occurs when a sequence of processing culminates in the creation of stored data that is not immediately used. These criteria are “natural” endpoints for MM-Paths. A second guideline for MM-Paths serves to distinguish integration from system testing: –atomic system function (ASF): is an action that is observable at the system level in terms of port input and output events. It begins with a port input event, traverses one or more MM-Paths, and terminates with a port output event. Atomic System Function

14 –When viewed from the system level, there is no compelling reason to decompose an ASF into lower levels of detail (hence the atomicity). –For example in the ATM case, an example of an ASF is card entry, cash dispensing, or session closing. While PIN entry would probably be too big since it might entail a molecular system function Atomic System Function

15 ASFs are an upper limit for MM-Paths: –MM-Paths should not cross ASF boundaries. –ASFs represent the seam between integration and system testing: they are the largest item to be tested during integration testing, and the smallest item for system testing. Atomic System Function

16 ASF Example A B C MM-path: Interleaved sequence of module exec path and messages Module exec path: entry-exit path in the same module Atomic System Function: port input, … {MM-paths}, … port output Test cases: exercise ASFs

17 Call Graph as a Model for Module Interaction 1. programSums 2. read(n); 3. i:= 1; 4. while i <= n 5. sum:=0; 6. Acc(sum,i); 7. write(sum, i); 8. i : = i + l ; 9. endwhile 10. end. 11. procedure Acc(x,y) refx, y 12. j:= 1: 13. whilejsy 14. Add(x,j); 15. Inc(j); 16. endwhile 17. return 18. procedure Inc(z) ref z 19. Add(z.1): 20. return 21. procedure Add(a,b) ref a; value b 22. a:=a+b; 23. return Sums Acc Inc Add sum, i j x, j z, 1 Call Graph

18 Program Summary Graph 1. programSums 2. read(n); 3. i:= 1; 4. while i <= n 5. sum:=0; 6. Acc(sum,i); 7. write(sum, i); 8. i : = i + l ; 9. endwhile 10. end. 11. procedure Acc(x,y) refx, y 12. j:= 1: 13. whilejsy 14. Add(x,j); 15. Inc(j); 16. endwhile 17. return 18. procedure Inc(z) ref z 19. Add(z.1): 20. return 21. procedure Add(a,b) ref a; value b 22. a:=a+b; 23. return Sums Acc Inc Add sum x y y x x x j j z z z z a a i i

19 Interprocedural Program Summary Graph Sums Acc Inc Add sum x y y x x x j j z z z z a a i i {U6} {U2, U3, U5}} {U6} {U2. U3} {U1} {U6} {U4, U6} {U1, U4, U6} U1: sum in line 7 U2: i in line 7 U3: i in line 8 U4: j in line 13 U5: y in line 13 U6: a in line programSums 2. read(n); 3. i:= 1; 4. while i <= n 5. sum:=0; 6. Acc(sum,i); 7. write(sum, i); 8. i : = i + l ; 9. endwhile 10. end. 11. procedure Acc(x,y) refx, y 12. j:= 1: 13. whilejsy 14. Add(x,j); 15. Inc(j); 16. endwhile 17. return 18. procedure Inc(z) ref z 19. Add(z.1): 20. return 21. procedure Add(a,b) ref a; value b 22. a:=a+b; 23. return

20 MM-Paths and ASFs are a hybrid of functional and structural testing. They are functional in sense that each represents an action with inputs and outputs, –all functional testing techniques are potentially applicable. –The structural side is how they are identified : MM-Path graph. Discussion on Behavioral Integration Testing

21 We avoid the pitfalls of structural testing, – and at same time, integration testing gains a fairly seamless junction with system testing. Advantages come at a price: –more effort to identify MM-Paths and ASFs. – Effort may be offset by elimination of stub/driver development. –Also may be an overkill for applications which are not event driven. Discussion on Behavioral Integration Testing