1. Chapter 13 Integration Testing
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

2. Levels of Testing (in the famous V-Model)
Requirements Specification
System Testing
Preliminary Design
Integration Testing
Detailed Design
Unit Testing
Coding
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

3. The Mars Climate Orbiter Mission
mission failed in September 1999
completed successful flight: 416,000,000 miles ( km)
41 weeks flight duration
lost at beginning of Mars orbit
An integration fault: Lockheed Martin used English units for acceleration calculations (pounds), and Jet Propulsion Laboratory used metric units (newtons).
NASA announced a US$ 50,000 project to discover how this happened.
(We know!)
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

4. Testing Level Assumptions and Objectives
Unit assumptions
All other units are correct
Compiles correctly
Integration assumptions
Unit testing complete
Unit goals
Correct unit function
Coverage metrics satisfied
Integration goals
Interfaces correct
Correct function across units
Fault isolation support
System goals
Correct system functions
Non-functional requirements tested
Customer satisfaction.
System assumptions
Integration testing complete
Tests occur at port boundary
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

5. Goals/Purpose of Integration Testing
Presumes previously tested units
Not system testing
Tests functionality "between" unit and system levels
Basis for test case identification?
Emphasis shifts from “how to test” to “what to test” (Model-Based Testing)
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

6. Approaches to Integration Testing
(“source” of test cases)
Functional Decomposition (most commonly described in the literature)
Top-down
Bottom-up
Sandwich
“Big bang”
Call graph
Pairwise integration
Neighborhood integration
Paths
MM-Paths
Atomic System Functions
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

7. Basis of Integration Testing Strategies
Functional Decomposition applies best to procedural code
Call Graph
applies to both procedural and object- oriented code
MM-Paths
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

8. Continuing Example (pseudo-code skeleton)
Main NextDate /*NextDate computes the date of the next day after a given date*/ Function monthEnd(month, year) /* returns number of days in a given month */
Function isLeap(year) /* determines if a year is a leap year */ End (Function isLeap)
/* calls isLeap */
End (Function monthEnd)
Procedure GetDate(aDate) /* gets an input date */
Function ValidDate(aDate)
/* calls MonthEnd *?
End (Function ValidDate)
/* calls ValidDate */
End (Procedure GetDate)
/* checks if a date is valid */
Procedure IncrementDate(aDate) /*increments a date */
/* calls ValidDate */ End (IncrementDate)
Procedure PrintDate(aDate) /* prints a date */ End (PrintDate)
/*Main program begins here */
/* calls GetDate, IncrementDate, and PrintDate */ End Main
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

9. Integration Based on Functional Decomposition
Start with functional decomposition (usually derived from code, sometimes designed first)
Choose integration order (top-down, bottom up, sandwich)
Create stubs or drivers
Create test cases
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

10. NextDate Functional Decomposition
Main
monthEnd
GetDate
incrementDate
PrintDate
isLeap
ValidDate
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

11. Step 1-Test main program logic with all stubs
monthEndStub
GetDateStub
incrementDateStub
PrintDateStub
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

12. Steps 2 - n: Test main program logic, replace one stub at a time
monthEnd
GetDateStub
incrementDateStub
PrintDateStub
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

13. (assures fault isolation to the unit level)
Replacing one stub at a time...
(assures fault isolation to the unit level)
Main
monthEnd
GetDate
incrementDateStub
PrintDateStub
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

14. Main monthEnd GetDate incrementDate PrintDateStub
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

15. Last Stub Replacement for Main
monthEnd
GetDate
incrementDate
PrintDate
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

16. Step n + 1 - Move down one level, and repeat the stub introduction
Main
monthEnd
GetDate
incrementDate
PrintDate
isLeapStub
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

17. But, there is a problem... The Main program does not invoke MonthEnd
MonthEnd is in the Functional Decomposition tree because of scoping rules of most compilers.
(this is more of a problem in larger systems)
Same problem occurs in bottom-up integration
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

18. (Start with leaf nodes, and test with drivers)
Bottom-up Integration
(Start with leaf nodes, and test with drivers)
Main
monthEnd
GetDateDriver
incrementDate
PrintDate
isLeap
ValidDate
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

19. At Each Step, Replace a Driver With Actual Code
Main
monthEnd
GetDate
incrementDate
PrintDate
isLeap
ValidDate
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

20. When All Drivers at a Level Have Been Replaced, Move up one Level and Replace One Unit at a Time
MainDriver
monthEnd
GetDate
incrementDate
PrintDate
isLeap
ValidDate
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

21. Note the Fault Isolation
MainDriver
monthEnd
GetDate
incrementDate
PrintDate
isLeap
ValidDate
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

22. Next Step in Integration
MainDriver
monthEnd
GetDate
incrementDate
PrintDate
isLeap
ValidDate
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

23. This Level Tests the MainDriver Logic
monthEnd
GetDate
incrementDate
PrintDate
isLeap
ValidDate
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

24. Last Step in Bottom-up Integration
Main
monthEnd
GetDate
incrementDate
PrintDate
isLeap
ValidDate
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

25. (no stubs, no drivers, but less fault isolation)
Sandwich Integration
(no stubs, no drivers, but less fault isolation)
Main
monthEnd
GetDate
incrementDate
PrintDate
isLeap
ValidDate
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

26. An Impossible Sandwich
(Why?)
Main
monthEnd
GetDate
incrementDate
PrintDate
isLeap
ValidDate
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

27. (No stubs, no drivers, no fault isolation)
Big Bang Integration
(No stubs, no drivers, no fault isolation)
Main
monthEnd
GetDate
incrementDate
PrintDate
isLeap
ValidDate
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

28. Call Graph Based Integration
Derive Call Graph from source code
Nodes are units
There is an edge from unit A to unit B if A calls B
Choose approach (pairwise or neighborhood)
Develop test cases
Need special tools (or code) to check “before and after” interface interaction
Resolves question of impossible interfaces (e.g., monthEnd)
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

29. NextDate Call Graph Main GetDate incrementDate PrintDate ValidDate
monthEnd
isLeap
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

30. Pairwise Integration - test each edge in Call Graph
Main pair 2
incrementDate
pair 1
pair 3
GetDate
PrintDate
pair 4
ValidDate
pair 7
pair 5
monthEnd
pair 6
isLeap
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

31. Redundancy in Pairwise Integration
Each node may be involved in several tests (degree of node) (and the non-existent interface problem is fixed)
Node
Number of pairs
Main 3
GetDate 2
incrementDate
PrintDate 1
ValidDate
MonthEnd
IsLeap
Total tests 14
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

32. Neighborhood Integration
A neighborhood of a node is the set of all other nodes a given distance (usually 1) away.
GetDate
Main
ValidDate
incrementDate
PrintDate
monthEnd
isLeap
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

33. Neighborhood of monthEnd
GetDate
Main
ValidDate
incrementDate
PrintDate
monthEnd
isLeap
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

34. Neighborhood of ValidDate
GetDate
Main
ValidDate
incrementDate
PrintDate
monthEnd
isLeap
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

35. Redundancy in Neighborhood Integration
There is one neighborhood (of radius 1) for each node.
7 nodes yield 7 integration sessions.
In general, neighborhood integration reduces the number of test sessions required by pairwise integration.
This reduction comes at the expense of increased fault isolation effort.
Some special code may still be needed.
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

36. NextDate Class Descriptions
Attributes
Methods
testIt
calendarItem (abstract class)
value
getValue setValue IncrementValue
Date
Day Month Year
getDate printDate incrementDate
Day
Month
Year
endOfMonth
isLeap
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

37. UML Collaboration Diagram for NextDate
1: create
2: increment
3: getYear
Year
1: create
2: increment 3:setMonth
1:isLeap
1: create
2: increment
3: printDate
4: getMonth
testIt
Date
Month
1: create
2: increment
3: setDay
4: getDay
Day
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

38. Integration Based on UML Collaboration Diagram
A UML Collaboration Diagram is essentially an undirected graph.
Not a problem since the direction of edges in Call Graph integration is not used.
Therefore UML Collaboration Diagram based integration testing reduces to Call Graph based testing.
The tendency in the object-oriented community is to use pairwise integration.
(Neighborhood integration is a better choice!)
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

39. Integration Based on MM-Paths
an MM-Path is an alternating sequence of unit execution paths and unit invocations (or messages).
In data-driven programs, MM-Paths begin in the Main Program.
In Event-driven programs, MM-Paths begin in the event sensing units.
MM-Paths “end” when no further messages are sent. (message quiescence)
Techniques apply equally well to procedural and object- oriented code.
MM-Paths are always feasible paths.
Increased effort to identify MM-Paths.
Increased identification effort offset by elimination of stub/driver code.
AND, offset by superb fault isolation capability.
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

40. Extensions to Definitions
A source node in a program is a statement fragment at which program execution begins or resumes.
A sink node in a program is a statement fragment at which program execution terminates.
A module execution path is a sequence of statements that begins with a source node and ends with a sink node, with no intervening sink nodes.
A message is a programming language mechanism by which one unit transfers control to another unit.
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

41. Definitions for Integration Testing
An MM-Path is an interleaved sequence of module execution paths and messages.
Given a set of units, their MM-Path graph is the directed graph in which nodes are module execution paths and edges correspond to messages and returns from one unit to another.
An atomic system function (ASF) is an action that is observable at the system level in terms of port input and output events.
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

42. An MM-Path A B C 1 1 1 2 2 2 3 3 4 4 3 5 5 4 6
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

43. Question: What data determines MM-Paths 1 and 2?
Two MM-Paths
Question: What data determines MM-Paths 1 and 2?
Main
incrementDate
PrintDate
GetDate
ValidDate
MM-Path 1
monthEnd
MM-Path 2
isLeap
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

44. Program Graphs of MonthEnd and IsLeap2 3
isLeap
monthEnd 15 4 16 18 20 5 12 7 17 19 21 6 8 9 22 23 10 24 11 13 25 14 26
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

45. An MM-Path Test Case as a Series of Assertions
Unit
Invocation Parameter/value
Unit Execution Paths
Return Parameter/value
main
today. Month =11
today.Day = 11
today.Year =2004
<87>
IncrementDate
aDate. Month=11
aDate.Day = 11
aDate.Year=2004
<69, 70>
monthEnd
aDate.Month 11
aDate.Year 2004
<2, 15, 18, 19, 25, 26 >
monthEnd= 30
< 71, 78, 79>
aDate.Month=11
aDate.Day = 12
tomorrow. Month =11
tomorrow.Day = 12
tomorrow.Year =2004
<87 >
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

46. Comparison of Integration Testing Strategies
Strategy Basis
Ability to test Interfaces
Ability to test co-functionality
fault isolation resolution
Functional
acceptable but can
limited to pairs of
good, to faulty unit
Decomposition
be deceptive
units
Call Graph
acceptable
MM-Path
excellent
complete
excellent, to faulty
unit execution path
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

47. Case Study: NextDate Revisited as a Main Program with Functions
Pseudo-code showing messages
Program graphs
Functional decomposition tree
Call graph
MM-Paths
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

48. (Integration Version)
NextDate Main Program
(Integration Version)
1. Main integrationNextDate Dim today As Date
Dim tomorrow As Date 2.
GetDate(today)
'msg1 3.
PrintDate(today)
'msg2 4.
tomorrow = IncrementDate(today)
'msg3 5.
PrintDate(tomorrow)
'msg4 6.
End Main
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

49. isLeap Function isLeap(year) If (year divisible by 4) Then 10 11 12 1314 15 16 17 18 19 20
Boolean
If (year is NOT divisible by 100) Then isLeap = True
Else
If (year is divisible by 400) Then isLeap = True
Else isLeap = False EndIf
EndIf
Else isLeap = False EndIf
End (Function isLeap)
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing

50. lastDayOfMonth 21 Function lastDayOfMonth(month, year) Integer 22
Case month Of 23
Case 1: 1, 3, 5, 7, 8, 10, 12 24
lastDayOfMonth = 31 25
Case 2: 4, 6, 9, 11 26
lastDayOfMonth = 30 27
Case 3: 2 28
If (isLeap(year))
'msg5 29
Then lastDayOfMonth = 29 30
Else lastDayOfMonth = 28 31
EndIf 32
EndCase 33
End (Function lastDayOfMonth)
Software Testing: A Craftsman’s Approach, 3rd Edition
Integration Testing