1. 1 ECE 453 – CS 447 – SE 465 Software Testing & Quality Assurance Instructor Kostas Kontogiannis

2. 2 Overview  Object Oriented Testing  General - Introduction  Unit Testing  Integration Testing  System Testing Ref: “Software Testing A Craftsman's Approach” 2nd edition, Paul C. Jorgensen

3. 3 Testing of Object Oriented Programs General Notes Testing Implications Effects of: –Encapsulation –Method Overriding –Inheritance Hierarchical Incremental Class Testing –Base Class Testing

4. 4 General Notes As with traditional development object-oriented development proceeds via structural, contextual, and behavioral models –Structural models focus on the encapsulation between actions and data or ports –Contextual models focus on port objects and boundaries of the system –Behavioral models focus on the states of an object in terms of the vents occurring However, Object Orientation poses new challenges into Testing

5. 5 The Basic Constructs in OO Data Port Action Thread Event n n n n n n Object

6. 6 Levels of Testing Classes as a subject of unit level object-oriented testing –Several methods in a class –Some methods can be large Methods are a better starting point for Unit Testing in object oriented systems System testing is generally unaffected by object orientation –MM Path as a sequence of method execution linked by messages –ASF is an input port event, followed by one or more MM-Paths and terminated with an output port event Messages not a decomposition tree as a driving force for Integration testing

7. 7 Testing Adequacy A program is “adequately” tested if it has been covered according to some criteria. –Specification-based (i.e. black-box) Determine if program meets the functional and non-functional specifications. –Program-based (i.e. white box) Inspect program statements and select test cases to cover program. Basic approaches are orthogonal and complementary

8. 8 Testing Implications Antiextensionality –Given two programs that compute the same function, then a test set that is adequate for one program is not necessarily adequate for the other. –From a specification-based testing point of view: Both programs have the same specification – hence a test adequate for one must be adequate for the other. But the implementations may be radically different - No guarantee that same coverage attained on both programs with same test set.

9. 9 General Multiple Change –When two programs are syntactically similar (have same shape) they usually require different test sets. –“same shape” if one program can be transformed into the other applying following rules: Replace relational operator r1 in a predicate with relational operator r2. Replace constant c1 in assignment or predicate with constant c2. Replace arithmetic operator a1 in assignment statement with operator a2. –A test case to drive a branch in one program may not drive the same branch in the other. Testing Implications

10. 10 Antidecomposition –A component’s test set may be adequate in the context of the enclosing program, but may not be adequate for other uses of that component. –An example may be that of unreachable code. Code that is unreachable within one context may in fact be reachable in another context. In specification-based testing, the enclosing program may not utilize all the functionality of the component. Testing Implications

11. 11 Anticomposition –Adequate testing of individual components in isolation may not necessarily provide adequate testing for the entire program. –The composition of components may produce interaction errors not visible in isolation. –If components are sequential, then less powerful but still true. –Consider: a specification captures only what the program should do and behavior outside the scope of the specification may not be visible in one context, but cause errors in another context. Testing Implications

12. 12 Testing Implications Encapsulation –If make change to implementation of a unit, then limit testing only to affected unit, users of the unit should be unaffected. –But consider now, anticomposition: The necessity to retest every dependent unit as well. Implication: integration testing is always necessary in addition to unit testing – regardless of program paradigm.

13. 13 Fortunately, need only to retest the changed unit and only those which depend on it. The use of subclasses adds an unexpected form of dependency: –We expect that if a superclass is modified, then we must retest all subclasses because of their inheritance dependency. –From antidecomposition: if an new subclass is added or an old one modified, we must retest the methods inherited from each of its ancestor superclasses. Reason: provides a new context for the inherited methods. The dependency is actually in both directions. Testing Implications

14. 14 –Exception: the new subclass is a pure extension of the superclass: Adds only new instance variables and new methods. There are no interactions with the new and old variables and methods.

15. 15 Testing Implications Method Overriding The overriding subclass must be retested and due to antiextensionality, a different test is often needed. –From a program-based test approach, the same coverage might not be obtained with the original test set. –Often, the overriding method has not only a structural change but also different eternal behavior

16. 16 Testing Implications Inheritance Consider the subclass definition as a modifier which defines attributes that differ from or alter the attributes in the parent class. The modifier + parent class + inheritance rules for the language define the subclass. The subclass designer specifies the modifier. Ref [Harrold]

17. 17 Although M transforms the parent class P into the result class R it does not totally constrain R. We must take into account the inheritance relation which determines the effects of composing P and M into R. P Parent class M modifier  R resultant class The composition operator,  unites M and P to obtain the new class R, where R = P  M

18. 18 The modifier approach allows a decomposition of the inheritance structure into overlapping sets of class inheritance relations. This decomposition allows the consideration of only the immediate parents and the modifier when testing a subclass. A B C A M1 M2   B C A M1  B B M2  C

19. 19 The inheritance relation imposes an ordering on the classes: –Class C can be determined without considering class A –But the relation from A to B must be resolved prior to describing the relation from B to C. –This allows consideration of only a class and its immediate parents to fully constrain the definition of that class.

20. 20 Hierarchical Incremental Class Testing A class testing technique that: –Initially test a base class by testing each member function individually. –Then test the interactions among member functions. Save test case design and execution information in a testing history.

21. 21 When a subclass is defined: Testing history of parent, Definition of modifier, Inheritance mapping –Determine which attributes to (re)-test in the subclass and which test cases of parent can be reused.

22. 22 Base Class Testing From antidecomposition: –Adequate testing of base class does not guarantee adequate of each member function. –Important to adequately test member function due to the inheritance by subclasses and operation in new contexts. –Individually test each member function using specification and program-based test cases. –Handle calls to other member functions using stubs.

23. 23 From anticomposition: –Testing each member function individually does not imply that the class has been tested adequately. –Use intra-class testing: test interactions among member functions within the same class. –Develop both specification and program-based test suites. Inter-class testing: when member functions of one class interact with member functions in another class: –Member function in one class receives an instance of another class and sends that class a message. –An instance of class is part of representation of another class and then sends that instance as a message.

24. 24 Testing Subclasses Actions dependent on the attribute type and the type of modifications due to the inheritance mapping: –New or virtual-new member function attribute A must be completely tested since it was not defined in P. Integration test A with other members of R with which it interacts. –Recursive or virtual-recursive member function attribute A needs limited retesting since it was previously tested in P. Its specification and implementation remain unchanged. –Refined or virtual-refined attribute A in M needs extensive retesting. Many of the existing specification-based tests can be reused since only implementation has changed.

25. 25 Use data flow testing to validate the interfaces between procedures. –Focus: the definitions and uses of variables that extend across procedure boundaries (include global variables and reference parameters). –Validate class member functions individually and test interface among member functions. –Use stubs/drivers to represent incomplete implementations.

26. 26 References 1.D.E. Perry and G.E. Kaiser: Adequate Testing and Object-Oriented Programming, Journal of Object-Oriented Programming, Jan. 1990, pp. 13-19. 2.M.J. Harrold, J.D. McGregor and K.J. Fitzpatrick: Incremental Testing of Object- Oriented Class Structures, Proceedings of 14 th Int’l Software Eng., 1992, pp. 68-80.