Object-Oriented Software Testing ICS 224A Joachim Feise

June 4, 1997OO Software Testing2 A Brief Meditation on the Essential Nature of Software Testing

June 4, 1997OO Software Testing3 Object-Oriented Testing Papers Object-Oriented Integration Testing –Jorgensen and Erickson –CACM Sept Object-Oriented Software Testing - A Hierarchical Approach –Shel Siegel –J. Wiley and Sons, 1996

June 4, 1997OO Software Testing4 Structure vs. Behavior most popular notations in software development portray software structure examples: E/R models, data flow diagrams, structure charts, etc. –used to show relationships between components, interfaces, control and dataflow, etc. –not very useful for testing SW testing is fundamentally concerned with behavior, i.e. what it does

June 4, 1997OO Software Testing5 Traditional Testing Notions Traditional software is –written in an imperative language –described by functional decomposition –developed in a waterfall life cycle –separated into three levels of testing unit level, integration level, system level these points do often not apply directly to object-oriented software

June 4, 1997OO Software Testing6 Imperative vs. Declarative Imperative languages lend themselves to a description as a directed graph declarative languages suppress sequentiality the event-driven nature of OO systems forces a “declarative spirit” on testing –not evident at unit level since most object- oriented languages are imperative –but pronounced at integration and system levels

June 4, 1997OO Software Testing7 Unit Testing Overview Units can be defined in several ways a single, cohesive function a function which, when coded fits on one page the smallest separately compilable segment of code the amount of code that can be written in 4 to 40 hours a task in a work breakdown structure code that is assigned to one person code that one person designs, codes and test in a three- month period

June 4, 1997OO Software Testing8 Unit Testing Overview (cont.) Many organizations have not chosen their definition of a software unit Units are always tested “by itself” –stubs and drivers are used to emulate inputs and outputs –goal: verification that, taken by itself, the unit functions correctly

June 4, 1997OO Software Testing9 Integration Testing Overview Once the units are separately tested, they are integrated together gradual replacement of stubs and drivers by separately tested units pairwise integration, i.e. each unit is integrated with its adjacent units bottom-up integration guided by functional decomposition tree “big bang” integration where all units are thrown together at once

June 4, 1997OO Software Testing10 System Testing Overview Conducted exclusively in terms of inputs and outputs that are visible at the port boundary of a system Tester only has access to those port events that are visible to the customer/user Can occur on the target platform only

June 4, 1997OO Software Testing11 System Testing Overview (cont.) Examples of system testing activities sequences of machine instructions sequences of source instructions scenarios of normal usage system-level test cases stimulus/response pairs behavior that results from sequences of system-level inputs sequences of transitions in a state machine description of the system

June 4, 1997OO Software Testing12 Object-Oriented Software –within an object, individual methods are imperative –declarative aspect because of event-driven nature –dynamic binding creates indefiniteness that resembles declarative programs –it is impossible to ever know the full set of “adjacent” objects with which a given object may be composed two object may be correct, but errors might result when they are composed

June 4, 1997OO Software Testing13 Composing Errors Illustrated

June 4, 1997OO Software Testing14 Levels of OO Testing Unit testing –method testing Integration testing –Message quiescence –Event quiescence System testing –Thread testing –Thread interaction testing

June 4, 1997OO Software Testing15 Integration Testing Method executions are linked by messages Quiescence provides natural endpoints Message quiescence –Method/Message path (MM-Path) sequence of message executions linked by messages starts with a method ends when method is reached that does not issue any messages of its own –endpoint is called message quiescence

June 4, 1997OO Software Testing16 Integration Testing (cont.) Atomic System Function (ASF) input port event, followed by a set of MM-Paths, terminated by output port event –ASFs constitute the point at which integration and system testing meet –elemental function visible at the system level

June 4, 1997OO Software Testing17 MM-Paths and ASFs Illustrated

June 4, 1997OO Software Testing18 ATM System Class Hierarchy

June 4, 1997OO Software Testing19 Example: ATM System Examples of –MM-Path: Digit entry –ASF: PIN Entry –Thread (sequence of ASFs): simple transaction: PIN entry, transaction type selection, account detail presentation, operation execution, result reporting –Thread sequence: a session consisting of multiple simple transactions

June 4, 1997OO Software Testing20 Digit Entry MM-Path Illustrated

June 4, 1997OO Software Testing21 PIN Entry ASF Illustrated

June 4, 1997OO Software Testing22 Observations Many-to-many relationships dominate (see E/R diagram) –an object may be involved in many threads –threads entail many objects –an object may be involved with many ASFs –ASFs may entail many objects

June 4, 1997OO Software Testing23 Observations (cont.) The five levels of OO testing result in distinct, useful testing goals Bottom-up testing order Seamless juncture between OO integration testing and system testing Constructs are directly usable as basis for test coverage metrics Works well with composition

June 4, 1997OO Software Testing24 The Hierarchical Approach Everything is a system Defines and applies testing standards for several levels of software components –objects –classes –foundation components –systems

June 4, 1997OO Software Testing25 Hierarchical Approach Illustrated

June 4, 1997OO Software Testing26 Safe Components Components that meet the testing standards for that kind of components only safe components can be integrated with other safe components to produce the next- level component Safe is a relative state –depends on risk management practices, attitude toward risk, what is enforced, the application, etc.

June 4, 1997OO Software Testing27 Foundation Components Can be one complete class hierarchy or some cluster of classes –that performs a core function, –that represents a logical or physical architectural component, etc. After testing to a safe level, can be integrated with other foundation components

June 4, 1997OO Software Testing28 Integration Testing of Foundation Components Needs to address the interconnections between the foundation components and any new composite functionality ONLY Eliminates the need to test all of the combinations of states during integration testing Improves productivity

June 4, 1997OO Software Testing29 Integration Testing Problem Illustrated

June 4, 1997OO Software Testing30 Test Suites Conditional Test Suite Hierarchical Incremental Test Suite Integration Test Suite System Test Suite Regression Test Suite

June 4, 1997OO Software Testing31 Conditional Test Suite Four types of test scripts –Assertions: abort if condition fails –Exceptions: raise exception if condition succeeds –Concurrent test operations: monitor condition –Message polling: evaluate condition on demand Conditions –class invariant, precondition, postcondition, loop invariant

June 4, 1997OO Software Testing32 Pre/Postconditions Illustrated

June 4, 1997OO Software Testing33 Hierarchical Incremental Test Suite Structural models –control-flow model models class as series of control-flow graphs deriving from the method implementations –data-flow model models class as series of control-flow graphs with data annotations showing the flow of data values through the method implementations

June 4, 1997OO Software Testing34 Hierarchical Incremental Test Suite (cont.) Functional models –state-transition model, aka OMT dynamic model models states and transitions for a class in the context of its position in the inheritance hierarchy –transaction-flow model models object life cycle as transaction from construction to destruction –exception model specifies what methods raise what exceptions under what conditions

June 4, 1997OO Software Testing35 Integration Test Suite Functional Test Suite –tests vertical integration of a cluster of components, i.e. the way the cluster implements specific functions from the requirements Abstraction Test Suite –tests horizontal integration of a cluster of components, i.e. the way the cluster abstracts a service contract into an API

June 4, 1997OO Software Testing36 Integration Test Suite (cont.) Possible test scripts –class-to-class test script interaction between classes based on message relationship protocol-interface, synchronization, exception handling tests –UI test script tests UI components of cluster and the components on which they depend true-time, character-recognition, widget-playback tests

June 4, 1997OO Software Testing37 System Test Suite System Acceptance Test Suite –derived from the set of use cases covering the system requirements User Options Test Suite –based on probable usage patterns of the features and functions delivered with the system –designed to simulate the way the end users will combine the features and functions

June 4, 1997OO Software Testing38 Regression Test Suite Series of tests run against a previous version of the CUT Almost always automated –performing manual tests is usually not cost effective

June 4, 1997OO Software Testing39 Epilog “Testing proves the presence, not the absence, of bugs.” E.W. Dijkstra “Absence of evidence is not evidence of absence.” Source unknown From: E.V. Berard, Issues in the Testing of OO Software Slides available at