CSE 7314 Software Testing and Reliability Robert Oshana Lecture 20

Introduction Chapter 1

Introduction Testing - Challenging but understood –Unit, integration, system, acceptance Testing OO systems –“the same, only different” ;-) We’ll talk about testing OO systems throughout the development cycle Testing is the process of uncovering evidence of defects in software systems

Introduction Bug; a mistake, misunderstanding, omission, or misguided intent on the part of the developer Testing is the process of finding defects Testing does NOT include efforts to track down bugs and fix them (e.g. debugging or repair of bugs)

Introduction Testing; making sure an application does everything it is supposed to do and nothing that it is not supposed to do Guards against time, property, customers, life

What is software? Instruction codes and data necessary to accomplish a task on a computer Also all representations of those instructions and data –Models created during analysis/design

What is software? Testing is not quality assurance Testing is a necessary but insufficient part of a quality assurance program QA; prevention and well as removal Testing will not necessarily improve the quality of a computer program

Testing OO systems OO features impact testing –Class inheritance –Interfaces/polymorphism –Language features to support data hiding impact testing Operation must be added just to support testing Can also support more reusable testing software Many approaches have counterparts in traditional testing

Testing OO systems With an OO approach, a program’s design is structured from the problem, and not from the immediately required solution Analysis models map straightforward to design models that map to code Can therefore start testing during analysis, and refine during design and implementation

Advantages to testing analysis and design models Test cases can be identified earlier in the process Bugs can be detected early in the development process Test cases can be reviewed early in the process for correctness Testing can also be used to check designs for reusability and scalability

Overview of testing approach Time and money constraints must be real concern Test early, test often, test enough –Analyze a little –Design a little –Code a little –Test what you can

Testing for OO software Model testing Class testing, which replaces unit testing Interaction testing, which replaces integration testing System (and subsystem) testing Acceptance testing Deployment/self testing

Test early Start testing at reasonable points during the analysis and design phases of a project Can uncover problems when they are easier and cheaper to fix Can help scope the size of the effort for system testing SW representations are abstract and/or incomplete

Test often Incremental or iterative development should require testing at each increment After first increment, some testing is in the form of regression testing

Test enough Complete testing of every aspect of a software system is infeasible Resources should be directed where they have the greatest impact Tests should be based in risk analysis, reuse of test cases, and statistical sampling of inputs for test cases

The testing perspective

Testing perspective A way of looking at the development product and questioning its validity Search for faults both systematic as well as intuitive insight Reviews; almost never find something that is missing –Validates what exists –No systematic search to determine things that should be in are actually in

Testing perspective Software must execute according to spec and only that spec –Make sure it does what it is supposed to do –Does not do what it is not supposed to do Can be performed by same developer or independent view

Review, inspection, execution Inspection; examination of SW based on checklist of typical problems –Most based on programming language semantics or coding conventions Program variables initialized Pointers or references set correctly –Modern compilers can find many of these

Review, inspection, execution Review; examination of SW to find faults before the SW is executed Delves into the meaning of each part of a program Missed or misunderstood requirements or faults in the program’s logic Whether variables are well chosen Whether algorithms are as efficient as they could be

Review, inspection, execution Test execution; testing SW in the context of a running program Tester tries to determine whether it has the required behavior by giving the program some input and verifying that the resulting output is correct Challenge is to identify suitable inputs, determining correct outputs, and how to observe the outputs

Testing perspective Skeptical; what proof of quality Objective; make no assumptions Thorough; don’t miss important areas Systematic; searches are reproducible

Object-Oriented Concepts Object Message Interface Class Inheritance Polymorphism

Object An operational entity that encapsulates both specific data values and the code that manipulates those values Basic computational entity Objects are created, modified, accessed and/or destroyed as a result of collaborations

Object A representation of some specific entity in a program or in its solution Objects are the direct target of the testing process –Behaves according to specification –Interacts appropriately with collaborating objects

Observations Objects encapsulate; makes the complete definition of the object easy to identify; easy to pass around in the system; and easy to manipulate Objects hide information; changes to the object sometimes hard to observe, makes checking of test results difficult

Observations Object has a state that persists for the life of the object; state can become inconsistent and can be the source of incorrect behavior Object has a lifetime; can be examined at various points in the lifetime to determine if in appropriate state based on lifetime (construction too late or destruction too early are common sources of failures)

Message A request that an operation be performed by some object Includes –Name of the operation –Actual parameters Collaboration achieved by sending messages –Sender and receiver

Observations Message has a sender; sender determines when to send the message (may make incorrect decision) Message has a receiver; may not be ready for the specific message it receives, or receiver may not make correct action when receiving unexpected message

Observations Message may include actual parameters; used and/or updated by the receiver while processing the message Objects passed as parameters must be in correct states before and after message is processed Must implement interfaces expected by the receiver

Object-Oriented concepts Interface; aggregation of behavioral declarations Grouped to define actions related to a single concept A building block for specifications which define to total set of public behaviors for a class

Interface example public interface Movable { public Point getPosition( ); public Velocity getVelocity( ); public void setVelocity(Velocity newVelocity); public void tick( ); public void move( ); public void collideWith(ArcadeGamePiece aPiece, Point aPoint); public void collideWithPaddle(Paddle aPaddle, Point aPoint); public void collideWithPuck(Puck aPuck, Point aPoint); public void reverseY( ); public void reverseX( ); }

Observations Interfaces encapsulate operation specifications These specifications incrementally build the specifications of larger grouping such as classes I/F must contain appropriate behaviors or leads to unsatisfactory designs I/F has relationships with other I/Fs and classes and may be specified as a parameter type for a behavior

Object-Oriented concepts Class; a set of objects that share a common conceptual basis A template or “cookie cutter” to create objects Classes are the basic elements for defining OO programs while objects are the basic elements for executing OO programs Create by instantiation

Object-Oriented concepts Class specification; declaration of what each of the objects can do Class implementation; definition of how each of the objects in the class do what they can do

Example class specification #ifndef PUCKSUPPLY_H #define PUCKSUPPLY_H Class PuckSupply { Public: PuckSupply( ); ~PuckSupply( ); Puck* get( ); int size( ) const; Private: static const int N = 3; int_count; Puck* _store[N]; }; #endif WHAT …

Example class implementation #include “PuckSupply.h” PuckSupply::PuckSupply( ) : _count(N) { int i; for ( i=0; i<N; i++) { _store[ i ] = new Puck; }; } PuckSupply::int i; for ( i=0; i<N; i++) { delete _store[ i ]; }; } Puck* PuckSupply::get( ) { return (_count > 0 ? _store[--count] : 0 ); } ….. HOW …

Class specification Must be a specification for each operation –Accessor (inspector) operations –Modifier (mutator) operations –Constructor –Destructor

Operation semantics Well specified operation semantics are critical to both development and testing –Preconditions –Postconditions –Invariants

Approaches to defining an interface Contract approach; emphasizes preconditions and has simpler postconditions Defensive programming; emphasizes postconditions and has simpler preconditions (I/F defined in terms of the receiver) –Goal is to define “garbage in” and eliminate “garbage out”

Class implementation A class implementation describes how an object represents its attributes and carries out operations –Set of data values stored in data members –Set of methods –Constructors –Destructors –Private operations

Class testing Class testing is important because they define the building blocks for OO programs Potential causes of failures –Class spec contains operations to construct instances that may not properly initialize attributes of new instances

Class testing –Class relies on collaboration with other classes that may not be implemented correctly and contribute to the failure of the class using the definition –Class implementation satisfies the spec but the spec may be wrong –Implementation might not support all required operations or may incorrectly perform operations

Class testing –Class specifies preconditions to each operation but the class may not provide a way for the precondition to be checked by a sender before sending a message The design approach used (contract or defensive) gives rise to different sets of problems

A word on abstraction The process of removing detail from a representation Allows us to look at a problem or its solution in various levels of detail Can leave out any details that are irrelevant to the current level of interest OO technologies make extensive use of this

A word on abstraction From a testing perspective, layers of abstraction in the development process are paralleled by layers of test analysis Start with the highest levels of abstraction and provide a more thorough examination of the development product with more accurate tests

Inheritance This is a relationship between classes that allows the definition of a new class based on the definition of an existing class –Reuse of specification and implementation –Preexisting class does not have to be modified

Inheritance Good OO design uses inheritance for is a (is a kind of) relationship Best use is with respect to specification and implementation

Inheritance from a testing perspective Provides a mechanism by which bugs can be propagated from a class to its descendents –Testing a class as it is developed eliminates faults early before they are passed to other classes Provides a mechanism to reuse test cases –Can reuse test cases from superclass to test subclass

Inheritance from a testing perspective Models an is a kind of relationship –Inheritance solely for code reuse leads to maintenance difficulties –Testers can check to make sure inheritance is used correctly

Polymorphism The ability to treat an object as belonging to more than one type More typing systems support designs that are flexible and easy to maintain

Inclusion polymorphism The occurrence of different forms in the same class Provides the ability to substitute an object whose spec matches another object’s spec for the latter object in a request for an operation –Sender can use an object as a parameter based on its implementation of an I/F rather than its full class

Set diagram for a Brick inheritance hierarchy Brick HardBrickPowerBrick

Abstract class A class whose purpose is primarily to define an interface that is supported by all of its descendents Use of abstractions allows polymorphism to be exploited during design

Set diagram for a Brick class inheritance hierarchy Brick HardBrickPowerBrick PlainBrick

10 minute break

CSE 7314 Software Testing and Reliability Robert Oshana Lecture 21

Inclusion polymorphism testing perspective Inclusion polymorphism allows systems to be extended incrementally by adding classes rather than modifying existing ones –Unanticipated interactions can occur in the extensions

Inclusion polymorphism testing perspective Inclusion polymorphism allows any operation to have one or more parameters of a polymorphic reference This increases the number of possible kinds of actual parameters that should be used

Inclusion polymorphism testing perspective Inclusion polymorphism allows an operation to specify replies that are polymorphic references The actual class of the referent could be incorrect or unanticipated by the sender

Inclusion polymorphism testing perspective The dynamic nature of object- oriented languages places more importance on testing a representative sample of runtime configurations Static analysis can provide the potential interactions that might occur, but only runtime configuration can illustrate what actually happens

Parametric polymorphism The capability to define a type in terms of one or more parameters Templates in C++ provide a compile time capability to instantiate a “new” class –Actual parameter is provided for the formal parameter in the definition

C++ List class template template class List{ public: void add(ItemType* item); ItemType* retrieve(Key searchValue); }

Testing perspective Parametric polymorphism supports a different type of relationship from inheritance If the template works for one instantiation, there is no guarantee that it will work for another! –Template code might assume the correct implementations of operations such as destructors –Should be checked during inspection

Development products Chapter 2

Development products Good documentation is critical for successful development and testing A collection of work products that represent the system and the requirements UML is an example of a conceptual modeling language C++ is the programming language

SW end products Code and documentation –User manuals –Maintenance documents Analysis and design models Architectural models Quality requirements UML is used to generate these models

Analysis models

Purpose of analysis is to define the problem to be solved and to determine requirements for a solution to that problem –Domain analysis; focuses on understanding the problem –Application analysis; focus is on specific problem and requirements for a solution (more concrete concepts)

Analysis models Represents a system from the perspective of what it is supposed to do Provides an understanding of the problem and requirements

Use case diagram A use case describes a use of the system by an actor to perform some task Actors represent roles users play with respect to the system Can be expressed at various levels of abstraction

Domain level use cases for arcade games NameDescription Starta player starts a match Pausea player pauses a match Resumea player resumes playing a match Stopa player stops a match

Application level use cases for Brickles arcade game NameDescription Start Bricklesa player starts playing Brickles by starting the application program under Windows Pause Bricklesa player pauses a Brickles match by pressing the mouse button Resume Bricklesa player resumes a Brickles match by releasing the mouse button Move Paddlea player moves the mouse right or left to move the paddle left or right

Use case hierarchy Two hierarchy relationships –Uses; to create more specific use cases –Extends; helps to organize a potentially large number of use cases Use cases represent requirements not software Scenario shows a particular application or instantiation of a use case

Use cases for Brickles Play Brickles Pause Brickles Stop Brickles Break Brick Move Paddle Lose Paddle Win Lose >

Class diagrams Presents a static view of a set of classes and the relationships between the classes Operations, attributes, constraints on relationships shown

Application analysis class diagram for Brickles

Package diagram Groups of classes can be represented as a package Allows easy reuse in the future Dependencies can be shown

Package diagram

State diagrams Describes the behavior of the objects in a class in terms of its observable states and how the object changes states as a result of events that affect the object –State; particular configuration of the values of data attributes –Transition; a change from one state to another

State diagrams –Event; a message arrival –Guard; a condition that must hold before the transition can be made –Action; specifies processing to be done while a transition is in progress –Concurrent state diagram can show groups of states that reflect the behavior of an object from the perspective of two or more independent ways

State diagram for class Timer

Class specification An approach to detail the semantics associated with each operation Object Constraint Language (OCL) is a specification technique used for this purpose Constraints involve –Attributes –Operations –Associations

OCL for operations of PuckSupply PuckSupply size >= 0 and size <= 3 PuckSupply:: PuckSupply( ) pre:- none post:size = 3 AND pucks->forAll( puck: Puck | not puck.inPlay( ) ) PuckSupply:: ~PuckSupply( ) pre:- none post:Puck->size( ) = ) – void PuckSupply:: size( ) const pre:- none post:result = size Puck * PuckSupply::get( ) pre:count > 0 post:result = pucks->asSequence->first AND size = -

Sequence diagram Captures the messaging between objects, object creation, and replies from messages Illustrate process in domain –How common tasks are carried out through the interaction of objects in a scenario Various levels of abstraction Can represent concurrency

Sequence diagrams for Brickles

Activity diagram Where sequence diagrams capture single traces through a set of object interactions, it is difficult to represent iteration and concurrency Activity diagram provides a more comprehensive representation –Flow chart –Petri nets

Activity diagram for move() method in Puck

Activity diagram

Design models

Represent how the software meets requirements Refinements and extensions of the analysis models Means we can reuse and extend test cases developed for analysis models Focus on solution rather than the problem

Class diagram Depicts the kinds of objects that will be created by the software Name, attributes, operations, relationships Adds detail to the analysis class diagram Maintains most of the structure and adds detail for the solution

Design class diagram for Brickles

State diagram Same as those used in analysis Add state diagrams for new classes in the design class diagram and potentially new substates More actions and activities may be shown

State diagram Should ensure test cases for a class that adequately transitions between states and provides proper processing of messages between states Possible to reuse some test cases and test drivers that were developed for testing other classes whose design is based on the same pattern

State diagram for class Timer

Sequence diagrams Used in design to describe algorithms Difference from analysis is the presence of solution level objects in the design diagram Testing perspective; possible errors are violation of contracts, failure to create objects of the correct class, sending of messages for which no navigation is indicated between sender and receiver

Sequence diagram for start- up for a Brickles match

Source code Source code is the final representation for software Translator makes software executable Should be an accurate translation from design models but must be tested

Source code Testing actual code is traditional focus Test as developed as well as completed product Major issues include –Who tests? –What is tested?

Source code Major issues include –When testing is done? –How testing is done? –How much testing is done?

Testing and programming language Programming language has an impact on testing –Enabling of certain types of error –Strong vs weak typing implies more or less errors caught by compiler –Pointer and memory problems –Data hiding –Interface testing

End of lecture