1. Chapter 10 – Software Testing

2. Software Testing Definition Purpose Process of finding errors
Ultimate review of specifications, design & coding
Purpose
guarantee all elements of an application work together properly, function as expected, and meet performance criteria

3. Testing ground rules
Testing is the process of executing a program with the intent of finding errors
A good test case is one that has a high probability of finding an error
A successful test is one that uncovers an error
Benefits
Uncover errors in software
Demonstrates that software functions appear to be working according to specification
Demonstrate performance requirements have been met
Provide indication of software reliability and quality
Testing cannot show the absence of defects

4. Verification – Validation
Verification: to ensure that the software correctly implements a specific function.
“Are we building the system right ?”
Validation: to ensure that the software correspond to customer requirement.
“Are we building the right system ?”

5. Testing Who tests What is tested Why test How (test cases) Programmers
Testers
Users
What is tested
Unit testing
Integration testing
System testing
Validation testing
Why test
Acceptance
Conformance
Configuration
Performance, stress
How (test cases)
Intuition
Specification based (black box)
Code based (white-box)
Existing cases (regression)
Faults

6. Testing Strategy
We begin by ‘testing-in-the-small’’ and move toward ‘testing-in-the-large’
For conventional software
The module (component) is our initial focus
Integration of modules follows
For OO software
our focus when “testing in the small” changes from an individual module (the conventional view) to an OO class that encompasses attributes and operations and implies communication and collaboration
Software Engineering: A Practitioner’s Approach, 6/e by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005

7. Unit Test Environment driver test cases Module software engineer stub
RESULTS
interface
local data structures
boundary conditions
independent paths
error handling paths
test cases

8. Unit Test Methods Black-Box Methods White-Box Methods
Statement coverage
Test cases cause every line of code to be executed
Branch coverage
Test cases cause every decision point to execute
Path coverage
Test cases cause every independent code path to be executed
Black-Box Methods
Equivalence partitioning Divide input values into equivalent groups
Boundary Value Analysis Test at boundary conditions

9. EX. Program Control Graph of computeFine()
False
True 1 2 3 4 5 6 7 9

10. Branch Coverage of computeFine()
Test Case #1
False
True 1 2 3 4 5 6 7
Test Case #2
False
True 1 2 3 4 5 6 7 10

11. Distinct Paths of computeFine()
False
True 1 2 3 4 5 6 7
Path #2
Path #3
False
True 1 2 3 4 5 6 7
Path #4
False
True 1 2 3 4 5 6 7
Path #1
False
True 1 2 3 4 5 6 7 11

12. Test Cases for Distinct Paths of computeFine()
daysLate
printON
Path 1
TRUE 2 60
FALSE 3 4 12

13. Basis Path Testing First, we compute the cyclomatic complexity:
number of simple decisions + 1 or
number of enclosed areas + 1
In this case, V(G) = 4
E - N + P =
Software Engineering: A Practitioner’s Approach, 6/e by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005

14. Cyclomatic Complexity
A number of industry studies have indicated
that the higher V(G), the higher the probability
or errors.
V(G)
modules
modules in this range are
more error prone
Software Engineering: A Practitioner’s Approach, 6/e by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005

15. Black-Box Testing requirements output input events
Software Engineering: A Practitioner’s Approach, 6/e by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005

16. Black-Box Testing How is functional validity tested?
How is system behavior and performance tested?
What classes of input will make good test cases?
Is the system particularly sensitive to certain input values?
How are the boundaries of a data class isolated?
What data rates and data volume can the system tolerate?
What effect will specific combinations of data have on system operation?
Software Engineering: A Practitioner’s Approach, 6/e by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005

17. Black-Box Testing
Test cases can not be selected to be ‘optimal’ as the structure of the system is unknown.
Equivalence classes:
The input domain is divided into a number of classes. The system is then tested with one sample from each class.
Boundary values:
Selecting test cases at the edge of the class. (Most errors occur at the boundaries of the input-domain)

18. Equivalence partitioning
Partition system inputs and outputs into ‘equivalence sets’
If input is a 5-digit integer between 10,000 and 99,999, equivalence partitions are
<10,000, 10,000-99,999 and > 10, 000
Choose test cases at the boundary of these sets

19. Integration Testing Strategies
Options:
The “big bang” approach
An incremental construction strategy
Big bang ทดสอบทั้งหมดทีเดียวพร้อมกันทั้งชุด
Incremental ค่อยๆทดสอบไปทีละส่วน โดยใช้ driver-stub มาช่วยเป็นตัวหลอก
Software Engineering: A Practitioner’s Approach, 6/e by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005

20. Top Down Integration top module is tested with stubs
stubs are replaced one at a time,
"depth first"
as new modules are integrated,
some subset of tests is re-run A B C D E F G
Software Engineering: A Practitioner’s Approach, 6/e by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005

21. Bottom-Up Integration
drivers are replaced one at a time,
"depth first"
worker modules are grouped into
builds and integrated A B C D E F G
cluster
Software Engineering: A Practitioner’s Approach, 6/e by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005

22. Regression testing
Regression testing is the repetition of previously executed test cases for the purpose of finding defects in software that previously passed the same set of tests.
A regression test is applied to a new version to verify that it still perform the same functions as the old version.
A set of essential /representative tests from previous version is repeated.
Test of functionality most likely affected by the changes.
Test of the new/modified module.
เป็นการทดสอบระบบเมื่อมีการแก้ไขบางส่วนของโปรแกรม ก็ให้ทำการทดสอบใหม่ทั้งหมดอีกครั้ง
เพื่อดูว่าส่วนที่เปลี่ยนแปลงเข้ากันได้กับส่วนที่เหลือหรือไม่ และยังคงทำงานตามที่เคยทำได้ อยู่หรือไม่

23. Object-Oriented Testing
begins by evaluating the correctness and consistency of the OOA and OOD models
testing strategy changes
the concept of the ‘unit’ broadens due to encapsulation
integration focuses on classes and their execution across a ‘thread’ or in the context of a usage scenario
validation uses conventional black box methods
test case design draws on conventional methods, but also encompasses special features
Software Engineering: A Practitioner’s Approach, 6/e by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005

24. OOT Strategy class testing is the equivalent of unit testing
operations within the class are tested
the state behavior of the class is examined
integration applied three different strategies
thread-based testing — integrates the set of classes required to respond to one input or event
use-based testing — integrates the set of classes required to respond to one use case
cluster testing — integrates the set of classes required to demonstrate one collaboration
Software Engineering: A Practitioner’s Approach, 6/e by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005

25. System test Test according to system specifications (validation).
Test for each function/requirement at the interface.
Thread testing
Test according to user scenarios/use cases.

26. Acceptance test
Purpose: convince the customer that the system correspond to the system requirement.
FAT (factory acceptance test)
A complete test according to system requirement carried out in the software house.
SAT (site acceptance test)
A complete test according to system requirement carried out after the system has been installed at the customer.

27. Operation/performance test
Timing test
Stress test
Volume test
Recovery test
Availability test
Such a test is normally carried out after the system has been put into operation (normally also coincide with the final payment for the system).

28. Comparison Testing
Used only in situations in which the reliability of software is absolutely critical (e.g., human-rated systems)
Separate software engineering teams develop independent versions of an application using the same specification
Each version can be tested with the same test data to ensure that all provide identical output
Then all versions are executed in parallel with real-time comparison of results to ensure consistency
Software Engineering: A Practitioner’s Approach, 6/e by R.S. Pressman & Associates, Inc., copyright © 1996, 2001, 2005

29. Test Only after the completion of the test plan the system and its
requirements were fully
understood by the
software developer.