… and after unit testing … Integration Testing, User Interface Testing, Validation, System Testing, …

Testing, testing, … Integration Testing User Interface Testing Validation System Testing

Integration Testing Each of the following modules, shown below in the application's control flow chart, has finally passed unit testing standards. How do you plan to conduct integration testing? A B C D E F G H I J K M L

Integration Methods Top-Down Bottom-Up Depth-First Sandwich using stubs Bottom-Up using drivers Depth-First Sandwich Regression Testing

User Interface Testing User Interface Validation paper prototypes made early are great User Interface Evaluation Expert / Heuristic Evaluation Label Testing is a "rose" still a rose by any other name?

User Interface Testing Cognitive Walkthroughs Usability Testing hopefully the SRS specifically defines usability criteria Random Testing automated random key/mouse presses

What's the problem? http://homepage.mac.com/bradster/iarchitect/color.htm

Dialog Boxes

Cognitive Walkthroughs Participants real end user - sits at keyboard and performs tasks evaluator - takes notes and asks questions, mostly quiet developer - probably hidden or watches video Results effectiveness Will the user associate the next action with the appropriate interface control? Will the user notice an action is available? If the correct action is performed, will the user see that progress is being made toward solution of the task?

Heuristics Simple and natural dialogue Speak the user's language Aesthetic and minimalist design Speak the user's language Minimize user memory load Recognition rather than recall Consistency Feedback Visibility of system status User control and freedom Clearly marked exits Shortcuts and Flexibility Good error messages Help users recognize, diagnose, and recover from errors modified from http://en.wikipedia.org/wiki/Heuristic_evaluation

Validation Testing Alpha Testing Beta Testing

System Testing Recovery Testing Security Testing Stress Testing use software to analyze source code for stack buffer overflow attacks, etc Stress Testing Performance Testing Pressman : 5th Edition

Object-Oriented Testing Why is OOT more difficult than regular testing? inheritance of functions inheritance of data abstract classes Testing the Design is much more important because the code level is very difficult to test. Testing should be aimed at the "class" not the "module" this is because the operation of a module probably depends on how its inherited

And the Moral of the Story is... Use Testing Tools

And the glue that holds it all together… The Test Plan who what when where how

Test Plan Considerations What are the critical or most complex modules? make sure they get integration tested first probably deserve white-box attention Where have you had problems in the past? Third-Party delivered components? What training is required? conducting formal reviews use of testing tools defect report logging

IEEE 829 - Standard for Software Test Documentation Recommends 8 types of testing documents: Test Plan next slide Test Design Specification expected results, pass criteria, ... Test Case Specification test data for use in running the tests Test Procedure Specification how to run each test Test Item Transmittal Report reporting on when components have progressed from one stage of testing to the next Test Log Test Incident Report for any test that failed, the actual versus expected result Test Summary Report management report http://en.wikipedia.org/wiki/IEEE_829

Test Plan Contents (IEEE 829 format) Test Plan Identifier References Introduction Test Items see next slide Software Risk Issues Features to be Tested Features not to be Tested Approach Item Pass/Fail Criteria Suspension Criteria and Resumption Requirements Test Deliverables Remaining Test Tasks Environmental Needs Staffing and Training Needs Responsibilities Schedule Planning Risks and Contingencies Approvals Glossary http://en.wikipedia.org/wiki/Test_Plan

4. Test Items Requirements Specification Design Modules User/Operator Material the user interface User Guide Operations Guide Features response time, data accuracy, security, etc System Validation alpha and beta testing somewhat based on IEEE 829

Software Quality Assurance Activities Reality Check When is more testing not cost effective? SQA + Failure $ Cost of SQA Cost of Failure Software Quality Assurance Activities Optimal Quality Level