2. Thomas L. Gilchrist tomg@tomgtomg.com Testing Basics Set 3: Testing Strategies By Tom Gilchrist Jan 2009

3. Thomas L. Gilchrist tomg@tomgtomg.com Testing Strategy

4. Thomas L. Gilchrist tomg@tomgtomg.com Testing Strategy The basis for the entire testing effort. It should be developed for each project.

5. Thomas L. Gilchrist tomg@tomgtomg.com Test Strategy Questions Who will conduct testing? What type of project? What type of software? When will testing occur? What are the tradeoffs? What is the project’s scope? What are the Critical Success Factors?

6. Thomas L. Gilchrist tomg@tomgtomg.com 1. Types of Projects …different approaches… …combinations of approaches...

7. Thomas L. Gilchrist tomg@tomgtomg.com Types of Projects Traditional System Development Client / Server Interative Development / Prototyping / CASE Object Oriented System Maintenance / Legacy Systems Purchased/Contracted Software (COTS)

8. Thomas L. Gilchrist tomg@tomgtomg.com 2. Types of Software …characterized by use...

9. Thomas L. Gilchrist tomg@tomgtomg.com Types of Software Batch Process Control Graphical User Interface Object Oriented Event Control Procedure Control Expert Systems Multimedia Advanced Mathematical Models Message Processing Diagnostic Software Sensor and Signal Processing

10. Thomas L. Gilchrist tomg@tomgtomg.com Types of Software Simulation Data Base Management Data Acquisition Data Presentation Decision and Planning Aids Pattern and Image Processing Computer system software Software development tools

11. Thomas L. Gilchrist tomg@tomgtomg.com Special Considerations for Testing GUI Applications Client Server Systems Legacy Systems (Object Oriented Systems)

12. Thomas L. Gilchrist tomg@tomgtomg.com Client / Server Testing …Risks… …special considerations...

13. Thomas L. Gilchrist tomg@tomgtomg.com Personal Computing

14. Thomas L. Gilchrist tomg@tomgtomg.com Mainframe Computing Mainframe in a Data Center Data Entry Terminals

15. Thomas L. Gilchrist tomg@tomgtomg.com Client/Server…Distributed Computing Server Client Network

16. Thomas L. Gilchrist tomg@tomgtomg.com Client / Server Risks Organization not ready for technology Inadequate server procedures Improper application selection Inadequate client processing methods Inadequate interface procedures Lack of consistency in processing Improper data usage

17. Thomas L. Gilchrist tomg@tomgtomg.com Client / Server Risks Inadequate security Lack of integration Inadequate client / server administration

18. Thomas L. Gilchrist tomg@tomgtomg.com Client / Server Considerations Control Complexity Rapid Development Timeframes New Skills Required Higher Level of User Involvement

19. Thomas L. Gilchrist tomg@tomgtomg.com Legacy Systems …maintenance techniques...

20. Thomas L. Gilchrist tomg@tomgtomg.com Legacy System Testing Considerations Large, complex programs –Decide what to test Incomplete System Documentation –System requirements –User manuals –System design documentation –Procedure manuals Continuous maintenance –Regression testing requirement

21. Thomas L. Gilchrist tomg@tomgtomg.com 3. What is Project’s Scope? …delimits testing efforts...

22. Thomas L. Gilchrist tomg@tomgtomg.com Project Scope Which Processes? Interfaces? Other Systems? Manual Processes? System Test Boundary System Under Test

23. Thomas L. Gilchrist tomg@tomgtomg.com Project Scope Considerations Scope of new systems development –Automating manual business process? –Which business processes will/will not be affected? –Which business areas will/won’t be affected? –Interfaces to existing systems? –Existing systems will/won’t be affected?

24. Thomas L. Gilchrist tomg@tomgtomg.com Project Scope Considerations Scope of changes to existing systems- –Corrective only? –Maintenance reengineering standards? –Fix to known latent defects in addition to enhancements? –Other systems affected? –Risk of regression?

25. Thomas L. Gilchrist tomg@tomgtomg.com 4. When will testing occur? …life cycle phase...

26. Thomas L. Gilchrist tomg@tomgtomg.com Test Process “V” Diagram Business Need Define Requirements Design System Build System Acceptance Test System Test Integration Test Unit Test

27. Thomas L. Gilchrist tomg@tomgtomg.com Test Process “V” Diagram Business Need Define Requirements Design System Build System Verify Acceptance Test System Test Integration Test Unit Test Verify Validate Validates

28. Thomas L. Gilchrist tomg@tomgtomg.com Test Process “V” Diagram Business Need Define Requirements Design System Build System Verify Acceptance Test System Test Integration Test Unit Test Verify Validate Validates Test Planning Test Objectives Test Execution (Static) Test Execution (Dynamic) & Test Reporting

29. Thomas L. Gilchrist tomg@tomgtomg.com Test Terms Functional –what a system does –based on external test cases Black Box Testing –without the knowledge of how system is constructed –data driven

30. Thomas L. Gilchrist tomg@tomgtomg.com Functional or Black Box Testing ActionActual ResultExpected Result Data InData OutputExpected Data Output 2+2=

31. Thomas L. Gilchrist tomg@tomgtomg.com Test Terms Structural Testing –what a system is –based on internal program logic or –based on system architecture White/Glass Box Testing –testing based on knowledge of internal structure or logic. Usually logic driven.

32. Thomas L. Gilchrist tomg@tomgtomg.com Structural or White Box Testing Logical Test Cases Perform Test Logic Works? If X = curr-date then set next_val to 03 else set next_val to 05

33. Thomas L. Gilchrist tomg@tomgtomg.com Functional AND Structural testing is needed to effectively test systems.

34. Thomas L. Gilchrist tomg@tomgtomg.com When will testing occur? Testing should occur throughout life cycle Requirements Phase Activities –Determine test strategy –Determine adequacy of requirements –Generate functional test conditions Design Phase Activities –Determine consistency of design with requirements –Determine adequacy of design –Generate structural and functional test conditions

35. Thomas L. Gilchrist tomg@tomgtomg.com When will testing occur? Build Phase Activities –Determine consistency with design –Determine adequacy of implementation –Generate structural and functional test conditions for programs / units Test Phase activities –Determine adequacy of test plan –Test application system

36. Thomas L. Gilchrist tomg@tomgtomg.com When will testing occur? Installation Phase Activities –Place tested system into production Maintenance Phase Activities –Modify and retest

37. Thomas L. Gilchrist tomg@tomgtomg.com Prototyping Considerations Requirement Gathering Quick Design Build Prototype Evaluate and Refine Requirement Engineer Product

38. Thomas L. Gilchrist tomg@tomgtomg.com Prototype Testing / Timing Focus on the nature of changing requirements and assumptions Carefully track revisions Remember after prototyping comes construction- and traditional forms of testing still occur.

39. Thomas L. Gilchrist tomg@tomgtomg.com 5. Critical Success Factors …what defines success...

40. Thomas L. Gilchrist tomg@tomgtomg.com Critical Success Factors Correctness Authorization Data Integrity Audit Trail Continuity of Processing Service Levels Access Control Methodology

41. Thomas L. Gilchrist tomg@tomgtomg.com Critical Success Factors Reliability Ease of Use Maintainable Portable Coupling Performance Ease of operations