Presentation is loading. Please wait.

Presentation is loading. Please wait.

CSDP Preparation Course Module V: Software Testing.

Similar presentations


Presentation on theme: "CSDP Preparation Course Module V: Software Testing."— Presentation transcript:

1

2 CSDP Preparation Course Module V: Software Testing

3 Module V. Software Testing2 Specifications The exam specific topics covered in this module are listed below, and are the basis for the outline of its’ content. A.Types of Tests B.Test Levels C.Testing Strategies D.Test Design E.Test Coverage of Code F.Test Coverage of Specifications G.Test Execution H.Test Documentation I.Test Management

4 Module V. Software Testing3 Objectives After completing this module, you should be able to… To determine the purpose of testing To identify the types of tests To identify the process and strategies for performing: Unit testing Integration testing System testing Acceptance testing To discuss test planning and other test documentation

5 Module V. Software Testing4 Organization The organization of information for each specification topic is as follows: Topic Content Slides - detail the important issues concerning each topic and support the module objectives Topic Reference Slides - detail the sources for the topical content and provides additional references for review Topic Quiz Slides - allow students to prepare for the exam

6 Module V. Software Testing5 Introduction Definition of Testing [SW04]: It is an activity performed for evaluating product quality, and for improving it, by identifying defects and problems. Software testing consists of the dynamic verification of the behavior of a program on a finite set of test cases, suitably selected from the usually infinite executions domain, against the expected behavior. Testing is currently considered that the right attitude towards quality is one of prevention: it is obviously much better to avoid problems than to correct them.

7 Module V. Software Testing6 Introduction - 2 Software Testing is the process of executing a computer program for the purpose of finding errors. Software Testing is not -- A means of determining that errors are not present in the program -- A proof that the software performs as required -- A demonstration that all logic paths are executed -- The only means of verifying that the system meets the requirement specifications -- The sole means of demonstrating the quality of the software.

8 Module V. Software Testing7 Introduction - 3 The purpose of Testing Testing is a process of executing a program with the intent of finding errors -- A good test is one that has a high probability of finding undiscovered error -- A good test uses a minimum number of tests to find a maximum number of errors -- A successful test is one that finds undiscovered error Testing cannot show the absence of defects; it can only show that software errors are present.

9 Module V. Software Testing8 Introduction - 4 Major issues in Software Testing Convincing stakeholders the thorough testing is vital to the success of a software project Convincing project managers not to cancel or reduce when the project is running short on time or money Convincing programmers that independent testing is worthwhile Faulty software requirements (resulting in faulty functional testing) Testing is time consuming and expensive (and tempting to reduce to save money and time) Errors are often difficult to reproduce (particularly without proper test planning) Fixing errors found during testing (should be fixed as a separate effort) Lack of explicit test planning, test procedures, and test cases

10 Module V. Software Testing9 Introduction - 5 Testing Principles All functional tests should be traceable to software requirements Tests should be planned at the beginning of the project (requirements phase) The Pareto principles applies (80% of the errors will be found in 20% of the code) Testing begins with software units and ends with systems tests Exhaustive testing is not possible Testing is more effective when conducted by a third party.

11 Module V. Software Testing10 Introduction - 6 Software Testing Fundamentals Testing-Related Terminology a) Definitions of Testing-Related Terminology [SW04, pp5-2] b) Faults vs. Failures [SW04, pp5-2]

12 Module V. Software Testing11 Introduction - 7 Key Issues a) Test selection criteria/Test adequacy criteria (or stopping rules) [SW04, pp5-3] b) Testing effectiveness/Objectives for testing [SW04, pp5-3] c) Testing for defect identification [SW04, pp5-3] d) The oracle problem [SW04, pp5-3] e) Theoretical and practical limitations of testing [SW04, pp5-3] f) The problem of infeasible paths [SW04, pp5-3] g) Testability [SW04, pp5-3] Relationships of testing to other activities

13 Module V. Software Testing12 Introduction References [SW04] Guide to the Software Engineering Body of Knowledge –Chapter 5

14 Module V. Software Testing13 Introduction References - 2 LIST OF STANDARDS (IEEE ) IEEE Std (R2002), IEEE Standard Glossary of Software Engineering Terminology: IEEE, (IEEE829-98) IEEE Std , Standard for Software Test Documentation: IEEE, (IEEE ) IEEE Std , IEEE Standard Dictionary of Measures to Produce Reliable Software IEEE, (IEEE ) IEEE Std (R2003), IEEE Standard for Software Unit Testing: IEEE, (IEEE ) IEEE Std (R2002), IEEE Standard for the Classification of Software Anomalies: IEEE, (IEEE ) IEEE Std , Standard for Software Safety Plans: IEEE, (IEEE ) IEEE/EIA //ISO/IEC12207:1995, Industry Implementation of Int. Std. ISO/IEC 12207:95, Standard for Information Technology-Software Life Cycle Processes, vol. IEEE, 1996.

15 Module V. Software Testing14 Introduction Quiz Two terms often associated with testing are a) Faults and Failures b) Verification and Validation c) Alpha and Beta d) Functional and Structural

16 Module V. Software Testing15 A. Types of Tests The types of tests are: Unit testing Integration testing System testing Acceptance testing

17 Module V. Software Testing16 A. Types of Tests - 2 V Model (Testing Model)

18 Module V. Software Testing17 A. Types of Tests - 3 X Model (Testing Model)

19 Module V. Software Testing18 A. Types of Tests - 4 Unit Testing Unit testing is the testing of the smallest component of the software system, i.e., a unit or a module Unit testing is normally done by the programmer (coder) who coded the module Unit test may or may not be part of the test plan Unit testing normally involves white-box testing Unit testing can be replaced by inspections, peer reviews and argumentative.

20 Module V. Software Testing19 A. Types of Tests - 5 Unit testing normally includes -- Statement testing: Every statement in the module is executed at least once -- Branch testing: Every decision point in the code is executed at least once -- Loop testing: Every loop is repeated at least two times -- Path testing: Every distinct path through the code is executed at least once.

21 Module V. Software Testing20 A. Types of Tests - 6 Verifying the Software Unit Types of Verification Techniques: Testing: The process of executing a computer program for the purpose of finding errors Code Walkthroughs: A peer review for the purpose of finding errors Code Inspections: A peer review for the purpose of finding errors (more formal than walkthroughs) Code reviews: A management/technical review of a software project for the purpose of assessing progress Formal proof techniques: Involves proving, using mathematical arguments, that a program is consistent with its specifications

22 Module V. Software Testing21 A. Types of Tests - 7 Integration testing Integration testing has two purposes: 1.Finding errors in implementing the design. 2.Finding errors in interfacing between components. Tests are derived from the architectural design of the system Integration Testing Methods: Bottom-up testing: The bottom components of the system are tested first – Requires component drivers Top-down testing: The top components of the system are tested first – Requires component stubs Big-bang testing: All components are tested at once – Also called a “smoke test” Sandwich testing: Combines a top-down strategy with a bottom-up strategy – Requires the selection of a target area

23 Module V. Software Testing22 A. Types of Tests - 8 System Testing Functional testing: Do the system’s functional requirements perform as specified in the requirements specifications? Performance testing: Do the system’s nonfunctional requirements perform as specified in the requirements specifications? (Nonfunctional requirements are: performance, external interfaces, design constraints, and quality attributes) Stress testing: The use of input data that is equal to or exceeds the capacity of the system. Regression testing: Selective retesting of a system or component to verify that modifications have not caused unintended effects and that the system or component still complies with its specified requirements. [IEEE Std ]

24 Module V. Software Testing23 A. Types of Tests - 9 Functional Testing Functional testing compares the system’s performance with its requirements Functional testing does “end-to-end testing Each function can be associated with those system components that accomplish it (sometimes called a thread) The tests are normally done one function at a time Functional tests should: -- Have a high probability of detecting a fault -- Use an independent testing team -- Know the expected actions and outputs -- Test both invalid and valid inputs -- Have a stopping criteria

25 Module V. Software Testing24 A. Types of Tests - 10 Types of Performance tests Stress tests: Evaluate the system when all variables are at their extreme settings Volume tests: Evaluate the system for handling large amounts of data Timing tests: Evaluate the requirements to respond to a user and time to perform a function Recovery tests: Evaluate the system’s response to the presence of faults and the effects of failures Quality tests: Evaluate the requirements for such quality attributes as reliability, maintainability, availability, usability, security, etc.

26 Module V. Software Testing25 A. Types of Tests - 11 Stress Testing Modeled after the hardware stress testing in which excess voltage and excess speed are applied to determine what will break first, thereby showing the weak spots in the system. In software, stress testing can involve: -- Adding more terminals to the system than it was original designed for -- Increasing the CPU cycle time beyond “normal” -- Increasing the capacity of data structures This approach may reveal areas that are likely to fail under abnormal conditions

27 Module V. Software Testing26 A. Types of Tests - 12 Regression Testing Regression testing is a selective retesting of a system or component to verify that modifications have not caused unintended effects and that the system or component still complies with its specified requirements. [IEEE Std ] A regression test identifies new results that have been introduced as old faults are corrected It is primarily used after system maintenance is performed

28 Module V. Software Testing27 A. Types of Tests - 13 Acceptance Test A test run by the customer or on behalf of the customer to determine acceptance or rejection of the system Four types of acceptance tests: -- Benchmark test : A set of test cases that reflect expected uses of the system -- Pilot test : A system installed on an experimental basis -- Alpha test : An in-house system test with the developing staff assuming the role of the user -- Beta test : An external system test with a select subset of the eventual users of the system

29 Module V. Software Testing28 A. References [SW04] Guide to the Software Engineering Body of Knowledge - Chapter 5 [IEEE Std ]

30 Module V. Software Testing29 A. Quiz ________ refers to ensuring correctness from phase to phase of the software development cycle. a) Verification b) Validation c) Testing d) None of the above _________ involves checking the software against the requirements a) Verification b) Validation c) Testing d) None of the above

31 Module V. Software Testing30 B. Test Levels Software testing is usually performed at different levels along the development and maintenance processes. That is to say, the target of the test can vary: a single module, a group of such modules (related by purpose, use, behavior, or structure), or a whole system. [SW04, pp5-3] Three big test stages can be conceptually distinguished, namely Unit, Integration, and System. Unit Testing: Unit testing verifies the functioning in isolation of software pieces which are separately testable. [SW04, pp5-3]

32 Module V. Software Testing31 B. Test Levels - 2 Integration testing is the process of verifying the interaction between software components. [SW04, pp5-4] System testing is concerned with the behavior of a whole system. System testing is usually considered appropriate for comparing the system to the non-functional system requirements, such as security, speed, accuracy, and reliability. [SW04, pp5-4]

33 Module V. Software Testing32 B. Test Levels - 3 Objectives of Testing: Testing is conducted in view of a specific objective, which is stated more or less explicitly, and with varying degrees of precision. Stating the objective in precise, quantitative terms allows control to be established over the test process. [SW04, pp5-4] The sub-topics listed below are some kinds of testing those most often cited in the literature. Acceptance/qualification testing [SW04, pp5-4] Installation testing [SW04, pp5-4] Alpha and beta testing [SW04, pp5-4]

34 Module V. Software Testing33 B. Test Levels - 4 Conformance testing/Functional testing/Correctness testing [SW04, pp5-4] Reliability achievement and evaluation [SW04, pp5-4] Regression testing [SW04, pp5-4] Performance testing [SW04, pp5-5] Stress testing [SW04, pp5-5]

35 Module V. Software Testing34 B. Test Levels - 5 Back-to-back testing [SW04, pp5-5] Recovery testing [SW04, pp5-5] Configuration testing [SW04, pp5-5] Usability testing [SW04, pp5-5] Test-driven development [SW04, pp5-5]

36 Module V. Software Testing35 B. References - 1 [SW04] Guide to the Software Engineering Body of Knowledge - Chapter 5

37 Module V. Software Testing36 B. References - 2 LIST OF STANDARDS (IEEE ) IEEE Std (R2002), IEEE Standard Glossary of Software Engineering Terminology: IEEE, (IEEE829-98) IEEE Std , Standard for Software Test Documentation: IEEE, (IEEE ) IEEE Std , IEEE Standard Dictionary of Measures to Produce Reliable Software IEEE, (IEEE ) IEEE Std (R2003), IEEE Standard for Software Unit Testing: IEEE, (IEEE ) IEEE Std (R2002), IEEE Standard for the Classification of Software Anomalies: IEEE, (IEEE ) IEEE Std , Standard for Software Safety Plans: IEEE, (IEEE ) IEEE/EIA //ISO/IEC12207:1995, Industry Implementation of Int. Std. ISO/IEC 12207:95, Standard for Information Technology-Software Life Cycle Processes, vol. IEEE, 1996.

38 Module V. Software Testing37 B. Quiz ________ is associated with formal proofs of correctness a) Validation b) Verification c) Testing d) All the above ________ is concerned with executing the software with test data. a) Validation b) Verification c) Testing d) All the above

39 Module V. Software Testing38 C. Testing Strategies Test techniques Based on the software engineer’s intuition and experience a) Ad hoc testing [SW04, pp5-5] b) Exploratory testing [SW04, pp5-5] Specification-based techniques a) Equivalence partitioning [SW04, pp5-5] b) Boundary-value analysis [SW04, pp5-5] c) Decision table [SW04, pp5-6] d) Finite-state machine-based [SW04, pp5-6] e) Testing from formal specifications [SW04, pp5-6] f) Random testing [SW04, pp5-6]

40 Module V. Software Testing39 C. Testing Strategies - 2 Test techniques Code-based techniques a) Control flow-based criteria [SW04, pp5-6] b) Data flow-based criteria [SW04, pp5-6] c) Reference models for code-based testing (flow graph, call graph) [SW04, pp5-6] Fault-based techniques [SW04, pp5-6] a)Error guessing [SW04, pp5-6] b) Mutation testing [SW04, pp5-6] Usage-based techniques a) Operational profile [SW04, pp5-7] b) Software Reliability Engineered Testing[SW04, pp5- 7]

41 Module V. Software Testing40 C. Testing Strategies - 3 Test techniques Techniques based on the nature of the application[SW04,pp5-7] Object-oriented testing Component-based testing Web-based testing GUI testing Testing of concurrent programs Protocol conformance testing Testing of real-time systems Testing of safety-critical systems Selecting and combining techniques [SW04,pp5-7] a) Functional and structural b) Deterministic vs. random

42 Module V. Software Testing41 C. Testing Strategies - 4 Functional Testing Functional Testing: Functional testing addresses itself to whether the program produces the correct output. -- Focuses on the functional requirements of the software, also called “black box testing” Functional strategy uses only the requirements defined in the specification as the basis for testing Attempts to find errors of the following categories -- Incorrect or missing functions -- Interface errors -- Errors in data structures or external database access -- Performance errors -- Initiation and termination errors Bases the test on the external view of the system

43 Module V. Software Testing42 C. Testing Strategies - 5 Structural Testing Structural testing: The testing strategy is based on deriving test data from the structure of a system. The structural strategy is based on the detailed design. -- Focuses on the control structure of the system design, also called “white box or glass box testing”. Focuses on -- Path testing: Exercising all independent paths within a module at least once -- Branch testing: Exercising all logical decisions on both their true and false sides -- Loop testing: Executing all loops at their boundaries and their operational bounds -- Exercising internal data structures to insure their correctness and availability. Bases the test on the internal structures of the software

44 Module V. Software Testing43 C. Testing Strategies - 6 Static Analysis A testing technique that does not involve the execution of the software with data. It directly analyzes the form and structure of a product without executing the product. Use of static analysis tools to scan the source text of a program and detect possible faults and anomalies. Includes -- Program proving. -- Symbolic execution. -- Anomaly analysis

45 Module V. Software Testing44 C. Testing Strategies - 7 Dynamic analysis Dynamic analysis requires that the software be executed and relies on instrumenting the program to measure internal data and logic states as well as outputs. The process of evaluating a program based on execution of the program Involves execution or simulation of a development activity product to detect errors by analyzing the response of a product to sets of input data. The software is exercised through the use of test cases The resulting data is compared with the computed data to check for errors

46 Module V. Software Testing45 C. Testing Strategies - 8 Dynamic Analysis Dynamic functional: This technique executes test cases without giving consideration to the detailed design of the software. Classified into: -- Domain testing -- Random testing -- Adaptive perturbation testing -- Cause-effect graphing

47 Module V. Software Testing46 C. Testing Strategies - 9 Dynamic Analysis Dynamic-Structural: This technique executes the test cases with creation of the test cases basing upon an analysis of the software. Classified into: -- Domain testing -- Computation testing -- Automatic test data generation -- Mutation analysis

48 Module V. Software Testing47 C. References [SW04] Guide to the Software Engineering Body of Knowledge - Chapter 5

49 Module V. Software Testing48 C. References - 2 LIST OF STANDARDS (IEEE ) IEEE Std (R2002), IEEE Standard Glossary of Software Engineering Terminology: IEEE, (IEEE829-98) IEEE Std , Standard for Software Test Documentation: IEEE, (IEEE ) IEEE Std , IEEE Standard Dictionary of Measures to Produce Reliable Software IEEE, (IEEE ) IEEE Std (R2003), IEEE Standard for Software Unit Testing: IEEE, (IEEE ) IEEE Std (R2002), IEEE Standard for the Classification of Software Anomalies: IEEE, (IEEE ) IEEE Std , Standard for Software Safety Plans: IEEE, (IEEE ) IEEE/EIA //ISO/IEC12207:1995, Industry Implementation of Int. Std. ISO/IEC 12207:95, Standard for Information Technology-Software Life Cycle Processes, vol. IEEE, 1996.

50 Module V. Software Testing49 C. Quiz What kind of testing has been included under both structural and functional strategies a) Computation testing b) Domain testing c) Random testing d) None of the above

51 Module V. Software Testing50 D. Test Design Attributes of a Test Design Determine the features to be tested (or not tested) Select the test cases to be used Select the process to test the features Determine the pass/fail criteria Design the software test as soon as possible after the establishment of requirements. This helps -- Non-testable requirements to be found -- Quality to become built in -- Costs to be reduced -- Time saved

52 Module V. Software Testing51 D. References [IEEE standard , Standard for software Test Documentation.] SMC 2003 IEEE CBT/Testing 07 Thayer & Dorfman, Software Engineering Volume1: The development Process Coward, P. David – “A Review of Software Testing”

53 Module V. Software Testing52 D. Quiz An engineer is tasked to verify a software release for a mission critical system. The plan is for the release of software for verification to occur on a Monday, with verification complete the following Friday. The release turns out not to be available until Thursday. The best route for the engineer is to: [a] verify release criteria regardless of time line. [b] do whatever testing they can by Friday. [c] volunteer to work the weekend. [d] relax release criteria.

54 Module V. Software Testing53 E. Test Coverage of Code Test coverage of code : The amount of code actually executed during the test process. It is stated as a percentage of the total instructions executed or paths traversed. Statement coverage is the number of statements exercised by the test set divided by the total number in the module being measured [Also known as the test effectiveness ratio] A 100% cover is achieved by having each line of code exercised at least once [100% coverage is required but this alone is not a sufficient indicator of a good test] Manual development of test data results in coverage of as low as 60-80%

55 Module V. Software Testing54 E. Test Coverage of Code - 2 Automated Test Tools Static analysis -- Code analyzers : Check for proper syntax -- Structure checker : Checks for structural flaws -- Data analyzers : Check for errors in data structures -- Sequence checker : Checks sequence of events Dynamic analysis : Monitors the activities of the program under operation Test execution tools -- Capture and replay : Captures the activities of the test and replays them as necessary -- Test case generators : Develops test cases for complete test coverage (test case generators can result in coverage of over 90%)

56 Module V. Software Testing55 E. References SMC 2003 IEEE CBT/Testing 07 Sommerville, Ian, Software Engineering, 5th edition, Addison-Wesley, 1996, chapter 18. Thayer & Dorfman, Software Engineering Volume1: The development Process Coward, P. David – “A Review of Software Testing”

57 Module V. Software Testing56 E. Quiz Which of the following do NOT affect the accuracy of the reliability estimate during statistical testing? I. The validity of the usage profile. II. The number of test cases executed. III. The programming language used to implement the code. IV. The cyclomatic complexity of the code. [a] I and II only. [b] II and III only. [c] III and IV only. [d] I and III only.

58 Module V. Software Testing57 F. Test Coverage of Specifications Requirements Specifications Test data is derived from the software requirements specifications Test coverage : The degree to which a given test or set of tests addresses all specified requirements for a given system or component Sample coverage of the functional domain is achieved by having at least one sample value from each functional equivalence partition Boundary coverage of the functional domain is achieved by having at least one test case on each side of the equivalence partition boundary Decision table represents logical relationships between input conditions and output actions Random testing in which tests are generated purely random

59 Module V. Software Testing58 F. Test Coverage of Specifications - 2 Design Description Test data is derived from the software design descriptions Path testing : Exercising all independent paths within a module Branch testing : Exercising all logical decisions on both their true and false sides Loop testing : Executing all loops at their boundaries and their operational bounds Interface testing : Test interfaces between modules and group of modules Code based testing : Test data derived from a flowchart

60 Module V. Software Testing59 F. References SMC 2003 IEEE CBT/Testing 07 IEEE/EIA , Standard for Information Technology—Software life cycle processes, 6.3 Quality Assurance Process Thayer & Dorfman, Software Engineering Volume1: The development Process Coward, P. David – “A Review of Software Testing”

61 Module V. Software Testing60 F. Quiz Quality assurance may be applied to: I. Requirements II. Design III. Code IV. Testing [a] I and II only [b] I, II, and III only [c] I, II, III, and IV [d] IV only

62 Module V. Software Testing61 G. Test Execution Q.Why not check All of the paths? There are too many paths (each “branching” or “loop” creates one to many more paths that can be reasonably tested in a finite period of time) There can be a number of infeasible paths (a path that cannot be reached by a test case)

63 Module V. Software Testing62 G. Test Execution - 2 Infeasible paths: An infeasible path is one which cannot be executed due to incorrect parameters in a branching statement For example: 1. Begin 2. Read A 3. If A > Then B = B Else C = C If A < Then D = D End. What path can never be executed?

64 Module V. Software Testing63 G. Test Execution - 3 Dead code: Code that is isolated by a “jump” statement and is never executed. Example: X1 Go to “y” X2 …… X3 …… X Y …… If the code block X2 – X4 is not entered by another “jump” statement, this block of code is “dead” or “isolated” This is typically caused by a programmer who has created a “patch” to fix a problem and is too lazy to remove the old code. This is an “excellent” place to hide a virus

65 Module V. Software Testing64 G. References SMC 2003 IEEE CBT/Testing 07 Thayer & Dorfman, Software Engineering Volume1: The development Process Coward, P. David – “A Review of Software Testing”

66 Module V. Software Testing65 G. Quiz ________ testing addresses itself to whether the program produces the correct output. a) Regression b) Functional c) Static d) System

67 Module V. Software Testing66 H. Test Documentation Software Testing Documents Test plan : Describes the scope, approach, resources and schedule for testing Test design specification : Specifies the details of the test approach Test procedure specification : Specifies the sequence of actions for the execution of tests Test case specification : Specifies input, predicted results, and a set execution conditions for each test. Test log : A chronological record of the test incident report - reports any event which requires investigation. A log is also called a Discrepancy Report (DR) Test summary report : Summarizes testing activities and results.

68 Module V. Software Testing67 H. Test Documentation - 2 Testing Documents 1.Test plan  2.Test Design  3.Test Cases  4.Test procedures  5. Test Test dataRequirementsDesignImplementationTest Unit Testing12,34,5 Integration Testing 123,45 System Testing 123,45 Acceptance Testing 1234,5

69 Module V. Software Testing68 H. Test Documentation - 3 Test Documentation Integration

70 Module V. Software Testing69 H. References IEEE Standard , Standard for Software Test Documentation. SMC 2003 IEEE CBT/Testing 07 Thayer & Dorfman, Software Engineering Volume1: The development Process Coward, P. David – “A Review of Software Testing”

71 Module V. Software Testing70 H. Quiz ________ testing is the name given to the functional testing that follows modification. a) Regression b) Functional c) Static d) Integration

72 Module V. Software Testing71 I. Test Management Test Planning Establishing the major tasks in conducting the test. For ex: -- Develop the test strategy -- Develop the process -- Develop the test cases -- Conduct the test -- Make the final report Identifying the test items Establishing any constraints in conducting the tests Developing a testing schedule Computing the cost and resources required to conduct the tests Identify the major testing risks Identifying the software and hardware necessary to conduct the tests

73 Module V. Software Testing72 I. Test Management - 2 The Role of the Software System Engineer Review and approve the test plan Determine the quantity and quality of testing to be used Develop the acceptance criteria Manage and control the test readiness review (TRR) to determine: -- If the software is ready for testing -- if the test specifications and procedures are adequate to test the software Observe and/or conduct integration, system, and acceptance tests Review and verify the test results Turn the final system over to the customer

74 Module V. Software Testing73 I. Test Management - 3 Testing Effectiveness Reduce the number of errors passed on to the customer Reduce the cost of effective testing -- Example : Inspections are more effective than unit testing in finding errors (argumentative) Test management pitfalls: -- Absence of testing policy or strategy -- Failing to plan for the testing -- Insufficient time allocated for testing -- Reduce testing when the project begins to overrun its budget -- Ineffective testing ( testing strategies not suitable for the application)

75 Module V. Software Testing74 I. Test Management - 4 Independent Test Team Testers need to take an independent view of the software to be tested -- A software developer can only find a small amount his/her own errors (leaving the rest of them to be delivered) -- It is difficult to effectively test “destructively” Solution: Use an independent test team -- An independent test team is an organization independent from the authority of the development manager who is given responsibility for testing the software being developed.

76 Module V. Software Testing75 I. Test Management - 5 Independent Test Team Advantages of an independent test team: -- Does not have a vested interest in the program -- Brings a new dimension to the software system -- Independently assists in the verification of the software design -- Has a goal to find errors in the system Disadvantages of an independent test team: -- Team must learn the system -- Independent teams approach gives the appearance of being more expensive than the non-independent test team -- Could cause resentment with the development team, thereby resulting in their lack of cooperation

77 Module V. Software Testing76 I. Test Management - 6 Essential Ingredients of a Good Test Team Detailed understanding of the system System and testing experience Knowledge of internal structure of the system to be tested Ability to create test cases Ability to select test strategies The use of comprehensive and sophisticated testing tools and simulations.

78 Module V. Software Testing77 I. References SMC 2003 IEEE CBT/Testing 07 Thayer & Dorfman, Software Engineering Volume1: The development Process Coward, P. David – “A Review of Software Testing”

79 Module V. Software Testing78 I. Quiz The two prominent strategy dimensions are a) Structural and functional b) function/structural and static/dynamic c) Static and dynamic d) Static and Integration

80 Module V. Software Testing79 ADDITIONAL MATERIAL Test Related Measures Evaluation of the program under test [SW04,pp5-7] Evaluation of the tests performed [SW04,pp5-8] Test Process Practical Considerations [SW04, pp5-8] Test Activities [SW04, pp5-8]

81 Module V. Software Testing80 Test Related Measures Evaluation of the program under test [SW04,pp5-7] a)Program measurements to aid in planning and designing testing b) Fault types, classification, and statistics c) Fault density d) Life test, reliability evaluation e) Reliability growth models

82 Module V. Software Testing81 Test Related Measures - 2 Evaluation of the tests performed [SW04,pp5-8] a) Coverage/thoroughness measures b) Fault seeding c) Mutation score d) Comparison and relative effectiveness of different techniques

83 Module V. Software Testing82 References [SW04] Guide to the Software Engineering Body of Knowledge - Chapter 5

84 Module V. Software Testing83 References - 2 LIST OF STANDARDS (IEEE ) IEEE Std (R2002), IEEE Standard Glossary of Software Engineering Terminology: IEEE, (IEEE829-98) IEEE Std , Standard for Software Test Documentation: IEEE, (IEEE ) IEEE Std , IEEE Standard Dictionary of Measures to Produce Reliable Software IEEE, (IEEE ) IEEE Std (R2003), IEEE Standard for Software Unit Testing: IEEE, (IEEE ) IEEE Std (R2002), IEEE Standard for the Classification of Software Anomalies: IEEE, (IEEE ) IEEE Std , Standard for Software Safety Plans: IEEE, (IEEE ) IEEE/EIA //ISO/IEC12207:1995, Industry Implementation of Int. Std. ISO/IEC 12207:95, Standard for Information Technology-Software Life Cycle Processes, vol. IEEE, 1996.

85 Module V. Software Testing84 Quiz Aiming to demonstrate that there are no faults is a) constructive. b) destructive c) Both d) none of the above

86 Module V. Software Testing85 Test Process Practical Considerations [SW04, pp5-8] a) Attitudes/Egoless programming b) Test guides c) Test process management d) Test documentation and work products e) Internal vs. independent test team f) Cost/effort estimation and other process measures g) Termination h) Test reuse and test patterns

87 Module V. Software Testing86 Test Process - 2 Test Activities [SW04, pp5-8] a) Planning b) Test-case generation c) Test environment development d) Execution e) Test results evaluation f) Problem reporting/Test log g) Defect tracking

88 Module V. Software Testing87 References [SW04] Guide to the Software Engineering Body of Knowledge - Chapter 5

89 Module V. Software Testing88 References - 2 LIST OF STANDARDS (IEEE ) IEEE Std (R2002), IEEE Standard Glossary of Software Engineering Terminology: IEEE, (IEEE829-98) IEEE Std , Standard for Software Test Documentation: IEEE, (IEEE ) IEEE Std , IEEE Standard Dictionary of Measures to Produce Reliable Software IEEE, (IEEE ) IEEE Std (R2003), IEEE Standard for Software Unit Testing: IEEE, (IEEE ) IEEE Std (R2002), IEEE Standard for the Classification of Software Anomalies: IEEE, (IEEE ) IEEE Std , Standard for Software Safety Plans: IEEE, (IEEE ) IEEE/EIA //ISO/IEC12207:1995, Industry Implementation of Int. Std. ISO/IEC 12207:95, Standard for Information Technology-Software Life Cycle Processes, vol. IEEE, 1996.

90 Module V. Software Testing89 Quiz Aiming to find faults is a _________process. a) destructive b) constructive. c) both d) None of the above


Download ppt "CSDP Preparation Course Module V: Software Testing."

Similar presentations


Ads by Google