1. Software Test Coverage TESTTCOV 1 Test Coverage Measures Code Coverage Measurement Levels of Code Coverage Method Coverage Method, Statement, Branch & Path Coverage White Box Test Coverage Data Coverage Measurement Levels of Data Coverage Class Coverage Interface Coverage State Coverage Data Table Coverage 2 3 6 7 5 4 8 9 10 13 Data Coverage Report Functional Coverage Measurement Levels of Functional Coverage Black Box Test Coverage Functional Test Coverage Predicted Error Coverage Inheriting Test Coverage Non-Transitivity of Test Coverage CRUD Object Access Coverage CRUD Object Access Coverage Matrix Document Coverage Differences in Test Coverage 14 15 18 19 17 16 20 21 22 1123 1224

2. Test Coverage Measures TESTTCOV-1 Test Coverage Functional Coverage Code Coverage Data Coverage Object Coverage Attribute Coverage Value Coverage State Coverage Function Coverage Function Outcome Coverage Function Chain Coverage Function State Coverage Method Coverage Statement Coverage Branch Coverage Path Coverage

3. Code Coverage Measurement TESTTCOV-2 Method Coverage Path Coverage Branch Statement Test Coverage = 100% could mean: Every Method was invoked Every Statement was executed Every Branch was traversed Every Control Path was traversed

4. Levels of Code Coverage TESTTCOV-3 Method coverage Interface coverage Statement coverage Branch coverage Path coverage 1 2 3 4 5 Increasing Reliability Increasing Costs

5. A_RetcodeA_Method (Param_A1, Param_A2, Param_A3) { Invoke Method B_RetcodeB_Method (Param_B1, Param_B2, Param_B3) { if (Param_B1 > Attr_B9) return 1; } Statement Coverage C_RetcodeC_Method (Param_C1, Param_C2, Param_C3) { if (Param_C1 > Attr_C9) return 1 else continue; return 0; } Branch Coverage 1 Test 2 Tests Method Coverage TESTTCOV-4

6. F1 B1 B2 B3 A4 E2 E1 A2 A3 Method Coverage: (1 test case) Statement Coverage: (2 test cases) Branch Coverage: (3 test cases) Path Coverage: (5 test cases) F1, B1, E1 (Method Coverage) F1, B1, E1 (Statement Coverage) F1, B1, B2, A2, B3, A3, A4, E2 F1, B1, E1 (Branch Coverage) F1, B1, B2, A2, B3, A3, A4, E2 F1, B1, B2, B3, A4, E2 F1, B1, E1 (Path Coverage) F1, B1, B2, A2, B3, A3, A4, E2 F1, B1, B2, A2, B3, A4, E2 F1, B1, B2, B3, A3, A4, E2 F1, B1, B2, B3, A4, E3 Test Cases Method, Statement, Branch and Path Coverage TESTTCOV-5

7. C1=All Methods are executed at least once. (Method Coverage) C2=All Statements of all Methods are executed at least once. (Statement Coverage) C3=All Branches of all Methods are executed at least once. (Branch Coverage) D1=All Attributes of an Object are altered. (Attribute Coverage) D2=All Attributes of an Object take on all specified values. (Value Coverage) D3=All Combinations of all values occur. (State Coverage) S1=All Messages are sent at least once. (Interface Coverage) S2=All Parameters of all Messages are altered. (Parameter Coverage) S3=All Combinations of all Parameters occur. (Interface State Coverage) White-Box Coverage Measures (Class Test) TESTTCOV-6

8. Data Coverage Measurement TESTTCOV-7 Object Coverage State Coverage Value Attribute Test Coverage = 100% could mean: Every Object was instantiated Every Attribute was altered Every relevant value was set Every possible state was generated.

9. Levels of Data Coverage TESTTCOV-8 1 2 3 4 5 Increasing Reliability Increasing Costs Window/File/DB Coverage Object Coverage Attribute Coverage Value Coverage State Coverage

10. Object StatesMethods AccountNumber AccountOwner AccountBalance AccountStatus CreditLimit Open_Account Deposit_Money Withdraw_Money Transfer_Money Close_Account Account non existant AccountBalance = 0 AccountBalance = plus AccountBalance = minus, but < CreditLimit AccountBalance = minus and > CreditLimit Account closed Attributes How many test cases are required to test all Methods with all Object states? Class Coverage TESTTCOV-9

11.  Parameter Coverage = the Interface is generated one time with a repesentative value for each individual Parameter  Value Coverage = the Interface is generated as many times as there are representative values for each Parameter.  State Coverage = the Interface is generated as many times as there are combinations of representative values for all Parameters. Interface Coverage TESTTCOV-10

12. All Paths All Paths All Branches All Branches All State Transitions All State Transitions All States All States All Events All Events Subsumption Hierarchy State Coverage TESTTCOV-11

13. 1 OBJECT-A 2 ATTR-A1 'A' 'B' 'C' 2 ATTR-A2 100 200 300 400 2 ATTR-A3 'XYZ''ABC''EFG' 2 ATTR-A4 -1 0 +1 1 PANEL-B 2 FIELD-B1 50 100 150 2 FIELD-B2 100 -50 0 2 FIELD-B3 'XY' 'AB' 'EF' 2 FIELD-B4 'B' 'A' 'C' Value Coverage: Every relevant Value of every Attribute & Field should be set at least one time. (you need as many Objects as the longest array of values) Data Table Coverage TESTTCOV-12

14. Data Coverage Report TESTTCOV-13 | DATA COVERAGE REPORT | | MODUL NAME: SVVL104 DATUM: 98.05.17 | +--------------------------------------------------------------------------------------------+ | DATA NAME |RECORD NAME |BASE NAME|PREDIC.|INPUT|OUTPUT|NOT USED| +--------------------------------------------------------------------------------------------+ | LISTNUMMER |FILLER |SVVL104 | | * | * | * | | K-ENDE-DER-LISTE |ALLGEMEINE-VIS-KONSTANTEN|SVVL104 | | * | | * | | CURRENT-DATE-MONAT |ALLGEMEINE-VIS-VARIABLEN |SVVL104 | | | * | | | CURRENT-DATE-TAG |ALLGEMEINE-VIS-VARIABLEN |SVVL104 | | | * | | | CURRENT-DATE-JAHR |ALLGEMEINE-VIS-VARIABLEN |SVVL104 | | | * | | | SH-ENDE-DER-LISTE |ALLGEMEINE-VIS-SCHALTER |SVVL104 | * | | * | | | SUMME-AWGR-VERK-MG |LISTSTEUERDATEN |SVVL104 | | | * | | | FIRMA |ABSATZWEGGRUPPENSTATISTIK|SVVL104 | | * | * | | | TEXT-ENDE-DER-AUSWERTUNG |ABSATZWEGGRUPPENSTATISTIK|SVVL104 | | | * | | | FIRMA-GRUPPE |LISTPARAMETER |SVVL104 | | * | * | | | LISTENNUMMER |LISTPARAMETER |SVVL104 | * | * | * | | | FIRMA-GRUPPE |LISTPARAMETER |SVVL104 | | | * | | | LISTENNUMMER |LISTPARAMETER |SVVL104 | | | * | | +--------------------------------------------------------------------------------------------+ | | | TOTAL NUMBER OF (NOT AUTOMATIC) DATA ITEMS REFERENCED = 13 | | NUMBER OF (NOT AUTOMATIC) DATA ITMES USED = 11 | | DATA COVERAGE RATION = 85% | S O F T E S T

15. Functional Coverage Measurement TESTTCOV-14 Function Coverage Function State Coverage Function Chain Function Outcome % of Application Functions

16. Levels of Functional Coverage TESTTCOV-15 Every application use case is executed at least once. Every application function is executed at least once. Every application function outcome is produced at least once. Every application function sequence is traversed at least once. Every possible combination of application functions is tested. 1 2 3 4 5 Increasing CostsIncreasing Reliability

17. F1=All specified Functions are tested at least once. F2=All specified Functions are tested with all valid Arguments at least one time. F3=All specified Functions are tested with all valid Arguments and all expected Results at least once. Results (n) = Function (Arguments (m)) Result (i) = Function (Argument (j)) i = 1 to n j = 1 to m Black-Box Test Coverage (Component Test) TESTTCOV-16

18. Functional Test Coverage TESTTCOV-17.1 TRAN:In|Out ART_NR:999 Amount:999 Use Case Coverage= > 1 Test Case TRAN ART_NR Amount One update transaction Function Coverage= > 2 Test Cases > one Increase > one Decrease TRANS ART-NR Menge Function Outcome Coverage = > 6 Test Cases > one Increase without Storno > one Increase with Storno of an existing order, generated earlier > one Increase with an error message > one Decrease without an order > one Decrease which generates an order > one Decrease with an error message

19. TESTTCOV-17.2 TRANS ART_NR Amount Function State Coverage= > 10 Test Cases > an Increase with invalid Art_Nr. > an increase with invalid amount > an increase to a non existing article > an increase with no storno > an increase with storno of existing order > a decrease with invalid Art_Nr. > a decrease with invalid amount > a decrease to a non existing article > a decrease without an order > a decrease which generates an order > an invalid transaction

20. Predicted Error Coverage TESTTCOV-18 100 200 300 400 500 123456 M o n t h s N u m b e r o f E r r o r s Specification Errors Design Errors Coding Errors Unit Test Integration Test System Test

21. Base Class Derived Class Is tested first with all possible variations Inherits test coverage of the base Class. Statement, Branch, Method & State Coverage Inheriting Test Coverage TESTTCOV-19

22. Base Class SubClass X Base Class SubClass Y Base Class SubClass Z Environment XEnvironment YEnvironment Z The Base Class must be retested in every new target Environment. According to Perry/Kaiser Study of 1990 Non Transitivity of Test Coverage TESTTCOV-20

23. For each Object TypeCreate= 4 Test Cases e.g. AccountRetrieve Update Delete For each Object State Create= 16 Test Cases e.g. plus AccountRetrieve minus Account Update overdrawn AccountDelete locked Account Testing the Creation, Retrieval, Update and Deletion of persistent Objects (relative to one, several or all Object Types and all Object States) CRUD Object Access Coverage TESTTCOV-21

24. Use Cases A_UseCase B_UseCase C_UseCase D_UseCase E_UseCase N_UseCase CRUDCRUDCRUD Test ObjectA_ObjectB_ObjectC_Object According to R. Binder, 1999 CRUD Object Access Coverage Matrix TESTTCOV-22

25. Class testComponent testSystem test Requirements Documentation User Documentation Use case Documentation Application Object Model System Object Model Object Interaction Model Interface Specification Class Specification Method Specification X X XX XX X X XX X X Software DocumentSoftware Testing Level Document Coverage TESTTCOV-23

26. Unit = Procedural Module Unit-Testing = Module Test White-Box Test is in the foreground Data are shared (global) Functional Decomposition Functional Coupling Data and Call Coupling Sub Program calls are stubbed Entity/Relationship-Diagrams Data Flow Diagrams Structure Diagrams Reuse is secondary Development is TOP-DOWN Waterfall Project Model Unit = Classe or Objekt Unit-Testing = Class Test Grey-Box Test is in the foreground Data are encapsulated Class Inheritance and Association Object Coupling Inheritance and Message Coupling Foreign Methods are invoked Object Sequence Diagrams Object State Transition Diagrams Object Collaboration Diagrams Reuse is primary Development is BOTTOM-UP Iterative Project Model Procedural SoftwareObject-orientierted Software Differences between procedural and object-oriented Software Differences in Test Coverage TESTTCOV-24