1. A General Discussion on Functional (Black-box) Testing What are some of the concerns of testers ? –Have we got enough time to test (effort & schedule)? –Have we tested all the areas (coverage & efficiency)? –Are we discovering the defects (effectiveness)?

2. Effort & (Schedule) Effort ≡ amount of time required to i) develop, ii) execute, and iii) analyze the result of each test case: –Decision table (highest) May be less number of test cases But requires deeper analysis of conditions (which come from looking at the various combinations and dependencies) –Completeness –Avoiding redundancy –Partitioned Equivalence class (middle) Picking a representative from each of the partitioned class –Avoiding redundancy Difficulty is in coming up with the equivalence relation –Complete and disjoint –Boundary value (lowest) Easy to identify the boundaries Generating test cases are easier May expand into many test cases (especially for robust worst-case)

3. Testing Efficiency Efficiency ≡ amount of “coverage” provided by each test case. –Coverage of what? Possible combinations of inputs and expected outputs Possible logical conditions –What about redundancy? With efficiency, we are striving to i) maximize coverage (completeness) and ii) reduce redundancy

4. Efficiency –Decision table (high) Start with listing all n conditions and 2 n combinations –Completeness Analyze the irrelevant and reduce the redundant ones –Avoiding redundancy –Partitioned Equivalence class (high to medium) Define an equivalence relationship –Completeness Pick a representative test case from each disjoint class –Minimizing redundancy “Strong Robust” Equivalence (e.g. illegal cases) can generate lots of test cases –Boundary value (medium to low) Define the boundaries of valid and invalid inputs (a special case of equivalence relation?) –Completeness Pick data from within, on, and outside of the boundaries –Reducing redundancy as much as possible

5. Effectiveness Effectiveness ≡ number of defects discovered per test case executed. It is difficult to gauge this except from past experience: –People tend to make mistakes with boundaries Boundary value testing Equivalence class testing –People tend to make mistakes with combinations of inputs (logic) where there are dependencies Decision table testing

6. Testing (Effort) = # of Test cases sophistication Boundary Value Decision Table Equivalence Class = Time to develop each test case

7. Need for Metrics and History In order for us to better understand and manage testing, we need to better define the attributes: –Effort –Efficiency –Effectiveness We need to keep records (time; people-time; defects discovery - type & rate) of test cases: –Development of Test Cases –Execution of Test Cases –Test Result Analysis