1. Testing: General Requirements, DFT, Multilevel Testing Testing: General Requirements DFT Multilevel Testing-- System, Black Box, White Box Tests

2. Testing--General Requirements Testing--general requirements: thorough ongoing DEVELOPED WITH DESIGN (DFT--design for test) note: this implies that several LEVELS of testing will be carried out efficient

3. Good, Bad, and Successful Tests good test: has a high probability of finding an error ("bad test": not likely to find anything new) successful test: finds a new error

4. Most Effective Testing Is Independent most effective testing: by an "independent” third party Question: what does this imply about your team testing strategy for the quarter project?

5. How Thoroughly Can We Test? how thoroughly can we test? example: VLSI chip 200 inputs 2000 flipflops (one-bit memory cells) # exhaustive tests? What is the overall time to test if we can do 1 test / msec? 1 test /  sec?  test  sec?

6. Matching Testing with Design and Language How will testing match with design/language? e.g.: *procedural: example--sort an array *oo: example--develop an "array class” what kinds of test need to be done in each case?

7. Design for Testability (DFT) Design for Testability (DFT)--what makes software "testable"?  operability: few bugs, incremental test  observability: you can see the results of the test  controllability: you can control state + input to test  decomposability: you can decompose into smaller problems and test each separately  simplicity: you choose the “simplest solution that will work”  stability: same test will give same results each time  understandability: you understand code, inputs, and outputs

8. Testing strategies testing strategies: verification--functions correctly implemented validation--we are implementing the correct functions (according to requirements)

9. Spiral design/testing strategy A general design/testing strategy can be described as a "spiral”: requirements  design  code system test  module,integ. tests  unit test (system) (black (white box) box) when is testing complete? One model: "logarithmic Poisson model” f(t)=(1/p)ln(I 0 pt+1) f(t) = cumulative expected failures at time t I 0 = failures per time unit at beginning of testing p = reduction rate in failure intensity START END Requirements, Specs/System Tests Design/Integration Tests Code/Unit Tests Design/Module Tests

10. Development and Testing Stages levels of test___ development stage white box code black box class integrationER-diag. systemuse case

11. Types of testing Types of testing:  white box--"internals” (also called "glass box")  black box—modules and their "interfaces” (also called "behavioral")  system--”functionality” (can be based on specs, use cases)  application-specific-- GUIs Client/Server Real-time Documentation/help

12. Good testing strategy steps in good test strategy: quantified requirements test objectives explicit user requirements clear use "rapid cycle testing" build self-testing software filter errors by technical reviews review test cases and strategy formally also continually improve testing process

13. OO testing strategy OO testing: emphasis is on interfaces use UML tools to support testing strategies and development of test cases --system tests: use cases; quality measurements --black box tests: ER diagrams, object message diagrams, dataflow and state diagrams --white box tests: class and state diagrams, CRC cards

14. Black box tests & system tests Black Box Tests System Tests

15. Black box testing--what to test black box testing: test functional requirements of components. Must check:  incorrect or missing components  interface errors  data structures or external data access  behavior /performance errors  initialization and termination errors

16. Black box testing--testing graph start point is usually a graph: objects to be modeled relationships connecting them ("links") link properties example: can use ER diagrams, object message diagrams, state diagrams, with additional information on links as necessary Example: p. 404, Pressman

17. Black box testing--examples Examples: CarGasStation: station P company employee Denotes link that leads to one or more test cases

18. Black box testing techniques Some useful techniques for black box testing: transaction flow data flow state modeling timing modeling (all "dynamic") much testing focuses on "boundary values" between components

19. Black box testing--input partitioning example: equivalence partitioning for input for black box test: condition# ”cases” of input range1 valid, 2 invalid value1 valid, 2 invalid set member1 valid, 1 invalid boolean1 valid, 1 invalid

20. Black box testing guidelines General guidelines: test BOUNDARIES test output also choose "orthogonal” cases if possible

21. Using specifications for system tests System tests should verify that specifications have been met For UML-based strategy: each use case ---> one or more system tests each quality / performance requirement  one or more system tests Additional qualitative or quantitative tests (not from use cases): examples: is system “user-friendly”? are timing requirements met? are available resources sufficient?

22. Using specifications for system tests Example: 1. Place call 2. Receive call 3. Use scheduler Cellular network User Associated sequence diagrams 1. 2. 3. Associated test cases 1. 2. 3. 3 use cases Tests verify use case supported