Testing: Chapter 13 "You make it, I’ll break it"

8/21/2015Page 2 Purpose: Does the product as a whole perform up to specifications? (validation?) Does it perform under normal, upper, and lower limits wrt the environment? Does it meet or exceed all claims (verification)?

Testing defined establishing confidence that a device does what it is supposed to do the process of operating a device with the intent of finding errors detecting specification errors and deviations from the specification 8/21/2015Page 3

Testing continued… verifying that a system satisfies its specified requirements or identifying differences between expected and actual results the process of operating a device or component under specified conditions, observing or recording the results, and making an evaluation of some aspect of the system or component. 8/21/2015Page 4

Testing … Is a positive act of destruction. Is needed for good product development Is necessary for consumer satisfaction… 8/21/2015Page 5

8/21/2015Page 6 Stressors: (depend on requirements…) TemperatureHumidity AltitudeShock VibrationEMI Cell phone, pagersBrownouts ESDSpillage ShippingPower Surges

8/21/2015Page 7 Figure 13-1 Requirements checklist

8/21/2015Page 8 Customer Misuse: Coffee spillage Drop in toilet Use as a seat Cross-connecting Reverse polarity Dirty hands Unplugging via jerk on cable

8/21/2015Page 9 Testing requires that you define failure! Intended application? Anticipated environment? Lifetime? Value vs. time Sunken ships make good coral reefs in time …

8/21/2015Page 10 Types of testing: Event testing – cycling, time to failure Environmental – operating & storage Altitude Threshold (door, elevator, bricks… ) Vibration ISTA shipping test EMI (standards!, see next slide: )

8/21/2015Page 11 EN 1000 series standards Radiated electric field immunity Fast transients Surge, line, magnetic Dips, variations, V, F variation On/off, ESU Walkie-talkie, cell phone ESU, etc.

Failure Definition For each test and for each device, a failure must be defined. This definition depends on the intended application and the anticipated environment. What is considered a failure for one component or device may not be a failure for another. The test protocol should be as detailed as possible in defining the failure. 8/21/2015Page 12

8/21/2015Page 13 We want to test some power supplies to prove a MTBF goal of 50,000 hours of operation. How many units do we test and for how long, assuming one failure and 90% confidence level? (sample size)(test time) = MTBF goal (7.779)/2 (sample size)(test time) = 50,000 (7.779)/2 = 194,475 unit-hours

8/21/2015Page 14 MTBF goal = (sample size)(test time) (2)/Χ 2 α;2r+2 Determining Sample Size and Test Length Confidence level = 1 ‑ α, where α is the risk factor and r = the number of failures Using the 90% confidence level, α = 0.10 & no failures (r=0) (Sample size)(test time) = (MTBF goal)( Χ 2 α;2 )/2 (sample size)(test time) = MTBF goal (4.61)/2 to prove a MTBF goal of 50,000 hours of operation: (sample size)(test time) = 50,000 (4.61)/2 = 115,250 unit-hours Example

Types of Testing: Verification Procedures that attempt to determine that the product of each phase of the development process is an implementation of a previous phase, i.e., it satisfies it. Each verification activity is a phase of the testing life cycle… 8/21/2015Page 15

Types of Testing: Validation Validation is the process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements

Types of Testing: Black Box Black box testing is a data driven testing scheme. The tester views the device or program as a black box, i.e., the tester is not concerned about the internal behavior and structure. The tester is only interested in finding circumstances in which the device or program does not behave according to its specification 8/21/2015Page 17

Types of Testing: White Box White box testing is concerned with the degree to which test cases exercise or cover the structure of the device or program. The ultimate while box test is an exhaustive path test. 8/21/2015Page 18

Types of testing: Software Module … Integration … System … Acceptance … 8/21/2015Page 19

Types of testing:Fuctional This type of testing verifies that given all the expected inputs then all of the expected outputs are produced. This type of testing is termed success oriented testing because the tests are expected to produce successful results. 8/21/2015Page 20

Types of testing - continued Robustness Testing Stress Testing Safety testing Regression Testing 8/21/2015Page 21

8/21/2015Page 22 Types of testing, continued Life test/reliability Customer misuse Fluid spillage Weight test Keyed connections Time related Failure related (MTBF)

Questions? 8/21/2015Page 23