1. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 1 Struktūrinis Testavimas

2. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 2 Testavimo procesas Komponentų testavimas: individualių programos komponentų testavimas, įprastai atsakomybė už komponentų testavimą tenka komponentų kūrėjams, išskyrus kritines sistemas, testai yra gaunami pagal kūrėjų patirtį. l Integravimo testavimas: komponentų, sujungtų į atskiras grupes, testavimas, sukuriant sistemas ar posistemes, atsakomybė tenka nepriklausomoms testavimo komandoms, testai remiasi sistemos specifikacija.

3. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 3 Testavimo fazės Komponentų testavimas Integravimo testavimas Programinės įrangos kūrėjas Nepriklausoma testavimo komanda

4. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 4 Introduction to Testing

5. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 5 First: a riddle about testing by Brian Marick l A mathematician, a physicist, and an engineer are told: “All odd numbers are prime.” The mathematician says, “That’s silly, nine is a non-prime odd number. The physicist says, “Let’s see, 3 is prime, 5 is prime, 7 is prime -- looks like it’s true.” The engineer says, “let’s see, 3 is prime, 5 is prime, 7 is prime, 9 is prime, 11 is prime -- looks like it’s true.”

6. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 6 Software testing l Historically, was not popular: with managers with testers with developers with students l testing and many software innovations evolved out of the “software crisis”

7. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 7 Software Failure rate (ideal)

8. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 8 Software Failure rate (real)

9. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 9 The Cost of Change

10. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 10 An error found after release costs four times (W. Perry) l 1st cost: developing program erroneously l 2nd cost: system has to be tested to detect the error l 3rd cost: wrong specs/code removed, correct specs/code added l 4th cost: system must be retested!

11. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 11 The “software crisis” l By the 1980’s, “quality” in software became a goal; SEI was born l “software engineering” became popular l the life cycle was studied l software developers and testers began to work together l by the 1990’s, testing tools became available

12. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 12 What is software testing l “The process of executing computer software in order to determine whether the results it produces are correct”, Glass ‘79 l “The process of executing a program with the intent of finding errors”, Myers ‘79 l “Program testing can be used to show the presence of bugs, but never their absence”, Dijkstra ‘72

13. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 13 What is software testing (cont) l “The aim is not to discover errors but to provide convincing evidence that there are none, or to show that particular classes of faults are not present”, Hennell ‘84 l “Testing is the measure of software quality”, Hetzel ‘85

14. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 14 What is software testing (cont) l “The process of operating a system or component under specified conditions, observing or recording the results, and making an evaluation of some aspect of the system or component.” IEEE/ANSI, 1990

15. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 15 Testing is a state of mind l “If our goal is to show the absence of errors, we will find very few of them” l “If our goal is to show the presence of errors, we will discover a large number of them”

16. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 16 Time spent on testing l 50%Brooks/Myers, 1970s l 80%Arthur Andersons’ Director of testing in North America, 1990s

17. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 17 Tester-to-developer ratios l 1:5-10 Mainframes i.e.,1 tester for every 5 to 10 developers l 2:3 Microsoft, 1992 l 2:1 Lotus (for 1-2-3 Windows) l 1:2 Average of 4 major companies,1992 Microsoft, Borland, WordPerfect, Novell

18. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 18 Difficulties in testing software l poorly expressed requirements l informal design techniques l nothing executable until coding stage l Huge input set: consider testing software that categorises an exam grade: 101 inputs consider testing software that categorises two exam grades: 101*101 inputs!

19. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 19 Difficulties in testing software (cont) l Exhaustive software testing is intractable l Even if all possible inputs could be identified, the problem of identifying non-halting cases is undecidable l Weyuker (1979) has shown that there is no algorithm that can determine if a given statement, branch or path will be exercised! l we’ll look at this difficulty in more detail after we understand graphs

20. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 20 l Dar vadinamas “baltos dėžės“ testu. l Testiniai atvejai gaunami iš programos struktūros. Žinios apie programą naudojamos nustatyti papildomus testinius atvejus. Tikslas yra išbandyti visus programos operatorius (ne visas kelių kombinacijas). Struktūrinis testavimas

21. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 21 “Baltos -dėžės” testavimas Gauna Testuoja Testavimo duomenys Testavimo rezultatai Komponento kodas

22. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 22 Kelių testavimas l Kelių testavimo tikslas yra įsitikinti, ar testinių atvejų rinkinys yra toks, kad kiekvienas kelias per programą yra įvykdytas bent kartą. l Kelių tikrinime esminis yra programos skaičiavimų (flow) grafas, kuris parodo programos mazgus priimančius sprendimus ir lankus, rodančius skaičiavimų valdymą. l Sąlyginiai operatoriai yra skaičiavimų grafo mazgai.

23. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 23 control flow graph l Directed graph G(V, E) V is set of vertices E is set of edges, E = VXV l The granularity of the vertices can be an operation, a statement or a basic block l The edges are directed; direction indicates flow of control from one vertex to another

24. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 24 Aprašo programos skaičiavimų valdymą. Kiekviena atšaka yra parodyta kaip atskiras kelias ir ciklai parodyti kaip rodyklės grįžimo į ciklo sąlygos mazgą. Yra naudojamas kaip pagrindas ciklomatiniam (cyclomatic ) sudėtingumui skaičiuoti: ciklomatinis sudėtingumas = briaunų skaičius – mazgų skaičius + 2 Programos skaičiavimų grafas

25. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 25 l Testų skaičius, kad patikrinti visas valdymo instrukcijas prilygsta ciklomatiniam sudėtingumui. l Ciklomatinis sudėtingumas lygus sąlygų skaičiui programoje. l Naudingas, tačiau reikia naudoti atsargiai, nes neadekvatus testavimui. Skaičiavimų vykdymas visais keliais neatitinka visų kelių kombinacijų vykdymo. Ciklomatinis sudėtingumas

26. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 26 Binary search flow graph

27. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 27 l 1,2,3,8,9; l 1,2,3,4,6,7,2; l 1,2,3,4,5,7,2; l 1,2,3,4,6,7,2,8,9; testiniai atvejai turi būti gauti taip, kad visi iš šių kelių būtų įvykdyti; l dinaminis programos analizatorius gali būti panaudotas patikrinti, ar tie keliai buvo įvykdyti. Nepriklausomi keliai

28. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 28 Operatoriai, šakos, keliai l Example: Proc(x); (1) If x > 17 then(2) x := x - 17(3) If x = 13 then(4) x = 0 ;(5) End;(6) Testiniai duomenys x = 30- visi operatoriai Testiniai duomenys x = 30, x = 17- visos šakos Testiniai duomenys x = 30, x = 17, x = 13, x = 21- visi keliai {1,2,3,4,5,6}{ 1,2,4,6}{1,2,4,5,6}{1,2,3,4,6} 1 2 4 6 3 5

29. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 29 basic block (defn) l sequence of statements such that the only entrance to the block is through the first statement and the only exit from the block is through the last statement

30. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 30 Let’s consider path testing l Construct test cases to exercise all paths through a program. l Called “path coverage”.

31. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 31 Finding the square root of an inputted value: an example start read number root = square_root(number) print root end

32. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 32 Finding the square root start read number if number > 0 root = square_root(number) print root else print error message endif end tf

33. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 33 Finding the square root start read number while number != 0 if number > 0 root = square_root(number) print root else print error message endif read number endwhile end

34. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 34 How many paths?

35. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 35 Exam processing example l consider a program to process one exam result for 10 students l categorise the result as A, B, C, D, F l How many paths through the program?

36. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 36 Find the number of paths for 10 inputs

37. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 37 White Box Testing l Aim to test every “path” through the program l This should ensure 100% correct programs? l How many paths in the following program? 100 lines of C code, starts with var decls. 2 nested loops, executing between 1 & 20 times Inside inner loop, four if-then-else statements l How long to test at 1ms per test?

38. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 38 Rationale for White Box Testing l Errors tend to occur in code written to handle special cases l Our assumptions about which parts of our programs are executed most often are frequently wrong l Typos can occur in out-of-the-way parts of the program as easily as in the main control flow.

39. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 39 Condition Testing l Branch Testing (Myers) Write test cases so that the true and false branches of every condition are executed at least once. l Domain Testing (White and Cohen) For relational comparisons (e.g. x < y) 3 test cases: x y Choose neighbouring x and y

40. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 40 Loop Testing l Beizer proposed this approach to testing loops l Different types of loop: simple nested concatenated unstructured

41. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 41 Testing Simple Loops l Create a test case for the following situations: the loop is never executed the loop is executed once the loop is executed twice the loop is executed m times (m < n) the loop is executed n - 1 times the loop is executed n times the loop is executed n + 1 times

42. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 42 Simple Loop Example empRec := read(taxFile); totalTax := 0; while not eof(taxFile) do begin totalTax := totalTax + empRec.tax; empRec := read(taxFile) end;

43. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 43 Testing Nested Loops l Simple loop testing for inner-most loop, with outer loops held at minimum iterations l Add other tests for excluded or out-of-range values l Work outwards, keeping outer loops at minimum iterations and inner loops at “typical” numbers of iterations l Continue until all loops have been tested

44. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 44 Testing Other Kinds of Loop l Concatenated loops test as for simple loops if independent test as for nested loops if dependent l Unstructured loops Don’t start from here!!!

45. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 45 Test Coverage l Refers to the proportion of the potential paths through a program that are covered by a given test set. l Ideally, we want to maximise test coverage while minimising the resources used during testing l McCabe proposed Basis Paths as solution

46. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 46 Basis Path Testing (1) l Depends upon view of a program or design as a flow graph initialise counter read first record while not eof do process record if okay then increment counter else report error endif read next record endwhile 10 1 2 4 6 9 11 5 3 7 8 T TF F

47. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 47 Basis Path Testing (2) l A basis set of paths through a program executes each instruction in that program at least once l An independent path in a basis set is one which differs from other paths in the set in at least one way

48. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 48 Independence of Paths l Example paths: 1,2,3,11 1,2,3,4,5,7,9,10,3,11 1,2,3,4,6,8,9,10,3,11 10 1 2 4 6 9 11 5 3 7 8 T TF F 1,2,3,11

49. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 49 Cyclomatic Complexity l The number of independent paths through a graph is called the cyclomatic complexity of the graph l v(G) = e(G) - n(G) + 1 (or sometimes 2) e(G) = number of edges in G n(G) = number of nodes in G l Gives an upper limit for number of test cases

50. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 50 Cyclomatic Complexity Example e(G) = 12 n(G) = 11 v(G)= 12 - 11 + 1 = 2 10 1 2 4 6 9 11 5 3 7 8 T TF F

51. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 20 Slide 51 Basis Path Testing Cont. l Once you have determined the cyclomatic complexity, v(G) test cases can be generated. l Determine basis set of independent paths l Prepare a test case that will cause execution of each such path l Some basis paths can only be tested in conjunction with others