1. 1 Programų testavimo metodai

2. 2 ĮVADAS  Modulio paskirtis

3. 3 Tikslai  Apibūdinti SWEBOK projektą  Apibrėžti modulio vietą mokymo plane  Aprašyti modulio žinių struktūrą  Apibūdinti praktines užduotis

4. 4 SWEBOK  Software Configuration Management Software Configuration Management Software Configuration Management  Software Construction (Version 0.5) Software Construction Software Construction  Software Design Software Design Software Design  Software Engineering Infrastructure Software Engineering Infrastructure Software Engineering Infrastructure  Software Engineering Management Software Engineering Management Software Engineering Management  Software Engineering Process Software Engineering Process Software Engineering Process  Software Evolution and Maintenance Software Evolution and Maintenance Software Evolution and Maintenance  Software Quality Analysis Software Quality Analysis Software Quality Analysis  Software Requirements Analysis Software Requirements Analysis Software Requirements Analysis  Software Testing Software Testing Software Testing

5. 5 Mokymo programa (Curricula ) Programų sistemų inžinerijos magistras (Master in Software Engineering) 4 sem. Teorija Programų priežiūra ir tobulinimas (Software Evolution and Maintenance) R.Marcinkevičius Teorija Programų kokybės analizė (Software Quality Analysis ) A.Ostreika R&D Magistro baigiamasis darbas (Master Thesis)D Magistro baigiamasis darbas Prof. R.Seinauskas 3 sem. Teorija Programų kūrimo priemonės (Software Engineering Tools) L.Lipavičius Laisvai pasirinkti (Electives) Informacinės technologijos projektavimo vadyboje (Project Management Tools) V.Limanauskienė R&D Programinės įrangos įdiegimo tyrimas (Software Design and Implementation)D Programinės įrangos įdiegimo tyrimas (V.Limanauskienė) 2 sem. Teorija Programų projektavimas (Software Design) D.Rubliauskas Teorija Duomenų projektavimas (Data Design) B.Paradauskas Teorija Programų testavimo metodai Programų testavimo metodai (Software Testing Methods) R.Šeinauskas Laisvai pasirinkti (Electives) R&D Programų sistemų architektūros analizė (Analysis of Software Architecture) K. Motiejūnas 1 sem. Teorija Programų inžinerijos procesas (Software Engineering Process) A.Targamadzė Teorija Sistemų modeliavimas ir validavimas (System Modeling and Validation) H.Pranevičius Teorija Reikalavimų programoms analizė (Software Requirements Analysis) R. Butleris Programų inžinerijos valdymas (Software Engineering Management) A. Tomkevičius R&D Reikalavimų specifikavimas (Requirements specifications) K. Motiejūnas 2000 09 12

6. 6 Bendros knygos  Pressmann Software Engineering Theory and Practice, 6 laida  J. Sommerville, I. Software Engineering 8 laida  http://www.elen.ktu.lt/~rsei/PT/index.htm

7. 7 Testavimo knygos  http://www.softwareqatest.com/ http://www.softwareqatest.com/

8. 8 Programavimo krizė  Programinės įrangos klaidos yra geriau žinomos negu sėkmės atvejai  Programavimo krizė prognozuota prieš 30 metų taip niekada ir neįvyko ir programuotojams pavyko sumažinti nesėkmių kiekį bei darbuotojų poreikį  Problemos, kurios kamuoja programuotojus pagrindinai susietos su tinkamu programinės įrangos kūrimu ir jos palaikymu negu kad su gavimu programinės įrangos, kuri funkcionuoja korektiškai

9. 9 Teisybė  Matematikui, fizikui ir inžinieriui buvo pasakyta “ Visi nelyginiai skaičiai yra pirminiai”  Matematikas pasakys: nesamonė, 9 yra nelyginis skaičius bet ne pirminis  Fizikas pasakys: pažiūrėkim, trys yra pirminis, penki yra pirminis, septyni yra pirminis, atrodo, kad teisybė  Inžinierius pasakys: pažiūrėkim, trys yra pirminis, penki yra pirminis, septyni yra pirminis, devyni yra pirminis, vienuolika yra pirminis, atrodo, kad teisybė

10. 10 Programų testavimas  Istoriškai nepopuliarus tarp:  vadybininkų  programuotojų  studentų  Testavimas ir daugelis programavimo inovacijų išaugo iš programavimo krizės

11. 11 Testavimas: Mūsų patyrimas Software to be tested Test Case Output

12. 12 Kada sustoti? Software to be tested Test Case Output Enough? No Verification Yes Test Coverage Test Case Generation

13. 13 Testavimo pavyzdys Sorting Program Sorting Program SPECS: Takes a list of numbers; returns a sorted list. {1,3,2} {1,2,3} {3,2,3} {} {-1, -2} Just a list. A sorted list. Repeated entry. Empty list. Negative numbers. Test Cases {1, 2, 3} Output {1, 2, 3} Output {2, 3, 3} Output {} Output {-2, -1} Output

14. 14 Automatinis testavimas Software to be tested Test Case Output Enough? No Verification Yes Test Coverage Test Case Generation

15. 15 Automatinis testavimas Software to be tested Test Case Output Coverage Evaluator Test Case Generator Verifier OR Test Oracle Test Specs

16. 16 Kas tai yra programų testavimas?  “The process of executing computer software in order to determine whether the results it produces are correct”, Glass ‘79  “The process of executing a program with the intent of finding errors”, Myers ‘79  “Program testing can be used to show the presence of bugs, but never their absence”, Dijkstra ‘72

17. 17 Kas tai yra programų testavimas?  “The aim is not to discover errors but to provide convincing evidence that there are none, or to show that particular classes of faults are not present”, Hennell ‘84  “Testing is the measure of software quality”, Hetzel ‘85 Kas vadinama programų testavimu?

18. 18 Kas tai yra programų testavimas?  “The process of operating a system or component under specified conditions, observing or recording the results, and making an evaluation of some aspect of the system or component.” IEEE/ANSI, 1990 IEEE/ANSI, 1990

19. 19 Testavimas priklauso nuo požiūrio  “If our goal is to show the absence of errors, we will find very few of them”  “If our goal is to show the presence of errors, we will discover a large number of them”

20. 20 Testavimo laikas  50%Brooks/Myers, 1970s  80%Arthur Andersons’ Director of testing in North America, 1990s

21. 21 Testuotojų ir programuotojų santykis  1:5-10 Mainframes i.e.,1 tester for every 5 to 10 developers i.e.,1 tester for every 5 to 10 developers  2:3 Microsoft, 1992  2:1 Lotus (for 1-2-3 Windows)  1:2 Average of 4 major companies,1992 Microsoft, Borland, WordPerfect, Novell Koks testuotojų ir programuotojų santykis?

22. 22 Programų testavimo problemos  poorly expressed requirements  informal design techniques  nothing executable until coding stage  Huge input set: consider testing software that categorises an exam grade: 101 inputs consider testing software that categorises two exam grades: 101*101 inputs! consider testing software that categorises two exam grades: 101*101 inputs! Kokios programų testavimo problemos?

23. 23 Kas tai yra programų testavimas?  Exhaustive software testing is intractable (sunkiai suvaldomas)  Even if all possible inputs could be identified, the problem of identifying expected results is undecidable (neišsprendžiama)  Weyuker (1979) has shown that there is no algorithm that can determine if a given statement, branch or path will be exercised!

24. 24 Programų testavimo apibrėžimas  Software testing consists of the dynamic verification of the behavior of a program on a finite set of test cases, suitably selected from the usually infinite executions domain, against the specified expected behavior. Kaip apibrėžiamas programų testavimas?

25. 25 Dinaminis  This term means testing always implies executing the program on valued inputs. Static analysis techniques, such as peer review and inspection (that sometimes are improperly referred to as "static testing"), are not considered as part of this KA; nor is program execution on symbolic inputs, or symbolic evaluation;

26. 26 Baigtinis  Clearly the tests are not selected literally from an infinite set of possible tests, but a very large finite one (the set of all bit strings shorter than some length). Nonetheless, for even simple programs, so many test cases are theoretically possible that exhaustive testing could require even years to execute. This is why in practice the number of tests can generally be considered infinite. However, the number of executions which can realistically be observed in testing must obviously be finite, and -more than this- it must be manageable. Indeed, testing always implies a trade-off between limited resources and schedules, and inherently unlimited test requirements: this conflict points to well known problems of testing, both technical in nature (criteria for deciding test adequacy) and managerial in nature (estimating the effort to put in testing); Kokią įtaką turi testo ilgis?

27. 27 Išrinktų  The many proposed test techniques essentially differ in how they select the (finite) test set, and testers must be aware that different selection criteria may yield largely different effectiveness. The problem of identifying the most suitable selection criterion under given conditions is still under research.

28. 28 Laukiami rezultatai  It must be possible to decide whether the observed outcomes of program execution are acceptable or not, otherwise the testing effort would be useless. The observed behavior may be checked against user's expectations (commonly referred to as testing for validation) or against a functional specification (testing for verification). The test pass/fail decision is referred to as the oracle problem, which can be addressed with different approaches. Kas tai yra orakulo problema?

29. 29 Statiniai metodai  Software testing is a means for quality evaluation. In the SQA chapter of the SWEBOK, activities and techniques for quality analysis are categorized into: static techniques (no code execution), and dynamic techniques (code execution). Both categories are useful. Although this chapter focuses on testing, that is dynamic, we would like to emphasize that static techniques are equally important for the purpose of building quality in a software product. Static techniques are covered into the SQA KA description Kas tai yra statiniai testavimo metodai?

30. 30 Testavimo veikla  In the years, the view of software testing has evolved towards a more constructive attitude. Testing is no longer seen as an activity that starts only after the coding phase is complete, with the limited purpose of finding mistakes. Software testing is nowadays seen as an activity that encompasses the whole development process, and is an important part itself of the actual construction. Indeed, planning for testing starts since the early stages of requirement analysis, and test plans and procedures are systematically and continuously refined as the development proceeds. These activities of planning and designing tests constitute themselves a useful input to designers for highlighting potential weaknesses. Ką apima testavimo veikla?

31. 31 Klaidų vengimas  As more extensively discussed in the SQA chapter, the right attitude towards quality is one of prevention: it is better to avoid problems, rather than repairing them. Testing must be seen as a means primarily for demonstrating that the prevention has been effective, but also for identifying anomalies in those cases in which, for some reason, it has been not. Finally, it is worth recognizing that even after a good testing, the system could still contain mistakes. The remedy to system failures after delivery is provided by (corrective) maintenance actions. The maintenance issues of software engineering are covered into the Software Evolution and Maintenance chapter of the SWEBOK Kuo remiasi klaidų vengimo strategija?

32. 32 Temos  A.Testing Basic Concepts and Definitions  B.Test Levels  C.Test Techniques  D.Test-related Measures  E.Managing the Test Process  F.Test Tools.

33. 33 A.Bazinės testavimo koncepcijos ir apibrėžimai  When the outcome of a test execution diverges from the expected, a failure is observed, whose cause is a fault. These and other basic definitions within software testing area, as well as an introduction to the terminology, will be provided in Part A Testing Basic Concepts and Definitions. In the same section, the scope of the KA will be laid down, also in relation with other activities.

34. 34 B.Testavimo lygiai  Software testing is usually performed at different levels along the development process. That is to say, the object of the test can vary: a whole program, part of it (functionally or structurally related), a single module. The latter subject is expanded in Part B Test Levels. This consists of two (orthogonal) subsections: B.1 lists the phases in which the testing of large software systems is traditionally subdivided. In B.2 testing for specific conditions or properties is instead considered, and is referred to as "Objectives of testing". Clearly not all listed types of testing apply to every system, nor has every possible type been listed, but those most generally applied.

35. 35 Testavimo paskirtis  Testing is conducted in view of a specific purpose (test objective), which is stated more or less explicitly, and with varying degrees of precision. Stating the objective in precise, quantitative terms allows for establishing control over the test process. Kokia testavimo paskirtis?

36. 36 Testavimo tikslas  One of testing aims is to expose failures (as many as possible), and many popular test techniques have been developed for this objective. These techniques variously attempt to "break" the program, by running identified classes of (deemed equivalent) executions: the leading principle underlying such techniques is being as much systematic as possible in identifying a representative set of program behaviors (generally in the form of subclasses of the input domain). However, a comprehensive view of the KA of testing as a means for quality must include other as important objectives for testing, e.g., reliability measurement, usability evaluation, contractor’s acceptance, for which different approaches would be taken. Note that the test objective varies with the test object, i.e., in general different purposes are addressed at the different levels of testing. Koks testavimo tikslas?

37. 37 Testavimo kriterijai  The test objective determines how the test set is identified, both with regard to its consistency -how much testing is enough for achieving the stated objective?- and its composition -which test cases should be selected for achieving the stated objective?-, although usually the "for achieving the stated objective" part is left implicit and only the first part of the two italicized questions above is posed. Criteria for addressing the first question are referred to as test adequacy criteria, while for the second as test selection criteria Kas tai yra testavimo kriterijai?

38. 38 C.Testavimo metodai  As already said, several Test Techniques have been developed in the last two decades according to various criteria, and new ones are still proposed. "Generally accepted" techniques will be described in Part C. Kas tai yra testavimo metodai?

39. 39 D.Testavimo matavimai  Sometimes, it can happen that confusion is made between test objectives and techniques. For instance, branch coverage is a popular test technique. Achieving a specified branch coverage measure should not be considered per se as the objective of testing: it is a means to improve the chances of finding failures (by systematically exercising every program branch out of a decision point). To avoid such misunderstandings, a clear distinction should be made between test measures which evaluate the test set, like measures of coverage, and those which instead provide an evaluation of the program under test, based on the observed test outputs, like reliability Kam reiklaingi testavimo matavimai?

40. 40 E. Testavimo proceso valdymas  Testing concepts, strategies, techniques and metrics need to be integrated into a defined and controlled process, which is run by people. The test process should support testing activities and testing teams, from test planning to test outputs evaluation, in such a way as to provide justified assurance that the test objectives are met cost-effectively. Issues relative to Managing the Test Process are expanded in Part E. Kokia testavimo proceso paskirtis?

41. 41 F. Testavimo priemonės  Software testing is a very expensive and labor-intensive part of development. For this reason, tools are instrumental to support test activities. Moreover, in order to enhance cost- effectiveness ratio, a key issue has always been pushing test automation as much as possible. Existing tools and concepts related to automating the constituent parts of test process are addressed in Part F Test Tools Kam skirtos testavimo priemonės?

42. 42 Bendra Užduotis  1. Suformuoti darbo grupę iš 4 asmenų  2. Parinkti darbo grupes pavadinimą  3Nusiųsti laišką dėstytojui adresu: vak@soften.ktu.lt @soften.ktu.lt  Susirašinėti su dėstytoju tik šiuo adresu. Laiške kaip subject nurodyti darbo grupės vardą ir per brūkšnelį laiško numerį. Grupei, kurios vardas “ Namas” pirmam laiške nurodyti subject Namas-1. Pirmam laiške nurodyti grupės vardą, grupės studentų vardą, pavardę, grupę ir paprašyti atsiųsti projekto užduotį. Atsakymas bus siunčiamas su “ Reply”.

43. 43 Bendra Užduotis  4. Duotam algoritmui sudaryti programą nurodytomis priemonėmis. Detaliai aprašyti užduotį ir algoritmą, pateikti struktūrogramą. Programa turi turėti ne mažiau kaip penkis atskirai testuojamus vienetus. Sudaryti programos testavimo planą. Bent vienas vienetas turi būti automatiškai ištestuotas įvairiais kriterijais. Parengti testavimo dokumentaciją. Programa pateikiama kompaktiniam diske ir pristatymo metu suinstaliuojama į nurodytą kompiuterį, o po pristatymo išinstaliuojama arba pateikiama serveryje. Pristatymo metu pateikti vieną lapą, kuriame nurodoma grupės vardas, sudėtis, pagrindiniai atliktos užduoties privalumai

44. 44 Programavimo būdas 5. Gali būti trys programavimo būdai: A – Programuojama Java kalba, programa paleidžiama su naršykle ir ją vykdo vartotojo kompiuteris B - Programuojama C arba C++ ir php, programa paleidžiama su naršykle ir vykdoma serveryje C- Programuojama C arba C++, instaliacinė programa nuskaitoma iš serverio į vartotojo kompiuterį, programa suinstaliuojama ir vykdoma vartotojo kompiuteryje

45. 45 Funkcijų sąsaja  1. Funkcijos sąsaja ( char *įėjimo-vektorius, char *išėjimo-vektorius, int įėjimų-skaičius, int išėjimų-skaičius, int sekmės-požymis)  Sekmės-požymis 1 –sėkmingai paskaičiuota, 0 - nepaskaičiuota

46. 46 Vertinimo Taisyklės  Galutinis pažymys G skaičiuojamas pagal formulę: G = ((A + E)/2)*0,5 + S*0,2 + P* 0,3 A – Apklausa iš pakartojimo E- Egzaminas S- pasisakymas paskirta tema P- projektinė užduotis  ir apvalinamas pagal matematines taisykles iki sveikaskaitinio skaičiaus.