1. In the name of God Sharif University of Technology, International Branch, Kish Island Dr. Mohsen Sadighi Moshkenani www.sadighim.ir Chapter 15

2. Outline Testing; Common issues Testing requirements Taxonomy of methods Walk trough Black box White box

3. Testing Any effort for fixing errors (identify and remove) Testing for both Managerial issues Technical issues; for all phases

4. Testing costs Testing needs time, effort and budget To reduce its costs, start sooner!! To make a test more effective, again start sooner.

5. Error resources مرحلهمنابع خطا خواسته ‌ هابسيار طراحيكمتر كد كردنباز هم كمتر آزمايش بسيار كمتر منابع خطا در مراحل مختلف توليد نرم ‌ افزار ‏‎ [Koskinen, 2003] ‎

6. Error fixing costs مرحلههزينه كشف و رفع خطا خواسته ‌ ها 1 طراحي 5 كدنويسي 10 آزمايش 30 نسبت هزينه ‌ ي كشف و رفع خطا در مراحل مختلف توليد ‏‎ [Koskinen, 2003] ‎

7. Maintenance costs سال نسبت هزينه ‌ هاي نگهداري تعريفمرجع 2000 90% > هزينه ‌ ي اختصاص يافته براي نگهداري و تكامل به كل هزينه ‌ هاي نرم ‌ افزار [Erlikh, 2000] 1993 75% > نسبت هزينه ‌ هاي نگهداري به بودجه ‌ ي سيستم اطلاعاتي ( در 1000 شركت موفق ) [Eastwood, 1993] 1990 90% > هزينه ‌ ي اختصاص يافته براي نگهداري و تكامل به كل هزينه ‌ هاي نرم ‌ افزار [Moad, 1990] 199060 تا 70% هزينه ‌ ي نگهداري به بودجه هاي عملياتي سيستم ‌ هاي اطلاعات مديريت (MIS) [Huff, 1990] 198860 تا 70% هزينه نگهداري به بودجه هاي عملياتي سيستم ‌ هاي اطلاعات مديريت (MIS) [Port, 1988] 198465 تا 75% تلاش براي نگهداري نرم ‌ افزار به كل تلاش ‌ هاي مهندسي نرم ‌ افزار [McKee, 1984] 1981 50% > زماني كه نيروهاي نگهداري صرف كرده ‌ اند به كل زمان ( در 487 سازمان ) [Lientz and Swanson, 1981] 197967% هزينه ‌ هاي نگهداري به كل هزينه ‌ هاي نرم ‌ افزار [Zelkowitz et al., 1979]

8. Effective test In my opinion a test is effective if: It shows that there is no problem If it shows the problem, the problem could be solved too (before the critical point) Suppose a test correctly shows that a person has cancer; but there is no time for any treatment. This test is not effective, because there is no solution Effectiveness is a fuzzy value

9. Testing: common issues Fight (with …), to detect errors More erroneous parts, may have other errors! Changes should be tested, regression test There are risky points (when tired, late, have local view and more)

10. Testing: common issues (Cont.) Gradual change, one change at a time; based on integration strategy Have systematic approach to test, specially for modules which have severe roles More difficult to schedule Testing has basic requirements: Goal Meter Tool Execution Interpretation

11. Testing requirements

12. Taxonomy of tests Small scale, module level Large scale, goes toward final system level Scientific

14. Manual test of module Related documents are the basis Preliminary design of the module Detailed design of the module If in be necessary other higher documents, such as use cases and context diagram should be considered Check readability; perhaps many times, to check different issues Special attention on parameters, their types and calls Initial values, specially on pointers Keep all initializations together (initialization block) Execute the code yourself

15. Black box test External behavior is tested Correct reaction to inputs is tested Input type Correct input Wrong input Test data Test data generator Test data as an important resource Classification Special cases and need special attention

16. No complete test To be error free in one test, does not say about other test To have correct reaction to a specific value, does not say that the reaction to another value is correct There are too many cases. Correct and incorrect inputs, both should be considered Classification is our tool to handle number of cases

17. Classification of possible cases

18. White box test Internal behavior of the module is tested Possible paths should be covered Complete test is not simple/ possible Again classification is the solution Sample data should cover different paths Block: set of non-branching instructions which are executing together Blocking simplifies the test

20. Coverage Sequence If Case Loop coverage Counting Conditional Entrance Leave

21. Clear errors Non executable codes Non matching then/else Bad exception handling Unsuitable termination

22. Data Structure test Invalid references Invalid pointers Initial value Released related area Special cases in data structures Selected data structure and its duty, does it work?

23. Performance test Order of the program CPU time Memory Connection time Passing data...

24. Large scale tests: Testing integrated modules Function test Structure test Performance test Stress test: testing special cases Alpha Beta Acceptance

25. Function test Similar to black box test Role of integration strategy Integrated parts are considered as a single black box Input/output of the new module

26. Structure test Similar to white box test Role of integration strategy Integrated parts are considered as a single white box Each module is a block of the integrated module Path coverage is a main goal

27. Stress test Put the testing part in the worst possible situation, such as: Data load Number of concurrent users Minimum response time The basis is requirement Advertising use for results

28. Alpha test In developer’s environment By the sponsor(‘s representatives) Sponsor view is important here

29. Beta test In the environment of the colleagues and testing users By testing users Feedbacks

30. Acceptance test In the sponsor’s environment Real load Using by real users By the sponsor Most important test to finish the project

31. Acceptance test: Important considerations Based on the time schedule Minimum time No disturbance for usual activities Good education for users and administration personnel Check its interaction with other systems Note to human relations Sponsor is responsible; but you are involved

32. For special projects Have special attention on the test process. You may request extra responsibilities or money, test environment and domain knowledge for special tests.