In the name of God Sharif University of Technology, International Branch, Kish Island Dr. Mohsen Sadighi Moshkenani Chapter 15

Outline Testing; Common issues Testing requirements Taxonomy of methods Walk trough Black box White box

Testing Any effort for fixing errors (identify and remove) Testing for both Managerial issues Technical issues; for all phases

Testing costs Testing needs time, effort and budget To reduce its costs, start sooner!! To make a test more effective, again start sooner.

Error resources مرحلهمنابع خطا خواسته ‌ هابسيار طراحيكمتر كد كردنباز هم كمتر آزمايش بسيار كمتر منابع خطا در مراحل مختلف توليد نرم ‌ افزار ‏‎ [Koskinen, 2003] ‎

Error fixing costs مرحلههزينه كشف و رفع خطا خواسته ‌ ها 1 طراحي 5 كدنويسي 10 آزمايش 30 نسبت هزينه ‌ ي كشف و رفع خطا در مراحل مختلف توليد ‏‎ [Koskinen, 2003] ‎

Maintenance costs سال نسبت هزينه ‌ هاي نگهداري تعريفمرجع % > هزينه ‌ ي اختصاص يافته براي نگهداري و تكامل به كل هزينه ‌ هاي نرم ‌ افزار [Erlikh, 2000] % > نسبت هزينه ‌ هاي نگهداري به بودجه ‌ ي سيستم اطلاعاتي ( در 1000 شركت موفق ) [Eastwood, 1993] % > هزينه ‌ ي اختصاص يافته براي نگهداري و تكامل به كل هزينه ‌ هاي نرم ‌ افزار [Moad, 1990] تا 70% هزينه ‌ ي نگهداري به بودجه هاي عملياتي سيستم ‌ هاي اطلاعات مديريت (MIS) [Huff, 1990] تا 70% هزينه نگهداري به بودجه هاي عملياتي سيستم ‌ هاي اطلاعات مديريت (MIS) [Port, 1988] تا 75% تلاش براي نگهداري نرم ‌ افزار به كل تلاش ‌ هاي مهندسي نرم ‌ افزار [McKee, 1984] % > زماني كه نيروهاي نگهداري صرف كرده ‌ اند به كل زمان ( در 487 سازمان ) [Lientz and Swanson, 1981] % هزينه ‌ هاي نگهداري به كل هزينه ‌ هاي نرم ‌ افزار [Zelkowitz et al., 1979]

Effective test In my opinion a test is effective if: It shows that there is no problem If it shows the problem, the problem could be solved too (before the critical point) Suppose a test correctly shows that a person has cancer; but there is no time for any treatment. This test is not effective, because there is no solution Effectiveness is a fuzzy value

Testing: common issues Fight (with …), to detect errors More erroneous parts, may have other errors! Changes should be tested, regression test There are risky points (when tired, late, have local view and more)

Testing: common issues (Cont.) Gradual change, one change at a time; based on integration strategy Have systematic approach to test, specially for modules which have severe roles More difficult to schedule Testing has basic requirements: Goal Meter Tool Execution Interpretation

Testing requirements

Taxonomy of tests Small scale, module level Large scale, goes toward final system level Scientific

Manual test of module Related documents are the basis Preliminary design of the module Detailed design of the module If in be necessary other higher documents, such as use cases and context diagram should be considered Check readability; perhaps many times, to check different issues Special attention on parameters, their types and calls Initial values, specially on pointers Keep all initializations together (initialization block) Execute the code yourself

Black box test External behavior is tested Correct reaction to inputs is tested Input type Correct input Wrong input Test data Test data generator Test data as an important resource Classification Special cases and need special attention

No complete test To be error free in one test, does not say about other test To have correct reaction to a specific value, does not say that the reaction to another value is correct There are too many cases. Correct and incorrect inputs, both should be considered Classification is our tool to handle number of cases

Classification of possible cases

White box test Internal behavior of the module is tested Possible paths should be covered Complete test is not simple/ possible Again classification is the solution Sample data should cover different paths Block: set of non-branching instructions which are executing together Blocking simplifies the test

Coverage Sequence If Case Loop coverage Counting Conditional Entrance Leave

Clear errors Non executable codes Non matching then/else Bad exception handling Unsuitable termination

Data Structure test Invalid references Invalid pointers Initial value Released related area Special cases in data structures Selected data structure and its duty, does it work?

Performance test Order of the program CPU time Memory Connection time Passing data...

Large scale tests: Testing integrated modules Function test Structure test Performance test Stress test: testing special cases Alpha Beta Acceptance

Function test Similar to black box test Role of integration strategy Integrated parts are considered as a single black box Input/output of the new module

Structure test Similar to white box test Role of integration strategy Integrated parts are considered as a single white box Each module is a block of the integrated module Path coverage is a main goal

Stress test Put the testing part in the worst possible situation, such as: Data load Number of concurrent users Minimum response time The basis is requirement Advertising use for results

Alpha test In developer’s environment By the sponsor(‘s representatives) Sponsor view is important here

Beta test In the environment of the colleagues and testing users By testing users Feedbacks

Acceptance test In the sponsor’s environment Real load Using by real users By the sponsor Most important test to finish the project

Acceptance test: Important considerations Based on the time schedule Minimum time No disturbance for usual activities Good education for users and administration personnel Check its interaction with other systems Note to human relations Sponsor is responsible; but you are involved

For special projects Have special attention on the test process. You may request extra responsibilities or money, test environment and domain knowledge for special tests.