Authors: Nirav Desai, prof. hema gaikwad International Conference on Computing, Communication and Automation 2015 Galgotias University Gurgaon Paper ID: 707 Graph based verification and identification of defects by binary search on a graph Authors: Nirav Desai, prof. hema gaikwad Symbiosis institute of computer studies and research, PUNE, MAHARASHTRA

VERIFICATION AND VALIDATION Software testing is divided into 2 parts: Verification and Validation. Verification relates to the passive inspection of code by visualizations and walk-throughs. It helps to identify whether we are building the system right Validation is dynamic testing of the code using different test patterns which is used to identify if the system is functioning correctly It helps to identify if we built the right system. ISO 9126 and IEEE STANDARD 829 define the standard methodologies for software testing. Test plan strategies (model based, analytical risk based or requirements based, standards driven, etc) Test environment setup Work breakdown structure and resource breakdown structure Test Results Documentation

Software Defect Lifecycle Defect Open Troubleshoot Defer Reject Defect Close Test Defect Identification Defects are identified by testers during the verification and validation process If the test lead accepts the identified defects, a new defect is opened The software developers can either debug and fix the defect, defer the debugging to new development cycle or reject the defect as duplicated Software Tester verifies the software operation against a reference set The debugged program is tested again and if no defects are found, defect is closed, else the defect is re-opened

VERIFICATION Problems in the software are identified by code walkthroughs and passive inspection Helps to check the process of software coding by checking for errors at the time they are introduced Graph based verification helps in easier walk-throughs of the code. A visual inspection of the graph of the software control flow helps to identify the number of test cases. Cyclomatic complexity of graph = e – n + 2*g Where e = number of edges, n = number of nodes and g = no. of independent sub graphs

CYCLOMATIC GRAPH OF CODE No. of edges = 8 No. of nodes = 7 No. of test cases = 8 – 7 + 2 = 3 Test Case 1: A = 354 Test Case 2: A != 354 and B > C Test Case 3: A != 354 and B < C Reference: Foundations of Software Testing by Rex Black

DEPENDENCY GRAPH OF CODE DEPENDENCY GRAPH GENERATED USING ENSOFT ATLAS: HIGHLIGHTS ALL DEPENDENCIES FOR A GIVEN LINE OF CODE

SEARCH FOR DEFECTS: LIBLIT05 Liblit05 algorithm uses statistical techniques to search for defects in a code For each predicate P’ in program P, Liblit05 estimates two conditional probabilities respectively: Pr1 = Pr(P fails|P’ is ever observed) and Pr2 = Pr(P fails|P’ is ever observed as true). It then treats the probability difference Pr2−Pr1 as an indicator of how relevant P is to the bug. Therefore, Liblit05 essentially regards the predicates whose true evaluations correlate with program failures as bug-relevant. Because Liblit05 only considers whether a predicate has ever been evaluated as true in one run, it loses its discrimination power when a predicate P is observed as true at least once in every execution. This algorithm does not look at how many times a given predicate is evaluated so the algorithm fails when there are problems with loops.

SEARCH FOR DEFECTS: SOBER A has an extra chance to be true in incorrect runs. Program 1 was tested with 5,542 test cases that are available in the Siemens suite. The average probability of A being evaluated as true within one incorrect run is 0.9024 and it is 0.2261 in correct ones.

SEARCH FOR DEFECTS: BINARY SEARCH ON A GRAPH Set the break point in execution half way through the code and test if the output is correct. If output is as expected, the defect is in the latter half of the code or else in the previous half. If the defect is in the latter half of the code, set the break point at the half way mark of the remaining code. For a defect in the preceding part of the code, set the break point at the half way mark of the preceding section. Repeat this process of setting breakpoints till the defect has been localized to a specific line of code. Plot a dependency graph of this line of code using ENSOFT ATLAS. This will highlight all the nodes that we need to look at to identify where the problem lies. The binary search approach should identify the defective line of code in log(N) number of steps.

SEARCH FOR DEFECTS: BINARY SEARCH ON A GRAPH Difficulties: Programs have call back functions and return functions that make the graph a 2d tree and not a simple 1d tree (linked list) The search algorithm has to be optimized for binary search on a 2d tree Typically, set breakpoints at the ends (entry and exit points) of the loops before setting any break points inside the loops Set breakpoints at the branching conditions instead of return statements This would reduce the effort required since loops are executed multiple number of times and you can eliminate a large chunk of repetitive code using a single break point if it is at the entry or exit point Scope for automation of this algorithm with evaluation of all conditions and storing results in a buffer is available for future work.

BINARY SEARCH: TRIAL WALKTHROUGH First breakpoint Second breakpoint

REFERENCES 1. Dorothy Graham, Erik van Veenendaal, Isabel Evans, Rex Black, “Foundations of Software Testing (ISTQB Certification)” 2. Zimmermann, T.; Nagappan, N., "Predicting defects with program dependencies," Empirical Software Engineering and Measurement, 2009. ESEM 2009. 3rd International Symposium on , vol., no., pp.435,438, 15-16 Oct. 2009 3. Vijay D’Silva, Daniel Kroening, and Georg Weissenbacher, “A Survey of Automated Techniques for Formal Software Verification” IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS, VOL. 27, NO. 7, JULY 2008 4. Eichinger, F. (2011). Data-mining techniques for call-graph-based software-defect localisation (Doctoral dissertation, Karlsruher Inst. für Technologie, Diss., 2011). 5. Deering, T., Kothari, S., Sauceda, J., & Mathews, J. (2014, May). Atlas: a new way to explore software, build analysis tools. In Companion Proceedings of the 36th International Conference on Software Engineering (pp. 588- 591). ACM. 6. Jon Jouis Benteley, Stanford University, “Multi-dimensional binary search trees, used for associative searching.” Communications of the ACM, September 1975, Volume 18, Number 9