1. Authors: Nirav Desai, prof. hema gaikwad
International Conference on Computing, Communication and Automation 2015 Galgotias University Gurgaon
Paper ID: 707
Graph based verification and identification of defects by binary search on a graph
Authors: Nirav Desai, prof. hema gaikwad
Symbiosis institute of computer studies and research, PUNE, MAHARASHTRA

2. VERIFICATION AND VALIDATION
Software testing is divided into 2 parts: Verification and Validation.
Verification relates to the passive inspection of code by visualizations and walk-throughs.
It helps to identify whether we are building the system right
Validation is dynamic testing of the code using different test patterns which is used to identify if the system is functioning correctly
It helps to identify if we built the right system.
ISO 9126 and IEEE STANDARD 829 define the standard methodologies for software testing.
Test plan strategies (model based, analytical risk based or requirements based, standards driven, etc)
Test environment setup
Work breakdown structure and resource breakdown structure
Test Results Documentation

3. Software Defect Lifecycle
Defect Open
Troubleshoot
Defer
Reject
Defect Close
Test
Defect Identification
Defects are identified
by testers during the
verification and validation process
If the test lead accepts the
identified defects,
a new defect is opened
The software developers can
either debug and fix the defect,
defer the debugging to new
development cycle or
reject the defect as duplicated
Software Tester verifies the software
operation against a reference set
The debugged program is tested again and if no defects are found,
defect is closed, else the defect is re-opened

4. VERIFICATION
Problems in the software are identified by code walkthroughs and passive inspection
Helps to check the process of software coding by checking for errors at the time they are introduced
Graph based verification helps in easier walk-throughs of the code.
A visual inspection of the graph of the software control flow helps to identify the number of test cases.
Cyclomatic complexity of graph = e – n + 2*g
Where e = number of edges, n = number of nodes and g = no. of independent sub graphs

5. CYCLOMATIC GRAPH OF CODE
No. of edges = 8
No. of nodes = 7
No. of test cases = 8 – = 3
Test Case 1: A = 354
Test Case 2: A != 354 and B > C
Test Case 3: A != 354 and B < C
Reference: Foundations of Software Testing by Rex Black

6. DEPENDENCY GRAPH OF CODE
DEPENDENCY GRAPH GENERATED USING ENSOFT ATLAS:
HIGHLIGHTS ALL DEPENDENCIES FOR A GIVEN LINE OF CODE

7. SEARCH FOR DEFECTS: LIBLIT05
Liblit05 algorithm uses statistical techniques to search for defects in a code
For each predicate P’ in program P, Liblit05 estimates two conditional probabilities respectively:
Pr1 = Pr(P fails|P’ is ever observed) and Pr2 = Pr(P fails|P’ is ever observed as true).
It then treats the probability difference Pr2−Pr1 as an indicator of how relevant P is to the bug.
Therefore, Liblit05 essentially regards the predicates whose true evaluations correlate with program failures as bug-relevant.
Because Liblit05 only considers whether a predicate has ever been evaluated as true in one run, it loses its discrimination power when a predicate P is observed as true at least once in every execution.
This algorithm does not look at how many times a given predicate is evaluated so the algorithm fails when there are problems with loops.

8. SEARCH FOR DEFECTS: SOBER
A has an extra chance to be true in incorrect runs.
Program 1 was tested with 5,542 test cases that are available in the Siemens suite.
The average probability of A being evaluated as true within one incorrect run is and it is in correct ones.

9. SEARCH FOR DEFECTS: BINARY SEARCH ON A GRAPH
Set the break point in execution half way through the code and test if the output is correct.
If output is as expected, the defect is in the latter half of the code or else in the previous half.
If the defect is in the latter half of the code, set the break point at the half way mark of the remaining code. For a defect in the preceding part of the code, set the break point at the half way mark of the preceding section.
Repeat this process of setting breakpoints till the defect has been localized to a specific line of code.
Plot a dependency graph of this line of code using ENSOFT ATLAS. This will highlight all the nodes that we need to look at to identify where the problem lies.
The binary search approach should identify the defective line of code in log(N) number of steps.

10. SEARCH FOR DEFECTS: BINARY SEARCH ON A GRAPH
Difficulties:
Programs have call back functions and return functions that make the graph a 2d tree and not a simple 1d tree (linked list)
The search algorithm has to be optimized for binary search on a 2d tree
Typically, set breakpoints at the ends (entry and exit points) of the loops before setting any break points inside the loops
Set breakpoints at the branching conditions instead of return statements
This would reduce the effort required since loops are executed multiple number of times and you can eliminate a large chunk of repetitive code using a single break point if it is at the entry or exit point
Scope for automation of this algorithm with evaluation of all conditions and storing results in a buffer is available for future work.

11. BINARY SEARCH: TRIAL WALKTHROUGH
First breakpoint
Second breakpoint

12. REFERENCES
1. Dorothy Graham, Erik van Veenendaal, Isabel Evans, Rex Black, “Foundations of Software Testing (ISTQB Certification)”
2. Zimmermann, T.; Nagappan, N., "Predicting defects with program dependencies," Empirical Software Engineering and Measurement, ESEM rd International Symposium on , vol., no., pp.435,438, Oct. 2009
3. Vijay D’Silva, Daniel Kroening, and Georg Weissenbacher, “A Survey of Automated Techniques for Formal Software Verification” IEEE TRANSACTIONS ON COMPUTER-AIDED DESIGN OF INTEGRATED CIRCUITS AND SYSTEMS, VOL. 27, NO. 7, JULY 2008
4. Eichinger, F. (2011). Data-mining techniques for call-graph-based software-defect localisation (Doctoral dissertation, Karlsruher Inst. für Technologie, Diss., 2011).
5. Deering, T., Kothari, S., Sauceda, J., & Mathews, J. (2014, May). Atlas: a new way to explore software, build analysis tools. In Companion Proceedings of the 36th International Conference on Software Engineering (pp ). ACM.
6. Jon Jouis Benteley, Stanford University, “Multi-dimensional binary search trees, used for associative searching.” Communications of the ACM, September 1975, Volume 18, Number 9