Precise Condition Synthesis for Program Repair Y. Xiong at el. International Conference on Software Engineering 2017 Presentor : Jee-weon Jung

Index Overview Introduction Motivating example Approach Evaluation

Overview Object : produce PRECISE patches Concentrate on condition synthesis Study three approaches and finally propose ACS Know what variables should exist in a ‘if’ condition  use sorting method based on dependency relations between variables Propose document analysis technique on API documents  filter variables Mine a set of frequently used predicates Successfully repaired 18 defects on 4 projects of Defects4J Best performance reported, precision was 78.3% (others usually 40%)

Introduction (1) Automatic program repair techniques generate patch that match specification Test suite is mostly used as specification Patch is considered ‘plausible’ if it passes all tests Concept of precision is adtopted in plausible patches Precision is VERY important Low precision patches decrease efficiency Precision of well known programs such as GenProg is about 4% Reason : precise patch is sparse in the space of repair systems

Introduction (2) Rank patches to achieve high precision Studied widely (Prophet, HistoricalFix, DirectFix, …) This paper proposes more accurate ranking criteria for condition synthesis Condition synthesis : insert / modify ‘if’ condition One of most strongest techniques Proposed method decompose condition synthesis Variable selection : decide which variable to use Predicate selection : decide what to do

Introduction (3) Example : ‘if (a > 10)’ First select variable ‘a’  select predicate ‘>10’ Use proposed 3 techniques for ranking variables and predicates Dependency-based ordering : more recent dependency showing variable is more likely to be used in condition Document analysis : analyze Javadoc comments in the src code Predicate mining : extract from existing projects  sort by frequency

Motivating example (1) Math99 defect in Defects4J dataset 2 test cases given ‘a=1,b=50’ / ‘a=Integer.MIN_VALUE,b=1’ Many plausible conditions exist  most are not precise Proposed ACS addes lines 2~4 Synthesize the condition at line 2

Motivating example (2) Dependency-based ordering Document Analysis In the previous example: ‘lcm’ depends on ‘a’ and ‘b’  ‘lcm’ is more likely to be used Use dependency in ordering Document Analysis Mnay java methods come with Javadoc comments  assist condition synthesis using comments Consider ‘@throws’ tag in the document Analyze the subject of sentence Consider only the mentioned variables when making guard condition

Motivating example (3) Predicate Mining Use variable type, name as context Ex : ‘hour’ frequently used with >24, <12, … Ex : ‘factorial’ frequently used with <21 (20! Is the largest factorial representable with 64bit integer

Approach (1) - overview Use 2 types of templates Directly return the oracle Identify the last executed statement s in the failed test  insert one of if (c) return v / if (c) throw e before the statement Modification of an existing condition Locate potential faulty ‘if’ condition c  apply either widening (c  c || c) / narrowing (c  c && c) c : synthesized condition Use SBFL & predicate switching for locating potential faulty condition

Approach (2) – extracting the oracle Three types Constant : directly copy Specified via XXXException.class annotation : throw an instance Function mapping the test input to the output : complicated.. First perform backward slicing from the oracle expression perform backward slicing from the test input arguments

Approach (3) – variable ranking 3 steps : preparing candidate  filtering by document analysis  sort using dependency-based ordering Preparing candidate variable Consider 4 types : local variable / method parameter / ‘this’ pointer / expressions used in other ‘if’ conditions in the current method Sorting by dependency Create a dependency graph between variables Node : variable / dege : dependency relation

Approach (4) – predicate ranking Mining related conditions Use variable type, name, and method name to decide context Two word is considered similar when word decomposition by capital letter has common word Variable name is considered meaningful when length >=2 Synthesize a condition c with variable x in the method m A conditional expression c0 is considered to be in a similar context of c, if (1) it contains one variable x0, (2) x0 has the same type as x, (3) the name of x0 is similar to x when the name of x is meaningful, or the name of the method surrounding c0 is similar to m when the name of x is not meaningful.

Approach (5) – counting predicates Using collected similar expressions, extracted used predicates Choose among only predefined predicates only Expanding search space often leads to incorrect patches Syntactically differenct predicates may semantically be the same Apply ‘pred’ function to extract multiset  sort by frequencies  heuristically select top 20

Evaluation (1) Implementation Dataset Based on the source ocde of Nopol Use fault localization library Gzoltar Apache OpenNLP is exploited Dataset Top five most starred Java projects on GitHub (2016/7/15) Four projects from Defects4J

Evaluation (2) Research questions RQ1: how do the three ranking techniques perform on ranking variables and predicates? RQ2: how does our approach perform on real world defects? RQ3: how does our approach compare with existing approaches? RQ4: to what extent does each component of our approach contribute to the overall performance?

Evaluation (3) RQ1: performance of the three techniques

Evaluation (4) RQ1: performance of the three techniques

Evaluation (5) RQ1: performance of the three techniques

Evaluation (6) RQ2: performance of ACS

Evaluation (7) RQ3: comparison with existing approaches

Evaluation (8) RQ4: detailed analysis of the components

Conclusion Study refined ranking techniques for condition synthesis Achieve high precision and reasonable recall on Defects4J