1. Anti-patterns in Search-based Program Repair
Shin Hwei Tan*
Hiroaki Yoshida+
Mukul R. Prasad+
Abhik Roychoudhury*
National University of Singapore*
Fujitsu Laboratories of America, Inc.+
**Part of this work was conducted during at an internship at Fujitsu Laboratories of America

2. What is automated program repair?
BUG!
Given failing Test T , buggy program P
Fault localization- Where to fix?
Patch Generation - How to fix?
Patch Validation – Are all tests passing?

3. Previous work on Search-Based Program Repair
GenProg [ICSE '12]
PAR [ICSE '13]
SPR [FSE '15]
Search
Genetic Programming
Improved version for adaptive equivalent search (AE)
Evolutionary Algorithm
Specialized Search with condition synthesis
Operators
Mutations & crossovers
Patterns learned from Human Patches
Patterns with and without abstract condition
Program
C Programs
Java Programs

4. Search-Based Program Repair
All Tests Pass
Final Patch
Candidate Patches
Search-Based Repair Tools
Patch Generation
Tests
contains at least one failing test
Tests Fail
How do the tests look like?
Patch Evaluation
How do the patches look like?

5. Search-Based Program Repair
$command $argument1 $argument2
RETVAL=$?
[ $RETVAL -eq 0 ] && echo Success
[ $RETVAL -ne 0 ] && echo Failure
Test Script
Check exit status of command
Non-zero exit status denotes
test failure
exit(-2);
Candidate Patches
Tests
Patch Evaluation

6. Repair patterns from human patches
Automatic Program Repair
Introduce Conditional Control-Flow
Loosen Condition
Insert Initialization …

7. Repair patterns from human patches
Automatic Program Repair
Introduce Conditional Control-Flow
Loosen Condition
Insert Initialization …

8. Repair patterns from human patches
Automatic Program Repair
Conditional Control Flow:
+if(a)
+ return b;
int foo(){
+ if(input1)
+ return(out1)
//compute something …}
Anti-patterns
Set of generic forbidden transformations
that can be enforced on top of any search-based repair tool.

9. Prevalence of Anti-patterns
Anti-patterns are prevalent in automatically generated patches

10. Problem: Weak Oracle
Statements like exit call/assertions serve as test proxies
Test proxies should not be randomly manipulated
$command $argument1 $argument2
RETVAL=$?
[ $RETVAL -eq 0 ] && echo Success
[ $RETVAL -ne 0 ] && echo Failure
Failing Test Script
Test outcome determined by exit status
A1: Anti-delete CFG exit node
Remove return statements, exit calls, functions with the word “error”, assertions.
static void BadPPM(char* file) {
fprintf(stderr, "%s: Not a PPM file.\n", file);
- exit(-2); }

11. Problem: Inadequate Test Coverage
Repair tools allow removal of code as long as all test passes
Statements are mistakenly considered as redundant code
Anti-patterns:
A2: Anti-delete Control Statement
A3: Anti-delete Single-statement CFG
A4: Anti-delete Set-Before-If
A2: Anti-delete Control Statement
Remove control statements (e.g., if-statements, switch-statements, loops).
call_result = call_user_function_ex(...);
- if (call_result == SUCCESS && ...) {
- if (SUCCESS == statbuf_from_array(...))
ret = 0;
- } else if (call_result == FAILURE) {…

12. Problem: Inadequate Test Coverage
int methodA (int x, int y) {
int z = x/y;
return x*y; }
Consider redundant code since x/y is computed but never used
Could be removed by dead-code elimination
But removing this line eliminates potential division-by-zero error, hence removal could cause different output
Statements may be considered redundant code to test-driven repair tools
Test suite with inadequate coverage allows removal of code as long as all test passes
Anti-patterns:
A2: Anti-delete Control Statement
A3: Anti-delete Single-statement CFG
A4: Anti-delete Set-Before-If

13. Problem: Inadequate Test Coverage
Anti-pattern: A2
Deletion of
CFG node 1 2 3 5 4 7 6 8
PASS
FAIL
A2: Anti-delete Control Statement
Remove control statements (e.g., if-statements, switch-statements, loops).
call_result = call_user_function_ex(...);
- if (call_result == SUCCESS && ...) {
- if (SUCCESS == statbuf_from_array(...))
ret = 0;
- } else if (call_result == FAILURE) {…

14. Problem: Inadequate Test Coverage
Original CFG 1 2 3 5 4 7 6 8
PASS
FAIL
Empty CFG node 1 2 3 5 7 6 8
PASS
FAIL
Anti-pattern: A3
A3: Anti-delete Single-statement CFG
Remove the statement in CFG node with only one statement
fail:{
- ret = 1; }

15. Problem: Mask Existing Vulnerabilities
Existing vulnerabilities can be masked by removing branches via implicit data-flow
A4: Anti-delete Set-Before-If
Remove variable definition if the variable is used in subsequent if-statement.
- tmp = EstimateStripByteCounts(tif, dir, dircount)<0;
if(tmp!=0)
goto bad;

16. Problem: Non-termination
Automatically generated patches may incorrectly removes loop update
Cause infinite loop
A5:Anti-delete Loop-Counter Update
Remove assignment statement A inside loop L if: 𝑉𝑎𝑟 𝑖𝑛 𝑇𝑒𝑟𝑚𝑖𝑛𝑎𝑡𝑖𝑜𝑛 𝐶𝑜𝑛𝑑𝑖𝑡𝑖𝑜𝑛 𝑜𝑓 𝐿 ∩ {𝑉𝑎𝑟 𝑖𝑛 𝐿𝐻𝑆 𝑜𝑓 𝑎𝑠𝑠𝑖𝑔𝑛𝑚𝑒𝑛𝑡 𝐴}=∅
while( x> 5)
- x++;

17. Problem: Trivial Patch
Trivial patch – patch that insert return-statements based on expected output
Ex: +if(test1)
+ return out1
A6: Anti-append Early Exit
Insert return/goto statement at any location except for after the last statement in a CFG node.
+ if ((type != 0))
+ return;
zend_error((1<<3L),"Uninitialized string offset:",...);

18. Problem: Functionality Removal
Removes functionality by inserting T/F
A7: Anti-append Trivial Conditions
Insert trivial condition.
A condition is trivial if and only if it is:
True/False Constant
Tautology/Contradiction in expression (e.g., if(x || y || !y))
Static analysis (e.g., if(x || y != 0), y is initialized)
- if ((fmap[j].key != format->ptr[i + 1]))
+ if ((fmap[j].key != format->ptr[i + 1]) && !(1))
continue;

19. Integrating Anti-patterns
Final Patch
Search-Based Repair Tools
Patch Generation
Tests Fail
All Tests Pass
Candidate Patches
Tests
contains at least one failing test
YES NO
Is Anti-pattern?
Patch Evaluation

20. Experimental Setup Corebench benchmark GenProg benchmark
Subjects
Description
kLoC
Tests
coreutils
File, Shell and Text manipulation Utility
83.1
4772
findutils
Directory Searching Utility
18.0
1054
grep
Pattern Matching Utility
9.4
1582
make
Program executable generation utilities
35.3
528
php
Programming Language
1046
8471
libtiff
Image Processing Library 77 78
python
407 35
gmp
Math Library
145
146
gzip
Data Compression Utility
491 12
wireshark
Network Packet Analyzer
2814 63
fbc
Compiler 97
773
lighthttpd
Web Server 62
295
Corebench
benchmark
GenProg
benchmark
Evaluation on:
GenProg AE
– Uses Adaptive Equivalent Search
SPR
– Uses staged repair algorithm with repair patterns
dGenProg
– GenProg without deletion operator

21. Evaluation of Anti-patterns
RQ1:
How do anti-patterns affect the quality of patches generated by search-based program repair tools?
RQ2:
How many nonsensical patches can our anti-patterns eliminate to reduce manual inspection costs?
RQ3:
When our modified tools produce the same patch, what is the speedup that we achieve?
RQ4:
How does the use of anti-patterns compare to an approach
that simply prohibits deletion?

22. Measuring Patch Quality (RQ1)
Human generated patch: H
Patch generated by Tool: A
Original Repair Tool: To
Modified Repair Tool: Tm
Localizes Correct Line. A and H modify the same line
Localizes Correct Function but Incorrect Line. A and H modify statements within the same function.
Removes Less Functionality. A removes or skips over fewer lines of code from original program.

23. Measuring Patch Quality (RQ1)
Human generated patch: H
Patch generated by Tool: A
Original Repair Tool: To
Modified Repair Tool: Tm
Same Patch. To and Tm generate same repair.
Localizes Correct Line. A and H modify the same line
Localizes Correct Function but Incorrect Line. A and H modify statements within the same function.
Removes Less Functionality. A removes or skips over fewer lines of code from original program.
No Repair. To generates a repair but Tm produce no patch.

24. RQ1: Patch Quality mGenProg outperforms Genprog in:
Localizes Correct Line
Localizes Correct Function but Incorrect Line
Removes Less Functionality
Similar improvement in mSPR

25. RQ4: Comparison with GenProg without deletion (dGenProg)
mGenProg outperforms dGenprog in:
Localizes Correct Line
Localizes Correct Function but Incorrect Line
mGenProg removes less functionality than GenProg without deletion!
dGenProg removes functionality via insertions of return/goto statements

26. RQ1: Patch Quality Same Patch: 26 Localizes Correct Line: 1 vs. 15
Localizes Correct Function but Incorrect Line: 15 vs. 12
Removes Less Functionality: 19
Anti-patterns produce better patches in terms of fix localization and removal of functionality in mGenProg
Same Patch: 42
Localizes Correct Line: 14 vs. 8
Removes Less Functionality: 10
Anti-patterns produce better patches in terms of fix localization and removal of functionality in mSPR

27. RQ1: Patch Quality GenProg vs. mGenProg
Same Patch: 26
Localizes Correct Line: 1 vs. 15
Localizes Correct Function but Incorrect Line: 15 vs. 12
Removes Less Functionality: 19
Anti-patterns produce better patches in terms of fix localization and removal of functionality in mGenProg

28. RQ1: Patch Quality SPR vs. mSPR
Same Patch: 42
Localizes Correct Line: 14 vs. 8
Localizes Correct Function but Incorrect Line: 15 vs. 12
Removes Less Functionality: 10
Anti-patterns produce better patches in terms of fix localization and removal of functionality in mSPR

29. RQ2: Reducing Manual Inspection Cost
Due to batch compilation, SPR may produce multiple patches
Overall, SPR generates 87 patches, mSPR only generates 54 patches.
All 33 additional patches are plausible but incorrect patches.
Anti-patterns reduce manual inspection cost by eliminating nonsensical repairs.

30. RQ3: Speedup mGenProg’s Average Repair Time Speedup: 1.42x
Repair Space Reduction (%) = 1-( 𝑀𝑜𝑑𝑖𝑓𝑖𝑒𝑑 𝑇𝑜𝑡𝑎𝑙 𝑅𝑒𝑝𝑎𝑖𝑟 𝐶𝑎𝑛𝑑𝑖𝑑𝑎𝑡𝑒𝑠 𝑂𝑟𝑖𝑔𝑖𝑛𝑎𝑙 𝑇𝑜𝑡𝑎𝑙 𝑅𝑒𝑝𝑎𝑖𝑟 𝐶𝑎𝑛𝑑𝑖𝑑𝑎𝑡𝑒𝑠 )*100
Repair Time Speedup = 𝑂𝑟𝑖𝑔𝑖𝑛𝑎𝑙 𝑅𝑒𝑝𝑎𝑖𝑟 𝑇𝑖𝑚𝑒 Modified Repair Time
mGenProg’s Average Repair Time Speedup: 1.42x
mSPR’s Average Repair Time Speedup: 1.69x
Anti-patterns reduce overall repair time via repair space pruning.

31. Conclusions and Future Work
Enforcing anti-patterns leads to patches that delete less functionality and better fix localization.
Tools integrated with anti-patterns generate patches faster due to repair space reduction.
Future Work
Anti-patterns as specification for guiding repair
Anti-patterns as selected “code smells”
Tools integrated with anti-patterns can be used as fix localization tools
Adapt anti-patterns to other search-based software engineering activities (e.g., specific code anti-patterns identifying energy hot-spots)