Precisely Selecting Regression Test for Aspect-Oriented Programs Guoqing Xu The Ohio State University

Outline Big picture and background Problem statement and motivation Our analysis Implementation status Related work

Big Picture AOSD improves separation of concerns Challenges for analysis of AO programs  Control flow in bytecode is not equivalent to the program logic in source code  Compiler-specific weaving  Dynamic/static parts of pointcut designators  No analysis framework implementation available Challenges for testing of AO programs  Coverage criteria; test generation for exercising of aspects; regression test selection

Big Picture Long-term goals  Framework that supports static analyses similar to popular analyses for OO program  Develop AO-specific techniques for analysis and testing Topic of this talk  Regression test selection for AO programs  Work in progress

Regression Test Selection Testing after software modifications Re-running the entire regression test suite is expensive Select a subset of tests to run Safe test selection chooses every test case that may reveal a fault

Related Previous Work Rothermel and Harrold, TOSEM 97  Graph traversal algorithms Harrold et al., OOPSLA 01  Java interclass graph (JIG) Copied from [Harrold et al., OOPSLA 01] Execute P/ Record Coverage Identify Dangerous Entities Select Tests Program P Program P’ Program P Program P’ P’s Edge Coverage Matrix Dangerous Entities containing edges in P

Regression Testing for AOSD When AO features are added/modified, the program needs to be regression tested Case 1: P is an OO program and P’ is an AO version of P Case 2: Both P and P’ are AO programs How should regression test selection be performed for AO software? How should regression test selection be performed for AO software?

Outline Big picture and background Problem statement and motivation Our analysis Implementation status Related work

Existing Work Applied to AO Programs How to compare two JIGs?  The JIG of the woven code includes redundant nodes and edges and does not correspond to the logical control flow as presented in the source  Need new representations How to recover CFG edges from the execution trace when computing edge- coverage matrix? (when P is AO program)  The execution trace is compiler-specific  Need instrumentation before/during weaving

Existing Work Applied to AO Programs 1.class C { 2. public void m(int i) {…} 3. } 4. 5.void bar(C p, int k) { 6. p.m(k); 7. } Java version P CFG edge Call edge Java Interclass Graph (JIG) bar() p.m() return C.m() … exit

Example 1.class C { 2. public void m(int i){…}; 3. } 4. 5.void bar(C p, int k) { 6. p.m(k); 7. } aspect Sample{ 10. void around(C c, int i): 11. call(C.m(int)) && 12. target(c) && 13. args(i) { 14. proceed(p, x); 15. } bar() p.m() Retur n C.m() … exit JIG for P AspectJ Version P’ C. m () … exit bar().. around$0 return … c.m() return JIG for P’ (from woven code)..Sample. around$0

Some Results When we applied the [Harrold et.al. 01] algorithm to several subjects: SubjectsV1 (java version) Num of tests V2 (AspectJ version) Num of selected tests bean 10 introduction 40 observer 10 telecom 30 tjp 10 tracing 40 testReadFile 20 19

Possible Approach Create “clean” CFGs in which the wrapper code inserted during weaving is removed  Graph traversal and comparison corresponds to the “logical” structure of the code, not the compiler-specific woven code New representation: AJIG  AspectJ Inter-module Graph – more later For regression test selection, need to consider additional issues

A More Complex Problem 1.class C { 2. public void m(int i){…}; 3. } 4. 5.void bar(C p, int k) { 6. C.m(k); 7. } aspect Sample{ 10. void around(C c, int i): 11. call(C.m(int)) && 12. target(c) && 13. args(i) { 14. System.out.println(i); 15. proceed(p, x); 15. } C. m () bar() return m() AJIG for P’ … exit System. out…. Do we need to select all the tests that go through the edge marked in red?

Why it needs to be addressed This is an issue not only for AO software, but also for procedural and OO software Advices are often free of side effects  Study in [Rinard et al. FSE 04] reported 6 “observer” advices out of ten inspected advices  Recommended for “safe” AO programming Adding side-effect-free advices should not result in overly conservative regression test selection Approach: use side-effect analysis

Outline Big picture and background Problem statement and motivation Our analysis Implementation status Related work

Our Work Consider both situations:  Case 1: P is an OO version, P’ is an AO version  Case 2: both P and P’ are AO versions Analysis to select regression tests  Build a new control flow representation: AJIG  Apply existing graph-traversal algorithm on AJIG  Side-effect analysis when comparing AJIGs

AJIG AspectJ Inter-module Graph (recent work)  For the Java parts, same as JIG.  Shadow node  A shadow node is associated with  a set of JIGs of advices  the precedence of these advices  an integrated shadow advice JIG  AJIG supports all static AspectJ join point types  Conservatively approximate the dynamic part of pointcut designators.

Side Effect Side effect node  Mutate the objects that have existed before the method/advice is invoked For an advice,  All the nodes in the CFG path that does not contain a proceed node  The proceed node has side effect if its actual parameters and parameters of the advice don’t point to same objects.  Mutates the object returned by the proceed node  The return node has side effect if the return value of the advice and that of the proceed node don’t point to the same object

Side Effect (Cont’d) Side effect related node in AJIG  Has side effects  Has some dependency on the nodes that have side effects Safe edge in AJIG  An edge is safe edge, if the sink node of this edge is not side effect related.

New Test Selection criterion New Test Selection criterion:  Dangerous set S computation: for each edge e in P, and its counter part e’ in P’ e is not equivalent to e’ and Neither e nor e’ is a safe edge. Select a test that execute one or more edges in S

Selection Computing dangerous set S by comparing AJIGs. We plan to use some form of side-effect analysis  Large body of existing work Selecting tests

Implementation progress The implementation of algorithms described in [Harrold et.al. 01] √ Building AspectJ Inter-module Graph √  Make an extension for the abc compiler that generates the Jimple based CFGs for aspects between the weaving of inter-type declarations and advices.  Instrument advices at different phases. Side effect analysis ongoing work Evaluation

Evaluation Plan Benchmarks: ten to twenty moderate size ( classes) AO projects taken from tmbenches, used by the abc compiler. Experiments  collect data. The number of tests selected over versions. The comparison between enabling side-effect analysis and disabling it.

Related Work Static/Dynamic Analysis for AO programs.  abc compiler [AOSD 05] [PLDI 05] [TR 04].  Static analysis of aspects [Sereni and Moor, AOSD 02].  Zhao’s work on the analysis and testing of AO programs [COMPSAC 03] [WPC 02][AOSD 06].  Classification system for AO programs [Rinard et. al. FSE 04].

Related Work (Cont’d) Regression Test Selection  [LW ICSM 91], [CRV ISCE 94], [RH TOSEM 97], [Ball ISSTA 98], [Harrold+ OOPSLA 01], [OSH FSE 04]… Change Impact Analysis  [KGH+ ICSM 94], [RT PASTE 01], [OAH FSE 03], [OAL+ ICSE 04], [RST+ OOPSLA 04], …

Thank you!! Questions??