1. A Novel Approach to Unit Test: The Aspect-Oriented Way Guoqing Xu and Zongyuan Yang Software Engineering Lab (SEL) East China Normal University http://www.cs.ecnu.edu.cn/sel/~harryxu Oct.21 2004 at ISFST Oct.21 2004 at ISFST

2. Background Test Oracle problem Test Oracle problem -- how to identify oracles in unit test? -- how to identify oracles in unit test? Automating the test oracle generation Automating the test oracle generation -- Manually writing test codes is a labor-intensive job. -- Manually writing test codes is a labor-intensive job. -- Cheon and Leavens automatically generate JUnit test codes and use JML runtime assertions as test oracle to test the Java programs annotated with JML specifications [CL02]. -- Cheon and Leavens automatically generate JUnit test codes and use JML runtime assertions as test oracle to test the Java programs annotated with JML specifications [CL02]. Automating the test case generation Automating the test case generation -- Korat generates test cases based on java predicts. [BKM02] -- Korat generates test cases based on java predicts. [BKM02] -- JMLAutoTest generates test cases based on JML class invariants specified in the program. [XY03] -- JMLAutoTest generates test cases based on JML class invariants specified in the program. [XY03]

3. Problems Current test automating process relies on formal assertions and predicts. Current test automating process relies on formal assertions and predicts. Traditional specifications only focus on programs ’ functional behaviors, with little support for specifying their non-functional behaviors, e.g. temporal logic, performance... Traditional specifications only focus on programs ’ functional behaviors, with little support for specifying their non-functional behaviors, e.g. temporal logic, performance... How to find oracles to test non-functional aspects? How to automate this process? How to find oracles to test non-functional aspects? How to automate this process?

4. Our approach In Aspect-Oriented Programming (AOP), crosscutting properties are monitored to model the program from different aspects, a lot of tasks which have been difficult to be handled in traditional ways are easily done. In Aspect-Oriented Programming (AOP), crosscutting properties are monitored to model the program from different aspects, a lot of tasks which have been difficult to be handled in traditional ways are easily done. Using a crosscutting property of the program as the criterion to check the correctness of the application in the corresponding aspect is well suited to the unit testing problems. Using a crosscutting property of the program as the criterion to check the correctness of the application in the corresponding aspect is well suited to the unit testing problems.

5. Application-specific Aspect A new notion: application-specific aspects (ASS) A new notion: application-specific aspects (ASS) -- top-level application related aspects. -- top-level application related aspects. -- established at the design level. -- established at the design level. -- may be picked up from low language level aspects. -- may be picked up from low language level aspects. -- all the ASS for the same use share some common features, e.g. testing ASS, tracing ASS... -- all the ASS for the same use share some common features, e.g. testing ASS, tracing ASS... -- may be translated into language level aspects. -- may be translated into language level aspects.

6. Aspect-Oriented Test Description Language How to describe ASS? How to describe ASS? -- a formal way is needed. -- a formal way is needed. -- can not be too complicated. -- can not be too complicated. Aspect-Oriented Test Description Language(AOTDL) Aspect-Oriented Test Description Language(AOTDL) -- uses the first order predict logic as part of its syntax -- uses the first order predict logic as part of its syntax -- used by the designer at design level -- used by the designer at design level -- can be translated into AspectJ aspects. -- can be translated into AspectJ aspects. Basic units in AOTDL Basic units in AOTDL -- Utility unit -- Utility unit -- MeaninglessCase Advice unit -- MeaninglessCase Advice unit -- Error Advice unit -- Error Advice unit advicetype (arguments): pointcuts: conditions: message advicetype (arguments): pointcuts: conditions: message

7. AOTDL (cond.) Class Stack{ public void init(){...} public void init(){...} public void push ( Node n){...} public void push ( Node n){...}......} TestingAspect tempLogic{ Utility{ protected boolean isInitialized = false; protected boolean isInitialized = false; //push is reached //push is reached pointcut pushReached(Stack st): pointcut pushReached(Stack st): target(st)&&call(void Stack.push(Integer)); target(st)&&call(void Stack.push(Integer)); //init is reached //init is reached pointcut initReached(Stack st): pointcut initReached(Stack st): target(st)&&call(void Stack.init(void)); target(st)&&call(void Stack.init(void)); // after advice // after advice after(Stack st):initReached (st){ after(Stack st):initReached (st){ isInitialized = true; isInitialized = true; }}

8. AOTDL (cond.) MeaninglessCase Advice{ /*advices for specifying criteria of meaningless test cases */ /*advices for specifying criteria of meaningless test cases */ before(Stack s) : before(Stack s) : pushReached(s) : pushReached(s) : s.getSize() >=MAX : ” Overflow ” ; s.getSize() >=MAX : ” Overflow ” ;...... } Error Advice{ /*advices for specifying criteria of test errors */ /*advices for specifying criteria of test errors */ before(Stack s): before(Stack s): pushReached(s): pushReached(s): ! isInitialized : ” Not Initialized ” ; ! isInitialized : ” Not Initialized ” ;...... }}

9. AOTDL (cond.) //error advices //error advices before(Stack s) throws TestErrorException: before(Stack s) throws TestErrorException: pushReached(s){ pushReached(s){ if (!isInitialized){ if (!isInitialized){ TestErrorException ex =new TestErrorException ex =new TestErrorException( “ Not Initialized ” ); TestErrorException( “ Not Initialized ” ); ex.setSource( “ TempLogic ” ); ex.setSource( “ TempLogic ” ); throw ex; throw ex; }}

10. JAOUT: Automated Generation of AO Testing Framework

11. JAOUT takes Java class M.java as the input, and automatically generate JUnit test framework. JAOUT takes Java class M.java as the input, and automatically generate JUnit test framework. -- Aspect_M_Test.java, the JUnit unit test class. -- Aspect_M_Test.java, the JUnit unit test class. -- Aspect_M_TestCase.java, the test case provider. -- Aspect_M_TestCase.java, the test case provider. -- Aspect_M_TestClient.java, JMLAutoTest test case generation class. -- Aspect_M_TestClient.java, JMLAutoTest test case generation class.

12. Test Suite Definition For class C and its method M(A1 a1, A2 a2 … An an), the generated test suite is defined as For class C and its method M(A1 a1, A2 a2 … An an), the generated test suite is defined as --- C[ ] receivers --- C[ ] receivers -- A1[ ] vA1;... ; An[ ] vAn; -- A1[ ] vA1;... ; An[ ] vAn; There is a corresponding init_Ai method for each type Ai and a method init_receiver in test case provider to initialize test cases. There is a corresponding init_Ai method for each type Ai and a method init_receiver in test case provider to initialize test cases. Testers use APIs provided by JMLAutoTest to generate test case space in test client, and pass it to the test case provider. Testers use APIs provided by JMLAutoTest to generate test case space in test client, and pass it to the test case provider.

13. Generated Test Method public void testM(){ for (int i0 = 0; io < receiver.length; i0++){ for (int i1 = 0; i1 < a1.length; i1++){ for (int i1 = 0; i1 < a1.length; i1++){ … try { try { receiver[i0].M(a1[i1], : : :, an[in]); receiver[i0].M(a1[i1], : : :, an[in]); } catch (MeaninglessCaseException e) { } catch (MeaninglessCaseException e) { /*... tell framework test case was meaningless... */ /*... tell framework test case was meaningless... */ continue; continue; } catch (TestErrorException e) { String msg = e.getSource(); String msg = e.getSource(); fail(msg + NEW LINE + e.getMessage()); fail(msg + NEW LINE + e.getMessage()); } catch (java.lang.Throwable e) { } catch (java.lang.Throwable e) { continue; continue; } finally { } finally { setUp(); // restore test cases setUp(); // restore test cases }}}

14. Test Result...in push...in push false false F Time: 0.06 Time: 0.06 There was 1 failure: There was 1 failure: 1) testPush (sample.Stack_Aspect_TestCase)junit.framework.AssertionFailedError: In Testing Aspect TempLogic: Not Initilized! 1) testPush (sample.Stack_Aspect_TestCase)junit.framework.AssertionFailedError: In Testing Aspect TempLogic: Not Initilized! at ample.Stack_Aspect_Test.testPush at ample.Stack_Aspect_Test.testPush (Stack_Aspect_Test.java:155) (Stack_Aspect_Test.java:155) at sun.reflect.NativeMethodAccessorImpl.invoke0 at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) (Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:39) (NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:25) (DelegatingMethodAccessorImpl.java:25) at sample.Stack_Aspect_Test.run(Stack_Aspect_Test.java:26) at sample.Stack_Aspect_Test.run(Stack_Aspect_Test.java:26) at sample.Stack_Aspect_TestCase.main at sample.Stack_Aspect_TestCase.main (Stack_Aspect_TestCase.java:24) (Stack_Aspect_TestCase.java:24) FAILURES!!! FAILURES!!! Tests run: 3, Failures: 1, Errors: 0, Meaningless:0 Tests run: 3, Failures: 1, Errors: 0, Meaningless:0

15. Related Work (Spec-based test) TestEra – Automating OO test generation. [MK01] MIT TestEra – Automating OO test generation. [MK01] MIT JMLUnit – Generating test oracles from JML runtime assertions. [CL02] Iowa State Univ. JMLUnit – Generating test oracles from JML runtime assertions. [CL02] Iowa State Univ. Korat – Generating test case based on Java predicts. [BKM02] MIT. Korat – Generating test case based on Java predicts. [BKM02] MIT. JMLAutoTest – Generating test framework from JML runtime assertions and test cases based on class invariants. [XY03] ECNU. JMLAutoTest – Generating test framework from JML runtime assertions and test cases based on class invariants. [XY03] ECNU. Jov -- java automated unit test based on inferred program properties. [XN03] Univ. of Washington. Jov -- java automated unit test based on inferred program properties. [XN03] Univ. of Washington.

16. Conclusions Traditional formal predicts tend not to deal with non-functional properties of the program. Traditional formal predicts tend not to deal with non-functional properties of the program. AOP is well suited to the unit test problems. AOP is well suited to the unit test problems. Designers use AOTDL to build Application- Specific Testing Aspects. Designers use AOTDL to build Application- Specific Testing Aspects. JAOUT translates Testing Aspects to AspectJ aspects automatically. JAOUT translates Testing Aspects to AspectJ aspects automatically. JAOUT automatically generates JUnit test framework and uses the runtime messages thrown by Testing Aspects as test oracles. JAOUT automatically generates JUnit test framework and uses the runtime messages thrown by Testing Aspects as test oracles.

17. Thank you … Questions?