1. A Test Case + Mock Class Generator for Coding Against Interfaces
Mainul Islam, Christoph Csallner
Software Engineering Research Center (SERC)
Computer Science and Engineering Department
University of Texas at Arlington, USA
February 07, 2011.
Published at: 8th International Workshop on Dynamic Analysis (WODA), 2010

2. Mock Class Generator for Coding Against Interfaces
Talk Outline
Test case generation
Dynamic Symbolic Execution
Mock Class
Motivating examples to generate Mock Class
Algorithm to generate Mock Class
Results
Demonstration
Mock Class Generator for Coding Against Interfaces

3. Mock Class Generator for Coding Against Interfaces
Test Case Generation
What is the goal behind testing ?
public void Test(int x) {
if (x > 0) {
if (x == ) {
throw new Exception(“Bug”); }
Probability to reach the bug (with a random test input) is:
½ * 1/2^31
What can be a good test case to test this method ?
Mock Class Generator for Coding Against Interfaces

4. Mock Class Generator for Coding Against Interfaces
Symbolic Execution
Explore all feasible execution paths
Initialize the input with symbolic values and execute the program over symbolic values
At conditional statements check if either of the branches can be taken (using a constraint solver)
For each path we get an accumulated path condition
If (s) then … else … C s
true
false
C` = C ⋀ s
C` = C ⋀⌝s
Dynamic symbolic execution: Combine Concrete and Symbolic execution
Mock Class Generator for Coding Against Interfaces

5. Dynamic Symbolic Execution by Example
Choose Next Path
void TestMe (int a[]) { if (a == null) return; if (a.length > 0) if (a[0] == 123) throw new Exception(“Error”); }
Solve
Execute
Constraints to Solve
Input
Observed Constraints
null
a == null
a == null
a != null {}
a != null &&
!(a.length > 0)
a == null T F
a != null &&
a.length > 0
{0}
a != null &&
a.length > 0 &&
a[0] != 123
a.length > 0 T
a != null &&
a.length > 0 &&
a[0] == 123
{123}
a != null &&
a.length > 0 &&
a[0] == 123 F
a[0] == 123 T F
Done: No path left
This example is taken from the slides of Nikolai Tillman (with permission)

6. Introduction to Mock Class
public interface A {
// … }
public class C {
public void foo (A a, int x) {
Mock Class Generator for Coding Against Interfaces

7. When we might need to generate mock-classes ?
At the initial stage of any development
Mock Class Generator for Coding Against Interfaces

8. Mock Class Generator for Coding Against Interfaces
Why is this important ?
Test case generation is important
Interfaces are very important
Existing techniques are not very good at this
Pex - can generate an instance of a class that implements an interface that is used in the code * *
Mock Class Generator for Coding Against Interfaces

9. Mock Class Generator for Coding Against Interfaces
Goal:
Support interfaces in test case generation for object-oriented programs
Mock Class Generator for Coding Against Interfaces

10. Mock Class Generator for Coding Against Interfaces
Motivating Example 1
public class C {
public void foo (A a, int x) {
int y = a.m1(x);
if (a instanceof B) {
B b = (B) a;
int z = b.m2(x); }
public interface A {
int m1(int x); }
public interface B {
int m2(int x);
To reach this block of code, ‘a’ must be an instance of A, as well as an instance of B
Mock Class Generator for Coding Against Interfaces

11. Mock Class Generator for Coding Against Interfaces
Motivating Example 2
public class K
implements A, B {
// … }
public class C {
public void bar (A a, int x) {
if (a == null) return;
if ( !(a instanceof B) ) {
B b = (B) a;
int z = b.m2(x);
public interface A {
int m1 (int x); }
public interface B {
int m2 (int x);
K implements all of the interfaces referred by the bar method, but fails reach the code block
Mock Class Generator for Coding Against Interfaces

12. Mock Class Generator for Coding Against Interfaces
Framework
DSC – Dynamic Symbolic Execution Engine (for Java)
Z3 – Constraint Solver
Mock Class Generator for Coding Against Interfaces

13. Our approach to generate Mock Class
Determining the type of the Mock Class
Generate meaningful method body if needed
Mock Class Generator for Coding Against Interfaces

14. Sub-/Supertype relation in Java
Java defines a binary subtype relation
If type B implements/extends type A then,
- A is a direct super type of B
- B is a direct sub type of A
Reflexive: A is also a subtype of itself
Transitive: if B is a subtype of A and C is a subtype of B then C is also a subtype of A
Mock Class Generator for Coding Against Interfaces

15. Sub-/Supertype relation in Java (Cont.)
A class has one direct class super type and arbitrarily many interface super types.
Exceptions:
type object – has no direct super type
type null – has no direct sub type
Mock Class Generator for Coding Against Interfaces

16. Subtype constraints Object
public interface A { int m1 (int x); } public interface B { int m2(int x); public class C { public void foo (A a, int x) { int y = a.m1(x); if (a instanceof B) { B b = (B) a; int z = b.m2(x);
Object A B C A B C M
null
Initial types in the system
null
A desired solution: with new type M
Constraints:
type(a) subtypeof A
type(a) != null type
type(a) subtypeof B
Mock Class Generator for Coding Against Interfaces

17. Subtype Relation Matrix
public interface A { int m1 (int x); } public interface B { int m2(int x); public class C { public void foo (A a, int x) { int y = a.m1(x); if (a instanceof B) { B b = (B) a; int z = b.m2(x);
Null
Object A B C x 1 2 3 4
Null
Object A B C M x 1 2 3 4 5
Null
Object A B C M x 1 2 3 4 5 MA MB MC
Null
Object x 1
Solution:
MA = true
MB = true
MC = false
Mock Class Generator for Coding Against Interfaces

18. Algorithm: Generating Mock Class
Program execution
Initialization:
NO_MC = 0
MAX_MC = 2
Build Subtype Relation matrix (Generate the constraints)
Yes
Found Solution? No
Solution not found
NO_MC < MAX_MC No
Yes
So, here is how our algorithm works.
Once in a while the normal execution of the program is suspended and the correctness of the data structure is checked. If this check by calling repOk() method fails, our
By the way, in this slide I-repOk means Instrumented repOk
Yes
Introduce a new Mock Class, N0_MC++
Encode the Mock Class in the type system
Generate Test Case
Dynamic Symbolic Data Structure Repair

19. Generating Method Body: for Example 1
public interface A {
int m1 (int x); }
public interface B {
int m2(int x);
public class C {
public void foo (A a, int x) {
int y = a.m1(x);
if (a instanceof B) {
B b = (B) a;
int z = b.m2(x);
if (z >= 10) {
//…
public class M implements A, B {
int m1 (int x){
return 0; }
int m2(int x){
Outline of the source code for Mock Class M
return 10;
Mock Class Generator for Coding Against Interfaces

20. Results We have run our solution on JMS (Java messaging system)
These are all hand-written test cases
Interfaces used from JMS
Number of goals
Number of goals reached w/o mock-classes
Number of goals reached with mock-classes
Message 6
ByteMessage 5
MapMessage 4
StreamMessage
TextMessage
ObjectMessage
Mock Class Generator for Coding Against Interfaces

21. Comparison with Pex: Motivating Example 1
public class C {
public void foo (A a, int x) {
int y = a.m1(x);
if (a instanceof B) {
B b = (B) a;
int z = b.m2(x); }
public interface A {
int m1(int x); }
public interface B {
int m2(int x);
Goal 1
Goal 2
PEX reaches 1/2 goal(s)
DSC reaches 2/2 goal(s)
Mock Class Generator for Coding Against Interfaces

22. Mock Class Generator for Coding Against Interfaces
Future Work
Extend the solution to support more types, e.g. ‘Array’
Determine a upper bound to the number of mock classes needed to solve a system
Evaluation
Mock Class Generator for Coding Against Interfaces

23. Mock Class Generator for Coding Against Interfaces
Future Work (Cont.)
Extend method body generation for complex types
public class C {
public void foo (A a, int x) {
int y = a.m1(x);
if (a instanceof B) {
B b = (B) a;
int z = b.m2(x);
if (z >= 10) {
//… }
public class M implements A, B {
int m1 (int x){
return 0; }
int m2(int x){
Outline of the source code for Mock Class M
return 10;
Mock Class Generator for Coding Against Interfaces

24. Mock Class Generator for Coding Against Interfaces
Demo
Mock Class Generator for Coding Against Interfaces

25. Mock Class Generator for Coding Against Interfaces
Thank you!
Mock Class Generator for Coding Against Interfaces