Presentation is loading. Please wait.

Presentation is loading. Please wait.

Quick Checking Your Java Classes With T2 Wishnu Prasetya Dept. of Information & Computing Sciences Utrecht University www.cs.uu.nl/~wishnu.

Similar presentations


Presentation on theme: "Quick Checking Your Java Classes With T2 Wishnu Prasetya Dept. of Information & Computing Sciences Utrecht University www.cs.uu.nl/~wishnu."— Presentation transcript:

1 Quick Checking Your Java Classes With T2 Wishnu Prasetya Dept. of Information & Computing Sciences Utrecht University

2 Overview What is T2, and its key ideas Implementation Programming Specifications Related Work Result & Research Status Conclusion 2

3 3 T2 = Testing Tool A tool to automatically test a Java class with some interesting features... Unit testing. Inspired by QuickCheck

4 4 QuickCheck and T2 Both generates tests randomly Not pure random, but with some direction  passive direction from specs. One is for functional, the other for OO

5 5 Example : my own sorted list public class SortedList { private LinkedList s private Comparable max public SortedList() // initialize s to empty public void insert(Comparable x) public Comparable get() }

6 6 In-code Specification public class SortedList { private LinkedList s private Comparable max public SortedList() public void insert(Comparable x) public Comparable get() // return and remove max } public boolean classinv() { return  s.isEmpty()  s.contains(max) } public Comparable get_spec() { assert  s.isEmpty() : "PRE" Comparable ret = get() assert …  ret.compareTo(s.getLast())  0 : "POST" return ret } Less abstract Cannot guarantee non-interference Declarative Powerful Minimum maintenance

7 7 What if my specification is partial ? private s private max // classinv :  s.isEmpty()  s.contains(max) public void insert(Comparable x) { int i = 0 for (Comparable y : s) { if (y.compareTo(x) > 0) break i++ } s.add(i,x) if (max==null || x.compareTo(max) < 0) max = x } No pre/post-conditions Class inv. Is partial Most tools do isolated testing; it will Leak faults produce losts of false positives

8 8 Sequences as Tests Simplification  We only have private and public members  Updating a field F is represented by a call to setF We’ll talk about a class C A test on C is: a (finite) sequence of calls to pubmeth(C)

9 9 private s private max // classinv :  s.isEmpty()  s.contains(max) public void insert(Comparable x) // faulty public Comparable get() // return and remove max public Comparable get_spec() { assert !s.isEmpty() : "PRE" Comparable ret = get() assert …  ret.compareTo(s.getLast())  0 : "POST" return ret } Back to example u = new SortedList u.insert(1) ; u.insert(10) u.get() Hey, with sequence I can catch the fault, despite my partial spec! It also autogenerates ‘interesting’ target objects.

10 10 Functions vs Objects f :: A  B  Its behavior is fully determined by the relation between its parameter and result.  Deterministic. An object u:C  Its behavior (transition from state to state) is determined by all C’s methods.  Non-deterministic.  From this persperctive it is natural to use sequence.

11 Implementation 1. Representation of tests 2. T2 algorithm

12 Representing a test As a Java (Jtest) code Most testing tools do this... Alternatively, represent a test as ordinary data  In Haskell you can exploit higher order functions  In Java  object. Possible via “meta” objects from Reflection library. 12 void test1 () { try { u = new SortedList u.insert(1) ; u.insert(10) u.get_spec() } catch (AssertError e) …

13 13 Meta representation of a test class CALL_METHOD Method m MkValStep[] params class CONST Object val A test is a list of : class CREATE_TARGET_OBJECT Constructor con MkValStep[] params class REF int index Object pool This is just a list of objects; you can program over it! E.g. : Producing the corresponding execution On the fly checking Save, reload, regress Pretty print, including intermediate states Mutate Difficult, if we are to do these directly over Junit code.

14 14 Simple Algorithm Top level:  Generate (and check) N number of tests  Save violating tests Generating a test   Create a target object u ;  =   Repeat until violation, or maxlegth : choose a method m  pubmeth(C) extend  with a call to m check classinv and m_spec

15 Checking a step u.m (... ) 1. If m_spec exists, call it instead. Catch these: 1. PRE AssertError  discard  2. Other AssertError  spec violation 3. Other Error or RunTimeException  internal violation 4. Other Exception  no violation (expected) 2. Check classinv on u, catch these: 1. APPMODEL AssertError  discard  2. Other Throwable  violation + warning 3. Nothing thrown, but returns false  violation 15

16 16 Every step in a test, is a call m( …. ) Create a fresh object by calling a constructor (via reflection). Reuse a previously created object.  Also cover over side effects!  Require you to maintain a pool of objects. T2 additionally allows you to supply a base domain:  Is searched first  can force range  Use cloning  side effect free Whatever the decision, it is represented as a MkValStep object

17 17 Main Testing API Class RndEngine static void RndTest (Pool p, BaseDomain d, InterfaceMap m, Class C, String[] options )

18 18 ** Error trace [1] : ** CREATING target object: 0 s 1 max NULL ** STEP 1. ** Calling method insert with: ** Receiver: target-obj ** Arg [0:] (Integer) : 29 ** Target object after the step: 0 s 3 [0] (Integer) : 29 max (Integer) : 29 ** STEP 2. ** Calling method insert with: ** Receiver: target-obj ** Arg [0:] (Integer) : 0 ** Target object after the step: 0 s 5 [0] (Integer) : 0 [1] (Integer) : 29 max (Integer) : 0 ** STEP 3. ** Calling method get_spec with: ** Receiver: target-obj ** Throwing java.lang.AssertionError: POST ** Target object after the step: 0 s 3 [0] (Integer) : 29 max (Integer) : 29 ** Assertion VIOLATED! ** Strack trace: java.lang.AssertionError: POST at U2.T2.examples.SortedList.get_spec(Sorted List.java:44)...

19 Programming Specifications 1. Application model 2. Temporal properties

20 Abstract View 20 CApplication m1m1 m2m2  -step.... skip the state is the object pool Automaton ANY(C). Implicitly assume App respects C’s ownership claim. A behavior of C is an (infinite) path in ANY. A property is a predicate over paths. A test is a finite prefix of a path. Btw, a single threaded model !

21 Application Model 21 m1m1 m2m2  -step.... skip What if we know something about App? Model this assumption as constraints over ANY. Don’t constrain  -steps. You can still constrain e.g. order in which methods of C are called. T2 only cheks the m-steps! To force T2 to check the constraint, put it in the the classinv.

22 Example 22 private String aux_laststep = null private int aux_delta = 0 public boolean appmodel() { if (aux_laststep.equals("insert")) aux_delta++ if (aux_laststep.equals("get")) aux_delta-- return aux_delta>=0 ; } will be filled by T2 boolean classinv() { assert appmodel() : “APPMODEL” return …. // old spec }

23 Temporal Property Properties are predicates over paths  so they are all temporal. Example. For all M,S we require: s == S && max == M unless open Checking a predicate over paths implies we have to maintain paths. Complicated!  T2 only maintains current state. If the the state-preds are closed in u, the prop is  - insensiteve! 23 Add it to SortedList; change methods etc.

24 Rephrase the property Introduce aux-vars to maintain past states. Now we can express the property as a classinv : aux_open_frozen || (aux_s_frozen.equals(s) && aux_max_frozen==max) || open 24 Also extends classinv to update the aux-vars accordingly.

25 Some Related Work 1. Verification 2. Other testing approaches

26 How is T2 related to Verification ? Checking temporal property  essentially as in explicit- state model checking (SPIN)  MC generates paths from models  Finite states  can generate all paths! JPAX : runtime verification tool  Generates finite paths from a real application  Obviously you can’t generate all paths; so just pick some.  Problem: no control over execution  fault may only surface after a long execution T2  generate finite paths from a real unit  Full control over the generation of paths 26 v v

27 Other Path-based Testing Approaches TestLog (Ducasse et al)  Mine paths from execution of a real application  Use Prolog to ‘query’ properties over those paths Palulu (Artzi et al)  Mine paths from execution of a real application  Use them to construct an application model CnC  Isolated (not path-based) approach  Use a combo of Hoare-logic/theorem prover/constraint solver to calculate tests data 27

28 Some Results Class#Meth Aver. Cyclo Max. Cyclo # Inst Meth Cov. (%) Block Cov. (%) Time (ms)Fault#step BinarySearch Tree No20 K SortedList yes0.5 K Show no5 K Pool no5 K

29 Not a Swiss Knife… Some classes are not suitable for T2. E.g. two main classes in T2 : RndEngine  only has one main API Its internal behavior (e.g. loop invariant) is more interesting. Trace  can’t generate meaningful instances on its own.. Class#Meth Aver. Cyclo Max. Cyclo#instr Trace146, RndEngine157,

30 Research Status On going. Tool  prototype, but fully functional  Paper and documentation provided Open issues (Master thesis topics)  Is it really useful? Can it scale up? Making the random testing more directed, e.g. by using WP-logic Dealing with ownership issue  Testing User Interface  Applying complementary ideas from other approaches 30

31 Conclusion On-going. S/W :  Not an all-purpose solution  Cannot handle concurrency  Strong features  Preliminary results seem promising 31


Download ppt "Quick Checking Your Java Classes With T2 Wishnu Prasetya Dept. of Information & Computing Sciences Utrecht University www.cs.uu.nl/~wishnu."

Similar presentations


Ads by Google