1. Modularity, Interfaces, and Verification Tevfik Bultan Department of Computer Science University of California, Santa Barbara

2. Joint Work with My Students Action Language Verifier (ALV) –Tuba Yavuz-Kahveci, University of Florida, Gainesville Web Service Analysis Tool (WSAT) –Xiang Fu, Hofstra University (co-advised with Jianwen Su) Design for verification –Aysu Betin-Can, Middle East Technical University Interface Grammars –Graham Hughes, PhD candidate http://www.cs.ucsb.edu/~bultan/vlab/publications.html

3. Outline Motivation PART 1: Verification of Synchronization Policies in Concurrent Programs via Interfaces PART 2: Verification of Conversations among Web Services via Interfaces PART 3: Modular Verification with Interface Grammars Conclusions

4. Model Checking Software Model checking –An automated software verification technique –Exhaustive exploration of the state space of a program to find bugs Systematically explore all possible behaviors of a program –look for violations of the properties of interest assertion violations, deadlock Software model checkers: Verisoft, Java PathFinder (JPF), SLAM, BLAST, CBMC

5. Two Challenges in Model Checking State space explosion –Exponential increase in the state space with increasing number of variables and threads State space includes everything: threads, variables, control stack, heap Environment generation –Finding models for parts of software that are either not available for analysis, or are outside the scope of the model checker

6. Modular Verification Modularity is key to scalability of any verification technique –Moreover, it can help in isolating the behavior you wish to focus on, removing the parts that are beyond the scope of your verification technique Modularity is also a key concept for successful software design –The question is finding effective ways of exploiting the modularity in software during verification

7. Interfaces for Modularity How do we do modular verification? –Divide the software to a set of modules –Check each module in isolation How do we isolate a module during verification/testing? –Provide stubs representing other modules (environment) How do we get the stubs representing other modules? –Write interfaces Interfaces specify the behavior of a module from the viewpoint of other modules Generate stubs from the interfaces

8. Interfaces and Modularity: Basic Idea 1.Write interface specifications for the modules 2.Automatically generate stubs from the interface specifications 3.Automatically generated stubs provide the environment during modular verification

9. Three Applications I will talk about three different instantiations of this basic idea: 1.Verification of synchronization policies in concurrent programs using finite state interfaces 2.Verification of conversations among web services using finite state interfaces 3.Verification of sequential interactions using interface grammars

10. PART 1 Concurrency Controller Pattern for Synchronization

11. An Infinite State Model Checker Action Language Parser Model Checker OmegaLibraryCUDDPackage MONA Composite Symbolic Library PresburgerArithmeticManipulator BDDManipulatorAutomataManipulator Action Language Specification + CTL property Counter-exampleVerified Not sure Action Language Verifier (ALV)

12. Read-Write Lock in Action Language module main() integer nr; boolean busy; restrict: nr>=0; initial: nr=0 and !busy; module ReaderWriter() enumerated state {idle, reading, writing}; initial: state=idle; r_enter: state=idle and !busy and nr’=nr+1 and state’=reading; r_exit: state=reading and nr’=nr-1 and state’=idle; w_enter: state=idle and !busy and nr=0 busy’ and state’=writing; w_exit: state=writing and !busy’ and state’=idle; ReaderWriter: r_enter | r_exit | w_enter | w_exit; endmodule main: ReaderWriter*(); spec: invariant(busy => nr=0) spec: invariant(busy => eventually(!busy)) endmodule S : Cartesian product of variable domains defines variable domains defines the set of states the set of states I : Predicates defining the initial states the initial states R : Atomic actions of a single process a single process R : Transition relation of a process, defined as asynchronous composition of its atomic actions R : Transition relation of main, defined as asynchronous composition of finite but arbitrary number of reader-writer modules

13. Read-Write Lock in Java class ReadWriteLock { private Object lockObj; private int totalReadLocksGiven; private boolean writeLockIssued; private int threadsWaitingForWriteLock; public ReadWriteLock() { lockObj = new Object(); writeLockIssued = false; } public void getReadLock() { synchronized (lockObj) { while ((writeLockIssued) || (threadsWaitingForWriteLock != 0)) { try { lockObj.wait(); } catch (InterruptedException e) { } } totalReadLocksGiven++; } } public void getWriteLock() { synchronized (lockObj) { threadsWaitingForWriteLock++; while ((totalReadLocksGiven != 0) || (writeLockIssued)) { try { lockObj.wait(); } catch (InterruptedException e) { // } } threadsWaitingForWriteLock--; writeLockIssued = true; } } public void done() { synchronized (lockObj) { //check for errors if ((totalReadLocksGiven == 0) && (!writeLockIssued)) { System.out.println(" Error: Invalid call to release the lock"); return; } if (writeLockIssued) writeLockIssued = false; else totalReadLocksGiven--; lockObj.notifyAll(); } } } How do we translate this to Action Language? Action Language Verifier Verification of Synchronization in Java Programs

14. A Design for Verification Approach Our design for verification approach is based on the following principles: 1.Use of design patterns that facilitate automated verification 2.Use of stateful, behavioral interfaces which isolate the behavior and enable modular verification 3.An assume-guarantee style modular verification strategy that separates verification of the behavior from the verification of the conformance to the interface specifications 4.A general model checking technique for interface verification 5.Domain specific and specialized verification techniques for behavior verification

15. Controller Shared Concurrency Controller Pattern ThreadA ThreadB StateMachine Controller -var1 -var2 +action1() +action2() Action +blocking() +nonblocking() -GuardedExecute SharedStub +a() +b() Shared +a() +b() GuardedCommand +guard() +update() int GuardedCommand ControllerStateMachine +action1() +action2() used at runtime used during interface verification used both times Helper classes

16. Concurrency Controller Pattern Avoids usage of error-prone Java synchronization primitives: synchronize, wait, notify Separates controller behavior from the threads that use the controller –Supports a modular verification approach that exploits this modularity for scalable verification

17. class Action{ protected final Object owner; … private boolean GuardedExecute(){ boolean result=false; for(int i=0; i<gcV.size(); i++) try{ if(((GuardedCommand)gcV.get(i)).guard()){ ((GuardedCommand)gcV.get(i)).update(); result=true; break; } }catch(Exception e){} return result; } public void blocking(){ synchronized(owner) { while(!GuardedExecute()) { try{owner.wait();} catch (Exception e){} } owner.notifyAll(); } } public boolean nonblocking(){ synchronized(owner) { boolean result=GuardedExecute(); if (result) owner.notifyAll(); return result; } } class RWController implements RWInterface{ int nR; boolean busy; final Action act_r_enter, act_r_exit; final Action act_w_enter, act_w_exit; RWController() {... gcs = new Vector(); gcs.add(new GuardedCommand() { public boolean guard(){ return (nR == 0 && !busy);} public void update(){busy = true;}} ); act_w_enter = new Action(this,gcs); } public void w_enter(){ act_w_enter.blocking();} public boolean w_exit(){ return act_w_exit.nonblocking();} public void r_enter(){ act_r_enter.blocking();} public boolean r_exit(){ return act_r_exit.nonblocking();} } Reader-Writer Controller This helper class is provided. No need to rewrite it!

18. Controller Interfaces A controller interface defines the acceptable call sequences for the threads that use the controller Interfaces are specified using finite state machines public class RWStateMachine implements RWInterface{ StateTable stateTable; final static int idle=0,reading=1,writing=2; public RWStateMachine(){... stateTable.insert("w_enter",idle,writing); } public void w_enter(){ stateTable.transition("w_enter"); }... } writing reading idle r_enter r_exit w_exit w_enter

19. Interface Machine Thread 1Thread 2Thread n Thread 1 Controller Shared Data Interface Machine Thread 2 Interface Machine Thread n Thread Modular Interface Verification Concurrent Program Controller Behavior Modular Behavior Verification Modular Design / Modular Verification Interface

20. Concurrent Program Controller Classes Thread Classes Controller Interface Machine Controller Behavior Machine Java Path Finder Action Language Verifier Thread Isolation Thread Class Counting Abstraction Interface Verification Behavior Verification Verification Framework

21. A Case Study Tactical Separation Assisted Flight Environment (TSAFE) –21,057 lines of code with 87 classes Reengineered TSAFE using 2 instances of a reader-writer controller and 3 instances of a mutex controller –In the reengineered version the synchronization statements appear only in the Action helper class Created 40 faulty versions of TSAFE by fault seeding –Caught 33 of them using interface and behavior verification –4 of the uncaught faults were spurious –3 uncaught faults were real faults and they were interface violations

22. Behavior Verification Performance RW0.171.03 Mutex0.010.23 Barrier0.010.64 BB-RW0.136.76 BB-Mutex0.631.99 ControllerTime(sec)Memory (MB)P-Time (sec)P-Memory (MB) 8.1012.05 0.980.03 0.010.50 0.6310.80 2.056.47 P denotes parameterized verification for arbitrary number of threads

23. Interface Verification Performance ThreadTime (sec)Memory (MB) TServer-Main67.7217.08 TServer-RMI91.7920.31 TServer-Event6.5710.95 TServer-Feed123.1283.49 TClient-Main2.002.32 TClient-RMI17.0640.96 TClient-Event663.2133.09

24. Falsification Performance TServer-RMI29.4324.74 TServer-Event6.889.56 TServer-Feed18.5194.72 TClient-RMI10.1242.64 TClient-Event15.6312.20 ThreadTime (sec)Memory (MB) RW-80.343.26 RW-161.6110.04 RW-P1.515.03 Mutex-80.020.19 Mutex-160.040.54 Mutex-p0.120.70 Concurrency ControllerTime (sec)Memory (MB)

25. Observations Falsification performance is better than verification performance Completeness of the controller properties is an issue For falsification, concrete instances were as effective as parameterized instances Unknown shared objects: need escape analysis Thread isolation is challenging Environment generation is the crucial problem in scalability of the interface verification

26. PART 2 Peer Controller Pattern for Web Services

27. Web Services Loosely coupled, interaction through standardized interfaces Standardized data transmission via XML Asynchronous messaging Platform independent (.NET, J2EE) Data Type Service Composition Message WSBPEL Web Service Standards Implementation Platforms Microsoft.Net, Sun J2EE WSDL SOAP XML Schema XML WSCDL Interaction

28. Web Service Conversations A composite web service consists of –a finite set of peers –and a finite set of message classes The messages among the peers are exchanged using reliable and asynchronous messaging –FIFO and unbounded message queues A conversation is a sequence of messages generated by the peers during an execution

29. ?query !suggest Agent !query ?suggest !reserve !query ?suggest Customer Hotel AgentCustomerHotel query suggest reserve confirm Input Queue... Conversation ?confirm !confirm ? G(query  F(confirm)) ?query !suggest ?reserve Checking Conversations This is an undecidable verification problem due to unbounded queues

30. Synchronizability Analysis A composite web service is synchronizable, if its conversation set does not change –when asynchronous communication is replaced with synchronous communication If a composite web service is synchronizable we can check its properties about its conversations using synchronous communication semantics –For finite state peers this is a finite state model checking problem

31. BPEL to GFSA Guarded automata GFSA to Promela (bounded queue) BPEL Web Services Promela Synchronizability Analysis GFSA to Promela (synchronous communication) Intermediate Representation Conversation Protocol Front End Realizability Analysis Guarded automaton skip GFSA parser success fail GFSA to Promela (single process, no communication) success fail AnalysisBack End (bottom-up) (top-down) Verification Languages Web Service Analysis Tool (WSAT)

32. Checking Service Implementations There are some problems: People write web service implementations using programming languages such as Java, C#, etc. Synchronizability analysis works on state machine models How do we generate the state machines from the Java code? Synchronizability Analysis Checking Service Implementations

33. Design for Verification Approach Use the same principles: 1.Use of design patterns that facilitate automated verification 2.Use of stateful, behavioral interfaces which isolate the behavior and enable modular verification 3.An assume-guarantee style modular verification strategy that separates verification of the behavior from the verification of the conformance to the interface specifications 4.A general model checking technique for interface verification 5.Domain specific and specialized verification techniques for behavior verification

34. ApplicationThreadCommunicationInterface StateMachine CommunicatorCommunicationController PeerServletThreadContainer sessionId Peer Controller Pattern used at runtime used during interface verification used both times Red Bordered classes are the ones the user has to implement

35. Interface Machine Peer 1 Peer 2Peer n Peer 1 Interface Machine Peer 2 Interface Machine Peer n Peer Modular Interface Verification Composite Service Conversation Behavior Modular Conversation Verification Modular Design / Modular Verification interface

36. Composite Service Peer State Machine Promela Java Path Finder Spin Peer Code Interface Verification Verification Framework Thread Peer State Machines WSAT Synchronizability Analysis Conversation Verification Promela Translation

37. Examples We used this approach to implement several simple web services –Travel agency –Loan approver –Product ordering Performance of both interface and behavior verification were reasonable

38. Interface Verification Interface Verification with JPF for Loan Approver ThreadsT (sec)M (MB) Customer8.863.84 Loan Approver 9.654.7 Risk Assesor 8.153.64

39. Behavior Verification Sample Property: Whenever a request with a small amount is sent, eventually an approval message accepting the loan request will be sent. Loan Approval system has 154 reachable states – because queue lengths never exceeds 1 Behavior verification used <1 sec and 1.49 MB SPIN requires restricted domains –Have to bound the channel sizes  bounded message queues In general there is no guarantee these results will hold for other queue sizes –Using synchronizability analysis we use queues of size 0 and still guarantee that the verification results hold for unbounded queues!

40. Observations Once the behavior is isolated (using concurrency controller or peer controller patterns) behavior verification is quite efficient Interface verification is challenging It is necessary to find effective behavioral interface specification and verification techniques

41. PART 3 Interface Grammars

42. Interface Grammars Component A Component B Interface Grammar Interface Compiler Component B Stub Component A Model Checker Interface Grammar

43. An Example An interface grammar for transactions –Specifies the appropriate ordering for method calls to a transaction manager –Method calls are the terminal symbols of the interface grammar S tart → Base Base →begin Tail Base |ε |ε Tail →commit |rollback |rollback

44. An Example Consider the call sequence begin rollback begin commit Here is a derivation: Start  Base  begin Tail Base  begin rollback Base  begin rollback begin Tail Base  begin rollback begin commit Base  begin rollback begin commit S tart → Base Base →begin Tail Base |ε |ε Tail →commit |rollback |rollback

45. Another Example This interface can also be specified as a Finite State Machine (FSM) However, the following grammar, which specifies nested transactions, cannot be specified as a FSM Start → Base Base →begin Base Tail Base |ε |ε Tail →commit |rollback |rollback begin commit rollback

46. Yet Another Example setrollbackonly which forces all the pending transactions to finish with rollback instead of commitLet’s add another method called setrollbackonly which forces all the pending transactions to finish with rollback instead of commit We achieve this by extending the interface grammars with semantic predicates and semantic actionsWe achieve this by extending the interface grammars with semantic predicates and semantic actions Start →«r:=false; l:=0» Base Base →begin «l:=l+1» Base Tail «l:=l-1; if l=0 then r:=false» Base «l:=l-1; if l=0 then r:=false» Base |setrollbackonly «r:=true» Base |setrollbackonly «r:=true» Base |ε |ε Tail →«r=false» commit |rollback |rollback

47. Interface Grammar Translation Our interface compiler translates interface grammars to executable code: –the generated code is the stub for the component The generated code is a parser that –parses the incoming method calls –while making sure that the incoming method calls conform to the interface grammar specification

48. Verification with Interface Grammars Interface Grammar Interface Compiler Program Model Checker Top-down parser parse table semantic predicates and semantic actions parser stack Component Stub method invocation (lookahead)

49. A Case Study We wrote an interface grammar for the EJB 3.0 Persistence API – This is an API specification for mapping Java object graphs to a relational database – Hibernate is an implementation of this API Hibernate distribution contains several example test clients that are designed to fail and test exceptional behavior by violating the interface specification

50. A Case Study, Continued We used these simple clients to check the fidelity of the stub generated from our interface specification – We used the JPF software model checker None of these examples can run under JPF directly Time taken to develop the interface was dominated by the need to understand EJB Persistence first – about a couple of hours

51. Experiments: Falsification sec. # obj.# iter. 1.91.8 2.111 1.91.8 2.115 1.91.8 2.151 1.91.8 2.155 Client 1Client 2Client 3 Client 4

52. Experiments: Verification sec. # obj.# iter. 3.11.91.82.511 26.911.19.917.115 10.15.94.211.851 126.363.943.5138.555 Client 1Client 2Client 3 Client 4

53. A Case Study, Continued For these simple clients, interface violations can be detected by JPF in a couple of seconds using the EJB stub generated by our interface compiler –Falsification time does not increase with the number of operations executed or the number of objects created by the clients When we fix the errors, JPF is able to verify the absence of interface violations –Verification time increases with the number of operations executed or the number of objects created by the clients

54. Interface Grammars: Uni/Bidirectional Interface grammars can be –Unidirectional: No callbacks –Bidirectional: Need to handle Callbacks CallerCallee Interface Comp AComp B Interface

55. Interface Grammars: Client/Server Interface grammars can be used for –Client verification: Generate a stub for the server –Server verification: Generate a driver for the server Interface Compiler Interface Client Stub Server Driver

56. Semantic Elements in JML To handle both client and server side verification –We need to generate stubs and drivers from the same specification Semantic predicate in one direction becomes a semantic action in the other direction and visa versa We focused on a subset of Java Modeling Language –A restricted subset that is reversable –Semantic predicates and actions are written in this subset –Interface compiler automatically generates code from them both for client and server side verification

57. Interface Grammars and Data A crucial part of the interface specification is specifying the allowable values for the method arguments and generating allowable return values Approach 1: These can be specified in the semantic actions and semantic predicates of the interface grammars Approach 2: Can we specify the constraints about the arguments and return values using the grammar rules? –Yes, grammar productions can be used to specify the structure of most recursive data structures.

58. Checking Arguments A crucial part of the interface specification is specifying the allowable values for the method arguments and generating allowable return values In what I discussed so far all these are done in the semantic actions and semantic predicates The question is, can we specify the constraints about the arguments and return values using the grammar rules –Recursive data structures are especially good candidates for this!

59. Shape Types Shape types [Fradet, Metayer, POPL 97] provide a formalism for specifying recursive data structures It is a specification formalism based on graph grammars Shape types can be used to specify the connections among the heap allocated objects Objects become the parameters of the nonterminals and the constraints on the connections among the objects are specified on the right-hand-sides of the grammar rules (similar to semantic predicates)

60. Shape Type for Doubly Linked List 12 next prev p 3 next prev 4 next prev next prev Doubly →p x, prev x null, L x L x →next x y, prev y x, L y L x →next x null p 1, prev 1 null, L 1 Doubly  p 1, prev 1 null, L 1  next 1 2, prev 2 1, L 2  next 2 3, prev 3 2, L 3  next 3 4, prev 4 3, L 4  next 4 null

61. Shape Type for Binary Tree 1 right left p 2 3 5 right left right left 4 right left Bintree →p x, B x B x →left x y, right x z, B y, B z B x →left x null, right x null

62. Extension to Interface Grammars In order to support shape types we extend the interface grammars as follows: –We allow nonterminals with parameters This extension is sufficient since the constraints about the connections among the objects can be stated using semantics predicates and semantic actions

63. Interface Grammars + Shape Types Doubly →p x, prev x null, L x L x →next x y, prev y x, L y L x →next x null Doubly[x] → ensure x == \new(Node) && x.getPrev() == null; L[x] L[x] → Node y; ensure y == \new(Node) && x.getNext() == y && y.getPrev() == x; L[y] | ensure x.getNext() == null;

64. Interface Grammars + Shape Types Bintree →p x, B x B x →left x y, right x z, B y, B z B x →left x null, right x null Bintree[x] → ensure x == \new(Node); B[x] B[x] → Node y, z; ensure y == \new(Node) && z == \new(Node) && x.getLeft() == y && x.getRight() == z ; B[y] B[z] | ensure x.getLeft() == null && x.getRight() == null;

65. Objection Generation vs. Validation The use of shape types in interface grammars has two purposes –For the objects that are passed as method arguments we need to check that their shape is allowed by the shape type We call this object validation –For the objects that are returned by the component we need to generate an object that is allowed by the shape type We call this object generation

66. Object Generation vs. Validation Object generation and validation tasks are broadly symmetric –The set of nonterminals and productions used for object generation and validation are the same and are dictated by the shape type specification –In object generation semantic actions are used to set the fields of objects to appropriate values dictated by the shape type specification –In object validation these are constraints that are checked using semantic predicates specified as guards Given the semantic elements specified in JML, our interface compile generates code both for object generation and validation

67. Object Generation vs. Validation There is a minor problem with object validation In shape type specifications, the assumption is that there is no aliasing among the objects unless it is explicitly specified This assumption is easy to enforce during object generation since every new statement creates a new object that has nothing else pointing to it In order to enforce the same constraint during object validation we need to make sure that there is no unspecified aliasing –This can be enforced by using a hash-set for storing and propagating all the observed objects

68. Modular Verification of Web Services We applied our modular verification approach based on interface grammars to both client and server side verification of Web services

69. Interface Grammars and Web Services Our approach: 1.A WSDL-to-interface grammar translator automatically generates grammar productions that generate and/or validate XML arguments and return values 2.User adds control flow constraints by modifying the grammar 3.Interface compiler automatically generates a stub for client side verification and a driver for server-side verification

70. Interface Grammars for Web Services

71. Another Case Study: AWS-ECS We tested the Amazon E-Commerce Web Service (AWS- ECS) using our approach AWS-ECS WSDL specification lists 40 operations –that provide differing ways of searching Amazon’s product database We focus on the core operations: –ItemSearch, CartCreate, CartAdd, CartModify, CartGet, CartClear

72. Client-side Verification For client verification we used a demonstration client provided by Amazon This client does not check any constraints such as –You should not try to insert an item to a shopping cart before creating a shopping cart When such requests are sent to AWS-ECS they would return an error message Using our approach we can easily check if the client allows such erroneous requests Falsification time changes with the type of faults we are looking for (data or control errors), changes from 10 to 60 seconds

73. AWS-ECS: Server Verification Our interface compiler automatically generates a driver that sends sequences of requests to AWS-ECS server and checks that the return values conform to the interface specification The driver is a sentence generator –It generates sequences of SOAP requests based on the interface specification We used two algorithms for sentence generation: –A random sentence generation algorithm –Purdom’s algorithm: A directed sentence generation algorithm that guarantees production coverage

74. Directed Sentence Generation Number of sentences generated: 5 Average derivation length: 24 Average number of SOAP requests/responses: 3.8 Verification time: 20 seconds

75. Random Sentence Algorithm Number of sentences generated: 100 Average derivation length: 17.5 Average number of SOAP requests/responses: 3.2

76. Server-side verification We found two errors during server side verification –Errors were discovered within 10 seconds These errors indicate a mismatch between the interface specification and the server implementation It may mean that we misunderstood the description of the Web service interface It may also mean that there is an error in the service implementation

77. Conclusions Modular verification is a necessity Interfaces are crucial for modular verification Interface grammars provide a new specification mechanism for interfaces Interface grammars can be used for automated stub and driver generation leading to modular verification

78. Conclusions Behavioral interfaces can be useful both –From software design perspective by enabling better modular designs –From verification perspective by enabling more efficient modular verification The challenge is to find behavioral interface specification mechanisms that serve both of these goals

79. Related Work: Modular Verification Clarke, Long, McMillan, Compositional Model Checking Henzinger, Qadeer, Rajamani, Assume Guarantee Reasoning in Model Checking Flanagan, Qadeer, Thread-Modular Model Checking Krishnamurthi, Fisler, Modular Verification of Feature Oriented Programs

80. Related Work: Design for Verification Meyer, Design by Contract Flanagan, Leino, et al. ESC Java Mehlitz, Penix, Design for Verification Using Design Patterns Sarna-Starosta, Stirewalt, Dillon, Design for Verification for Synchronization

81. Related Work: Interfaces L. de Alfaro and T. A. Henzinger. Interface automata. O. Tkachuk, M. B. Dwyer, and C. Pasareanu. Automated environment generation for software model checking. T. Ball and S. K. Rajamani. SLAM interface specification language. G. T. Leavens et al.: JML

82. Related: Grammar-based Testing A. G. Duncan, J. S. Hurchinson: Using attributed grammars to test designs and implementations P. M. Maurer: Generating test data with enhanced context free grammars P. M. Maurer: The design and implementation of a grammar-based data generator E. G. Sirer and B. N. Bershad: Using production grammars in software testing