1. Formal Models for Programming and Composing Correct Distributed Systems Jury : Gordon BlairFabienne BoyerEric Madelaine Pascal PoizatMichel RiveillDavide Sangiorgi Alan Schmitt Ludovic Henrio Habilitation thesis (HDR) OASIS Team INRIA – UNS – I3S – CNRS Sophia Antipolis

2. Agenda 2 I.Introduction: Formal Methods II.A Distributed Component Model III.Behavioural Specification IV.Dynamicity V.A Few Hot Topics

3. My objective Help the programmer write correct distributed applications, and run them safely. By designing languages and middlewares By proving their properties By providing tools to support the development and proof of correct programs 3 Programming easily correct concurrent programs is still a challenge - ERC Bertrand Meyer: “devise a powerful, efficient and secure concurrent solution to address the needs of the programming community” Distributed programming is even more difficult, and more and more useful (cloud computing, service oriented computing …) Programming easily correct concurrent programs is still a challenge - ERC Bertrand Meyer: “devise a powerful, efficient and secure concurrent solution to address the needs of the programming community” Distributed programming is even more difficult, and more and more useful (cloud computing, service oriented computing …)

4. What are Formal Methods (here)? Mathematical techniques for developping computer- based systems:  Programs  Languages  Systems What tools?  Pen and paper (PP)  Theorem proving (TP) = proof assistant  Model checking (MC) = check a formula on (an abstraction of) all possible executions  … 4

5. For what? For proving properties / correctness of Languages (ASP TP+PP, algorithmic skeletons PP ) Programs(work on behavioural specification MC ) Algorithms (CAN TP ) Middleware (future update strategies TP, component platform TP, fault-tolerance PP ) …  Increase the confidence people have in the system  Help my colleagues implement correct (and efficient) middlewares  Help the programmer write, compose, and run correct and efficient distributed programs  Increase the confidence people have in the system  Help my colleagues implement correct (and efficient) middlewares  Help the programmer write, compose, and run correct and efficient distributed programs 5

6. My general approach 6 Programming model and definitions Optimizations Implementation Optimizations Verification and tools GenericpropertiesGenericproperties

7. 7 Programming model and definitions Programming model and definitions Optimizations Implementation Optimizations Verification tools GenericpropertiesGenericproperties definitions properties Definitions and Properties are easy to understand check the proofs My general approach

8. 8 Programming model and definitions Programming model and definitions Optimizations Implementation Optimizations Verification tools GenericpropertiesGenericproperties definitions properties Definitions and Properties are easy to understand My general approach

9. Agenda 9

10. What are Components? 10 Business code Primitive component Server / input Client / output

11. What are Components? 11 Business code Primitive component Business code Primitive component Composite component  Grid Component Model (GCM) An extension of Fractal for Distributed computing  Grid Component Model (GCM) An extension of Fractal for Distributed computing

12. But what is a Good size for a (primitive) Component? Not a strict requirement, but somehow imposed by the model design According to CCA or SCA, a service (a component contains a provided business function) According to Fractal, a few objects According to GCM, a process 12 GCM: A Grid Extension to Fractal for Autonomous Distributed Components - F. Baude, D. Caromel, C. Dalmasso, M. Danelutto, V. Getov, L. Henrio, C. Pérez - Annals of Telecom. - 2008  In GCM/ProActive, 1 Component (data/code unit) = 1 Active object (1 thread = unit of concurrency) = 1 Location (unit of distribution)  In GCM/ProActive, 1 Component (data/code unit) = 1 Active object (1 thread = unit of concurrency) = 1 Location (unit of distribution)

13. A Primitive GCM Component 13 CI.foo(p) In ProActive/GCM a primitive component is an active object  Primitive components communicating by asynchronous requests on interfaces  Components abstract away distribution and concurrency In ProActive/GCM a primitive component is an active object  Primitive components communicating by asynchronous requests on interfaces  Components abstract away distribution and concurrency CI

14. Futures for Components 14 f=CI.foo(p) ………. g=f+3 Component are independent entities (threads are isolated in a component) + Asynchronous requests with results  Futures are necessary Component are independent entities (threads are isolated in a component) + Asynchronous requests with results  Futures are necessary 1 2 3

15. First-class Futures 15 f=CI.foo(p) ……………… CI.foo(f) Only strict operations are blocking (access to a future) Communicating a future is not a strict operation In ProActive and ASP, futures are created and accessed implicitly (no explicit type “future”) IN contrast with Creol, Jcobox,... Only strict operations are blocking (access to a future) Communicating a future is not a strict operation In ProActive and ASP, futures are created and accessed implicitly (no explicit type “future”) IN contrast with Creol, Jcobox,...

16. First-class Futures and Hierarchy … … … Without first-class futures, one thread is systematically blocked in the composite component. A lot of blocked threads In GCM/ProActive  systematic deadlock Without first-class futures, one thread is systematically blocked in the composite component. A lot of blocked threads In GCM/ProActive  systematic deadlock return C1.foo(x) 16

17. Back to Formal Methods: a Framework for Reasoning on Components Formalise GCM in a theorem prover (Isabelle/HOL ) Component hierarchical Structure Bindings, etc… Design Choices  Suitable abstraction level  Suitable representation (List / Finite Set, etc …) Basic lemmas on component structure 17 Business code Primitive component Composite component

18. A semantics of Primitive Components Primitive components are defined by interfaces plus an internal behaviour, they can:  emit requests  serve requests  send results  receive results (at any time)  do internal actions some rules define a correct behaviour, e.g. one can only send result for a served request 18

19. Communication inside Composites 19 Composites only delegate calls between components Use the bindings to know where to transmit requests Component system behaviour is expressed as a small step semantics, and specified on paper and in Isabelle/HOL A Framework for Reasoning on Component Composition Ludovic Henrio, Florian Kammüller, and Muhammad Uzair Khan - FMCO 2009, Springer

20. Future Update Strategies (Muhammad Khan) How to bring future values to components that need them? A “naive” approach: Any component can receive a value for a future reference it holds. More operational is the lazy approach: 20 require future value

21. Eager home-based future update avoids to store future values indefinitely Relies on future registration 21 register future Results sent as soon as available Every component with future reference is registered Un-necessary transfers Garbage collection of computed results possible Formalised in Isabelle Results sent as soon as available Every component with future reference is registered Un-necessary transfers Garbage collection of computed results possible Formalised in Isabelle First Class Futures: Specification and Implementation of Update Strategies L. Henrio, Muhammad U. Khan, Nadia Ranaldo, Eugenio Zimeo CoreGRID@Europar 2010.

22. Properties on Future updates Future updates remove all references to a given future All Future references are registered during reduction register future Garbage collectable Complete registration Ludovic Henrio and Muhammad Uzair Khan “Asynchronous Components with Futures: Semantics and Proofs in Isabelle/HOL” - FESCA 2010 - ENTCS 22

23. A refined GCM model in Isabelle/HOL More precise than GCM, give a semantics to the model:  asynchronous communications: future / requests  request queues  no shared memory between components  notion of request service More abstract than ProActive/GCM  can be multithreaded  no active object, not particularly object-oriented 23 Similarities with: SCA and Fractal (structure), Creol (futures) A guide for implementing and proving properties of component middlewares

24. Agenda 24

25. How to ensure the correct behaviour of a given program? Theorem proving too complicated for the ProActive programmer Our approach: behavioural specification 25 Service methods pNets: Behavioural Models for Distributed Fractal Components Antonio Cansado, Ludovic Henrio, and Eric Madelaine - Annals of Telecommunications - 2008  Trust the implementation step  Or static analysis  Generate correct (skeletons of) components (+static and/or runtime checks)  Trust the implementation step  Or static analysis  Generate correct (skeletons of) components (+static and/or runtime checks)

26. Use-case: Fault-tolerant storage 26 1 composite component with 2 external services Read/Write. The service requests are delegated to the Master. 1 multicast interface sending write/read/commit requests to all slaves. the slaves reply asynchronously, the master only needs enough coherent answers to terminate Verifying Safety of Fault-Tolerant Distributed Components Rabéa Ameur-Boulifa, Raluca Halalai, Ludovic Henrio, and Eric Madelaine - FACS 2011

27. Full picture: a pNet 27 !Q_Write(b) ?Q_Write(x)

28. Basic pNets: parameterized LTS 28 Labelled transition systems, with: Value passing Local variables Guards…. Can be written as a UML diagram Eric MADELAINE

29. Properties proved Reachability(*): 1- The Read service can terminate  fid:nat among {0...2}. ∃ b:bool. true 2- Is the BFT hypothesis respected by the model ? true Termination: After receiving a Q_Write(f,x) request, it is (fairly) inevitable that the Write services terminates with a R_Write(f) answer, or an Error is raised. Functional correctness: After receiving a ?Q_Write(f1,x), and before the next ?Q_Write, a ?Q_Read requests raises a !R_Read(y) response, with y=x (*) Model Checking Language (MCL), Mateescu et al, FM’08 29 Prove  generic properties like absence of deadlock  or properties specific to the application logic Prove  generic properties like absence of deadlock  or properties specific to the application logic

30. Recent advances in behavioural specification for GCM Scaling-up : gained orders of magnitude by a combination of:  data abstraction,  compositional and contextual minimization,  distributed state-space generation. Specified formally the whole generation process (in progress) 30 pNets: 2004-2008 CoCoME: hier. & synch. FACS’08: 1 st class futures WCSI’10: group communication FACS’11: GCM & multicast interfaces Application to BFT More to come ? Reconfiguration...

31. Agenda 31

32. Adaptation in the GCM Functional adaptation: adapt the architecture + behaviour of the application to new requirements/objectives Non-functional adaptation(Paul Naoumenko): adapt the architecture of the container+middleware to changing environment/NF requirements (QoS …) Additional support for reconfiguration (Marcela Rivera): A stopping algorithm for GCM components A Scripting language for reconfiguring distributed components 32 A Component Platform for Experimenting with Autonomic Composition Françoise Baude, Ludovic Henrio, and Paul Naoumenko. Autonomics 2007. Both functional and non-functional adaptation are expressed as reconfigurations Language support for distributed reconfiguration: GCM-script A platform for designing and running autonomic components Both functional and non-functional adaptation are expressed as reconfigurations Language support for distributed reconfiguration: GCM-script A platform for designing and running autonomic components

33. Formalising Reconfiguration (preliminary) In our Isabelle/HOL model component structure known at runtime  Two reconfiguration primitives formalised (remove and replace)  Illustrates the flexibility of the approach  Basic proofs In our pNets model  Old results (Tomas): start/stop/bind for Fractal components  Concerning GCM: formalisation and experiments in progress 33

34. Agenda 34

35. Correct component reconfiguration Towards safety and autonomicity Verify reconfiguration procedures (Nuno Gaspar) Safe adaptation procedures as a solid ground for autonomic applications Some directions:  Parameterized topologies: ADL[N] ; behavioural specification (pNets) ; reconfiguration primitives (Alexandra Savu)  Use of theorem proving + prove equivalence between the Isabelle GCM model and the behavioural specification  prove and use generic properties of reconfiguration procedures 35

36. Correct component reconfiguration Towards safety and autonomicity Verify reconfiguration procedures (Nuno Gaspar) Safe adaptation procedures as a solid ground for autonomic applications Some directions:  Parameterized topologies: ADL[N] ; behavioural specification (pNets) ; reconfiguration primitives (Alexandra Savu)  Use of theorem proving + prove equivalence between the Isabelle GCM model and the behavioural specification  prove and use generic properties of reconfiguration procedures 36 [N]

37. Multi-active objects Mono-threaded active objects have limitations:  Deadlocks  Inadapted to multicore architecture (no shared memory) Solutions in the active-object community:  Jcobox: « cooperative multithreading », only one thread active at a time (release points)  JAC: Java annotations for Concurrency – real multithreading, not distributed Our proposal, a programming model that mixes local parallelism and distribution with high-level programming constructs 37 Adapting Active Objects to Multicore Architectures - Ludovic Henrio, Fabrice Huet, Zsolt István, and Gheorghen Sebestyén - ISPDC 2011

38. Primitive GCM Component 38 foo() { … CI.foo(p) }

39. Primitive Multi-Active GCM Component 39 add() { … … } monitor() {… … } add() { CI.foo(p) } Provided add, add and monitor are compatible join()

40. Proposal – Annotation Example Groups (Collection of related methods) Rules (Compatibility relationships between groups) Memberships (To which group each method belongs) 40

41. Implemented multi-active objects above ProActive Added dynamic compatibility rules Used on case studies (NAS, CAN) Specified the semantics of Multi-ASP Proved first properties, i.e., the semantics guarantees that two requests executed in parallel are compatible Next steps:  Publish the model  Formalisation in Isabelle/HOL  Use the new model  Prove stronger properties (confluence?) Results / Status 41

42. CAN dissemination algorithm (Francesco Bongiovani) CANs are P2P networks organised in a cartesian space of N-dimensions (used for RDF data-storage) Objective: disseminate efficiently information (no duplicate) Designed an efficient algorithm (tested) Proved the existence of an efficient broadcast in Isabelle Next steps: Prove the designed algorithm is efficient Conduct large-scale experiments Study churns 42 K F A C EG H I B J DD E C K G H J B I F Mechanical Support for Efficient Dissemination on the CAN Overlay Network - Francesco Bongiovanni, and Ludovic Henrio - CFSE 2011

43. What I could not fit in this talk ASP: Active object calculus ASP fun a functional variant formalised in Isabelle (typing) Fault tolerance (Christian Delbé) and garbage collection (Guillame Chazarain) for active objects Work on behavioural skeletons (typing and exceptions) (Mario Leyton) JDC: a language for specifying behaviour of components (Antonio Cansado) Details on the behavioural specification of components and active objects (first-class futures, …) And of course, a lot of inference rules and greek letters … 43

44. THANK YOU ! 44 Eric Madelaine Antonio Cansado Marcela Rivera Paul Naoumenko Muhammad Khan Boutheina Bannour Florian Kammueller Françoise Baude CoreGrid partners Denis Caromel Bernard Serpette Special thanks to the people I worked with, mainly: Mario Leyton Christian Delbé Francesco Bongiovani Fabrice Huet Zsolt Istvan Rabéa Boulifa And others !!! Guillaume Chazarain Tomas Barros

45. 45

46. Primitive Components in Isabelle Primitive components are defined by interfaces plus an internal behaviour, they can:  emit requests  serve requests  send results  receive results (at any time)  do internal actions some rules define a correct behaviour, e.g. one can only send result for a served request We define the behaviour of the whole components as small-step operational semantics 46 A Framework for Reasoning on Component Composition Ludovic Henrio, Florian Kammüller, and Muhammad Uzair Khan - FMCO 2009, Springer

47. Non-functional Component Structure Non-functional aspects as a composition of components (inside a membrane)  A component structure for the membrane  New kind of interfaces and bindings  An API for reconfiguring the membrane  Non-functional code is l components or objects l distributed or not Towards autonomic components 47 A Component Platform for Experimenting with Autonomic Composition Françoise Baude, Ludovic Henrio, and Paul Naoumenko. Autonomics 2007.

48. Triggering Distributed Reconfigurations in the Scripting Language Fscript is a language to reconfigure Fractal components A primitive for the distributed script interpretation + other minor extensions 48 Reconfiguration interface action(arguments) Interpreter target_component remote_call(target_component, action_name, parameters,...);

49. definition action1... remote_call(C2,’action2’) remote_call(C11,’action11’) remote_call(C12,’action12’)... definition action12... remote_call(C121,’action121’)... Interpreter Membrane C2C2 RC 1 RC 12 RC 121 RC 2 RC 11 Reconfiguration scripts C11 Example 49 Reconfiguration interface

50. definition action1... remote_call(C2,’action2’) remote_call(C11,’action11’) remote_call(C12,’action12’)... definition action12... remote_call(C121,’action121’)... Interpreter RC 1 RC 12 RC 121 RC 2 RC 11 C121 C11 C12 Example 50 action1( ) Reconfiguration interface Membrane C2C2 Reconfiguration scripts

51. definition action1... remote_call(C2,’action2’) remote_call(C11,’action11’) remote_call(C12,’action12’)... definition action12... remote_call(C121,’action121’)... Interpreter RC 1 RC 12 RC 121 RC 2 RC 11 C121 C11 C1C1 C12 Example 51 action1( ) action11( ) action2( ) action12( ) Reconfiguration interface Membrane C2C2 Reconfiguration scripts The reconfiguration script is executed in a parallel and distributed manner  This language is the basis of the GCM-script language The reconfiguration script is executed in a parallel and distributed manner  This language is the basis of the GCM-script language A Reconfiguration Framework for Distributed Components Marcela Rivera, Ludovic Henrio and Boutheina Bannour. SINTER Workshop 2009

52. Can we do more dynamicity? YES: ProActive/ASP! Active objects:  newActive  Reference to active objects passed as request parameters or replies What formal models/properties  ASP(old): confluence and determinacy (POPL 2004, I&C) l future updates can occur at any time, execution uniquely characterised by the order of request senders  Fault tolerance (Europar’05, PPoPP’07)  ASP fun : in Isabelle/HOL, typed, proved progress, studied binder representations (SCP 2011)  Behavioural specification: possible but much more restricted (need strong static analysis) 52  

53. GCM Formalisation in Isabelle > 4000 lines, mostly proofs (do not read the proofs!) Almost 300 lemmas and theorems, ~ 500 lines definitions: component model and semantics 1800 lines for future update specific properties A few tricky design choices (lists vs sets, how to store names, …) A couple of axioms remaining (reduction correctness) short-term goal: correctness of various strategies Longer term: reconfiguration and execution 53

54. Support Constructs and Lemmas Operators to manipulate components inside hierarchies  E.g., Traverse hierarchy: find/replace, (recursive) set of subcomponents: cpSet To manipulate states  Retrieve referenced futures, all computed results, etc Properties on component structure  “Well-defined” and “correct” Components Support lemmas, e.g. 54

55. Component re-configuration Transformation of component structure or content at runtime  Two reconfiguration primitives formalised (remove and replace)  Illustrates the flexibility of the approach Basic properties on reconfiguration primitives Ludovic Henrio, Florian Kammüller, and Muhammad Uzair Khan A Framework for Reasoning on Component Composition - FMCO 2009, Springer (2010) 55

56. Eager forward-based strategy A strategy avoiding to store future values indefinitely Future updates follow the same path as future flow Each component remembers only the components to which it forwarded the future 56 Results sent as soon as available No additional message Future updates form a chain  intermediate components Easy to garbage collect computed results

57. GCM-like Component with Eager-home 57

58. Composite Component Semantics – CommChild 58

59. What is a Future? Future = Placeholder for an awaited result Creation of a future  How and when are futures created? l Implicit creation = automatic upon asynchronous invocation (on a remote object) l Explicit creation = there is a future construct Manipulation and access  How to manipulate the futures? l Explicit access = get operation (+ future type) l Implicit (transparent) access = any variable can contain a future 59