1. Formal Models for Programming and Composing Correct Distributed Systems Ludovic Henrio OASIS Team INRIA – UNS – I3S – CNRS Sophia Antipolis

2. Agenda 2 I.Introduction: Formal Methods II.A Distributed Component Model III.Behavioural Specification IV.Dynamicity V.A Few Hot Topics

3. My objective Help the programmer write correct distributed applications, and run them safely. By designing languages and middlewares By proving their properties By providing tools to support the development and proof of correct programs 3 Programming easily correct concurrent programs is still a challenge Distributed programming is even more difficult, and more and more useful Application domains: cloud computing, service oriented computing … Programming easily correct concurrent programs is still a challenge Distributed programming is even more difficult, and more and more useful Application domains: cloud computing, service oriented computing …

4. What are Formal Methods (here)? Mathematical techniques for developping computer- based systems:  Programs  Languages  Systems What tools?  Pen and paper (PP)  Theorem proving (TP) = proof assistant  Model checking (MC) = check a formula on (an abstraction of) all possible executions  … 4

5. For what? For proving properties / correctness of Languages (ASP TP+PP, algorithmic skeletons PP ) Programs(work on behavioural specification MC ) Algorithms (CAN TP ) Middleware (future update strategies TP, component platform TP, fault-tolerance PP ) …  Increase the confidence people have in the system  Help my colleagues implement correct (and efficient) middlewares  Help the programmer write, compose, and run correct and efficient distributed programs  Increase the confidence people have in the system  Help my colleagues implement correct (and efficient) middlewares  Help the programmer write, compose, and run correct and efficient distributed programs 5

6. My general approach 6 Programming model and definitions Optimizations Implementation Optimizations Verification and tools GenericpropertiesGenericproperties

7. 7 Programming model and definitions Programming model and definitions Optimizations Implementation Optimizations Verification tools GenericpropertiesGenericproperties definitions properties Definitions and Properties are easy to understand check the proofs My general approach

8. 8 Programming model and definitions Programming model and definitions Optimizations Implementation Optimizations Verification tools GenericpropertiesGenericproperties definitions properties Definitions and Properties are easy to understand My general approach

9. Agenda I.Introduction: Formal Methods II.A Distributed Component Model 1. Programming model 2. Formalisation III.Behavioural Specification IV.Dynamicity V.A Few Hot Topics 9

10. What is a component? / Why components? Piece of code (+data) encapsulated with well defined interface Very interesting for reasoning on programs (and for formal methods) because:  components encapsulate isolated code  compositional approach (verification, …)  interaction (only) through interfaces  well identified interaction  easy/safe composition  Reasoning and programming is easier and compositional

11. What are Components? 11 Business code Primitive component Server / input Client / output

12. What are Components? 12 Business code Primitive component Business code Primitive component Composite component  Grid Component Model (GCM) An extension of Fractal for Distributed computing  Grid Component Model (GCM) An extension of Fractal for Distributed computing

13. But what is a Good size for a (primitive) Component? Not a strict requirement, but somehow imposed by the model design According to CCA or SCA, a service (a component contains a provided business function) According to Fractal, a few objects According to GCM, a process 13 GCM: A Grid Extension to Fractal for Autonomous Distributed Components - F. Baude, D. Caromel, C. Dalmasso, M. Danelutto, V. Getov, L. Henrio, C. Pérez - Annals of Telecom. - 2008  In GCM/ProActive, 1 Component (data/code unit) = 1 Active object (1 thread = unit of concurrency) = 1 Location (unit of distribution)  In GCM/ProActive, 1 Component (data/code unit) = 1 Active object (1 thread = unit of concurrency) = 1 Location (unit of distribution)

14. GCM: “Asynchronous” Fractal Components Designed in the CoreGrid Network of Excellence, Add distribution to Fractal components  Many-to-many communications ProActive/GCM implemented in the GridCOMP European project OUR point of view in OASIS (ProActive/GCM):  No shared memory between components  Components evolve asynchronously  Components communicate by request/replies (Futures)

15. A Primitive GCM Component CI.foo(p) CI 15  Primitive components communicate by asynchronous requests on interfaces  Components abstract away distribution and concurrency  In ProActive/GCM a primitive component is an active object  Primitive components communicate by asynchronous requests on interfaces  Components abstract away distribution and concurrency  In ProActive/GCM a primitive component is an active object

16. Futures for Components 16 f=CI.foo(p) ………. g=f+3 Component are independent entities (threads are isolated in a component) + Asynchronous requests with results  Futures are necessary Component are independent entities (threads are isolated in a component) + Asynchronous requests with results  Futures are necessary 1 2 3

17. A Classification of Futures Future = Placeholder for an awaited result Creation of a future  How and when are futures created? l Implicit creation = automatic upon asynchronous invocation (on a client interface) l Explicit creation = there is a future construct Manipulation and access  How to manipulate the futures? l Explicit access = get operation (+ future type) l Implicit (transparent) access = any variable can contain a future 17

18. First-class Futures 18 f=CI.foo(p) ……………… CI.foo(f) Only strict operations are blocking (access to a future) Communicating a future is not a strict operation Only strict operations are blocking (access to a future) Communicating a future is not a strict operation

19. First-class Futures and Hierarchy … … … Without first-class futures, one thread is systematically blocked in the composite component. A lot of blocked threads In GCM/ProActive  systematic deadlock Without first-class futures, one thread is systematically blocked in the composite component. A lot of blocked threads In GCM/ProActive  systematic deadlock return C1.foo(x) 19

20. Collective interfaces One-to-many = multicast Many-to-one = gathercast Distribution and synchronisation/collection policies for invocation and results Business code Primitive component Business code Primitive component Composite component Business code Primitive component Business code Primitive component 20

21. A Distributed Component Model with Futures Primitive components contain the business code Primitive components act as the unit of distribution and concurrency (each thread is isolated in a component) Communication is performed on interfaces and follows component bindings Futures allow communication to be asynchronous requests Futures are transparent can lead to optimisations and are a convenient programming abstraction

22. Agenda I.Introduction: Formal Methods II.A Distributed Component Model 1. Programming model 2. Formalisation III.Behavioural Specification IV.Dynamicity V.A Few Hot Topics 22

23. Back to Formal Methods: a Framework for Reasoning on Components Formalise GCM in a theorem prover (Isabelle/HOL ) Component hierarchical Structure Bindings, etc… Design Choices  Suitable abstraction level  Suitable representation (List / Finite Set, etc …) Basic lemmas on component structure 23 Business code Primitive component Composite component

24. A semantics of Primitive Components Primitive components are defined by interfaces plus an internal behaviour, they can:  emit requests  serve requests  send results  receive results (at any time)  do internal actions some rules define a correct behaviour, e.g. one can only send result for a served request 24

25. Communication inside Composites 25 Composites only delegate calls between components Use the bindings to know where to transmit requests Component system behaviour is expressed as a small step semantics, and specified on paper and in Isabelle/HOL A Framework for Reasoning on Component Composition Ludovic Henrio, Florian Kammüller, and Muhammad Uzair Khan - FMCO 2009, Springer

26. Future Update Strategies (Muhammad Khan) How to bring future values to components that need them? A “naive” approach: Any component can receive a value for a future reference it holds. More operational is the lazy approach: 26 require future value

27. Eager home-based future update avoids to store future values indefinitely Relies on future registration 27 register future Results sent as soon as available Every component with future reference is registered Un-necessary transfers Garbage collection of computed results possible Formalised in Isabelle Results sent as soon as available Every component with future reference is registered Un-necessary transfers Garbage collection of computed results possible Formalised in Isabelle First Class Futures: Specification and Implementation of Update Strategies L. Henrio, Muhammad U. Khan, Nadia Ranaldo, Eugenio Zimeo CoreGRID@Europar 2010.

28. Eager forward-based strategy A strategy avoiding to store future values indefinitely Future updates follow the same path as future flow Each component remembers only the components to which it forwarded the future 28 Results sent as soon as available No additional message Future updates form a chain  intermediate components Easy to garbage collect computed results Results sent as soon as available No additional message Future updates form a chain  intermediate components Easy to garbage collect computed results

29. Composite Component Semantics – CommChild 29

30. Properties on Future updates Future updates remove all references to a given future All Future references are registered during reduction register future Garbage collectable Complete registration Ludovic Henrio and Muhammad Uzair Khan “Asynchronous Components with Futures: Semantics and Proofs in Isabelle/HOL” - FESCA 2010 - ENTCS 30

31. A refined GCM model in Isabelle/HOL More precise than GCM, give a semantics to the model:  asynchronous communications: future / requests  request queues  no shared memory between components  notion of request service More abstract than ProActive/GCM  can be multithreaded  no active object, not particularly object-oriented 31 Similarities with: SCA and Fractal (structure), Creol (futures) A guide for implementing and proving properties of component middlewares

32. Agenda I.Introduction: Formal Methods II.A Distributed Component Model III.Behavioural Specification IV.Dynamicity V.A Few Hot Topics 32

33. Motivating example: What Can Create Deadlocks in ProActive/GCM? A race condition: Detecting deadlocks can be difficult  behavioural specification and verification techniques

34. How to ensure the correct behaviour of a given program? Theorem proving too complicated for the ProActive programmer Our approach: behavioural specification 34 Service methods pNets: Behavioural Models for Distributed Fractal Components Antonio Cansado, Ludovic Henrio, and Eric Madelaine - Annals of Telecommunications - 2008  Trust the implementation step  Or static analysis  Generate correct (skeletons of) components (+static and/or runtime checks)  Trust the implementation step  Or static analysis  Generate correct (skeletons of) components (+static and/or runtime checks)

35. Use-case: Fault-tolerant storage 35 1 multicast interface sending write/read/commit requests to all slaves. the slaves reply asynchronously, the master only needs enough coherent answers to terminate Verifying Safety of Fault-Tolerant Distributed Components Rabéa Ameur-Boulifa, Raluca Halalai, Ludovic Henrio, and Eric Madelaine - FACS 2011

36. Full picture: a pNet !Q_Write(b) ?Q_Write(x) Support for parameterised families Synchronisation vectors 36

37. Basic pNets: parameterized LTS 37 Labelled transition systems, with: Value passing Local variables Guards…. Can be written as a UML diagram Eric MADELAINE

38. Properties proved Reachability: 1- The Read service can terminate  fid:nat among {0...2}. ∃ b:bool. true 2- Is the BFT hypothesis respected by the model ? true Inevitability: After receiving a Q_Write(f,x) request, it is (fairly) inevitable that the Write services terminates with a R_Write(f) answer, or an Error is raised. Functional correctness: After receiving a ?Q_Write(f1,x), and before the next ?Q_Write, a ?Q_Read requests raises a !R_Read(y) response, with y=x (written in mu-calculus or Model Checking Language (MCL), Mateescu et al, FM’08) 38 Prove  generic properties like absence of deadlock  or properties specific to the application logic Prove  generic properties like absence of deadlock  or properties specific to the application logic

39. Tool Architecture Currently: production of the CADP input formats only partially (~50%) available. Goal: full automatisation 39 Vercors Editors: ADL ++ Interfaces Behaviors Properties Behav. Sem. *.fractal pNets N2F LNT exp svl++ CADP: Caesar.open Distributor Exp.open Evaluator MCL Abs Diagnostic visualisation

40. Recent advances in behavioural specification for GCM Scaling-up : gained orders of magnitude by a combination of:  data abstraction,  compositional and contextual minimization,  distributed state-space generation. Biggest intermediate model: 5M states / estimated brute force: ~ 10 32 Specified formally the whole generation process pNets: 2004-2008 CoCoME: hierarchical & synchronous FACS’08: 1 st class futures WCSI’10: group communication FACS’11: GCM & multicast interfaces Application to BFT Ongoing work Reconfiguration... 40

41. Agenda 41

42. Adaptation in the GCM Functional adaptation: adapt the architecture + behaviour of the application to new requirements/objectives/environment Non-functional adaptation(with Paul Naoumenko): adapt the architecture of the container+middleware to changing environment/NF requirements (QoS …) Additional support for reconfiguration (with Marcela Rivera): A stopping algorithm for GCM components A Scripting language for reconfiguring distributed components 42 A Component Platform for Experimenting with Autonomic Composition Françoise Baude, Ludovic Henrio, and Paul Naoumenko. Autonomics 2007. Both functional and non-functional adaptation are expressed as reconfigurations Language support for distributed reconfiguration: GCM-script A platform for designing and running autonomic components Both functional and non-functional adaptation are expressed as reconfigurations Language support for distributed reconfiguration: GCM-script A platform for designing and running autonomic components

43. Triggering Distributed Reconfigurations in the Scripting Language Fscript is a language to reconfigure Fractal components A primitive for the distributed script interpretation + other minor extensions 43 Reconfiguration interface action(arguments) Interpreter target_component remote_call(target_component, action_name, parameters,...);

44. definition action1... remote_call(C2,’action2’) remote_call(C11,’action11’) remote_call(C12,’action12’)... definition action12... remote_call(C121,’action121’)... Interpreter Membrane C2C2 RC 1 RC 12 RC 121 RC 2 RC 11 Reconfiguration scripts C11 Example 44 Reconfiguration interface

45. definition action1... remote_call(C2,’action2’) remote_call(C11,’action11’) remote_call(C12,’action12’)... definition action12... remote_call(C121,’action121’)... Interpreter RC 1 RC 12 RC 121 RC 2 RC 11 C121 C11 C12 Example 45 action1( ) Reconfiguration interface Membrane C2C2 Reconfiguration scripts

46. definition action1... remote_call(C2,’action2’) remote_call(C11,’action11’) remote_call(C12,’action12’)... definition action12... remote_call(C121,’action121’)... Interpreter RC 1 RC 12 RC 121 RC 2 RC 11 C121 C11 C1C1 C12 Example 46 action1( ) action11( ) action2( ) action12( ) Reconfiguration interface Membrane C2C2 Reconfiguration scripts The reconfiguration script is executed in a parallel and distributed manner  This language is the basis of the GCM-script language The reconfiguration script is executed in a parallel and distributed manner  This language is the basis of the GCM-script language A Reconfiguration Framework for Distributed Components Marcela Rivera, Ludovic Henrio and Boutheina Bannour. SINTER Workshop 2009

47. Formalising Reconfiguration (preliminary) In our Isabelle/HOL model component structure known at runtime  Two reconfiguration primitives formalised (remove and replace)  Illustrates the flexibility of the approach  Basic proofs In our pNets model  Old results (Tomas): start/stop/bind for Fractal components  Concerning GCM: formalisation and experiments in progress 47

48. Agenda 48

49. Correct component reconfiguration Towards safety and autonomicity Verify reconfiguration procedures Safe adaptation procedures as a solid ground for autonomic applications Some directions:  Parameterized topologies: ADL[N] ; behavioural specification (pNets) ; reconfiguration primitives  Use of theorem proving + prove equivalence between the Isabelle GCM model and the behavioural specification  prove and use generic properties of reconfiguration procedures 49

50. Correct component reconfiguration Towards safety and autonomicity Verify reconfiguration procedures Safe adaptation procedures as a solid ground for autonomic applications Some directions:  Parameterized topologies: ADL[N] ; behavioural specification (pNets) ; reconfiguration primitives  Use of theorem proving + prove equivalence between the Isabelle GCM model and the behavioural specification  prove and use generic properties of reconfiguration procedures 50 [N]

51. Multi-active objects Mono-threaded active objects (primitive components) have limitations:  Deadlocks  Inadapted to multicore architecture (no shared memory) Solutions in the active-object community:  Jcobox: « cooperative multithreading », only one thread active at a time (release points)  JAC: Java annotations for Concurrency – real multithreading, not distributed Our proposal: a programming model that mixes local parallelism and distribution based on high-level programming constructs 51 Henrio, Ludovic, Fabrice Huet, Zsolt István, and Zsolt Istv. 2013. “Multi-threaded Active Objects.” in COORDINATION 2013.

52. Primitive GCM Component 52 foo() { … CI.foo(p) }

53. Primitive Multi-Active GCM Component 53 add() { … … } monitor() {… … } add() { CI.foo(p) } Provided add, add and monitor are compatible join() Note: monitor is compatible with join

54. Proposal – Annotation Example Groups (Collection of related methods) Rules (Compatibility relationships between groups) Memberships (To which group each method belongs) 54

55. A new programming model Multi-active objects feature:  Active object model l Easy to program l Support for distribution  Local concurrency and efficiency on multi-cores l Transparent multi-threading l Safe parallel execution  Possibility to write non-blocking re-entrant code  Simple annotations

56. Implemented multi-active objects above ProActive Dynamic compatibility rules and thread limitation Case studies/benchmarks: NAS, CAN Specified the semantics of Multi-ASP Proved that parallelism is maximised:  Two requests served by two different threads are compatible  Each request is incompatible with another request served by the same thread (that precede it) Next steps:  Static guarantees/verification of annotations  Use the new model (new use-cases and applications)  Prove stronger properties, mechanised formalisaiton Results and Status 56 Mechanical Support for Efficient Dissemination on the CAN Overlay Network - Francesco Bongiovanni, and Ludovic Henrio - CFSE 2011

57. CAN dissemination algorithm (with Francesco Bongiovani, Justine Rochas) CANs are P2P networks organised in a cartesian space of N-dimensions (used for RDF data-storage) Objective: disseminate efficiently information (no duplicate) Proved the existence of an efficient broadcast in Isabelle Designed an efficient algorithm Current steps: Prove the designed algorithm is efficient (done on paper) Conduct large-scale experiments (~ done) Study churns 57 K F A C EG H I B J DD E C K G H J B I F Francesco Bongiovanni, and Ludovic Henrio. “A Mechanized Model for CAN Protocols.” in FASE 2013.

58. A few things I could not fit in this talk ASP: Active object calculus (D. Caromel and B. Serpette) ASP fun a functional variant formalised in Isabelle (F. Kammueller) Fault tolerance (C. Delbé) and garbage collection (G. Chazarain) for active objects Behavioural skeletons (typing + exceptions) (M. Leyton) JDC: a language for specifying behaviour of components (A. Cansado) And of course, a lot of inference rules and greek letters … 58

59. MERCI !!! http://www-sop.inria.fr/oasis/personnel/Ludovic.Henrio/ Ludovic.henrio@cnrs.fr

60. GCM Formalisation in Isabelle > 4000 lines, mostly proofs (do not read the proofs!) Almost 300 lemmas and theorems, ~ 500 lines definitions: component model and semantics 1800 lines for future update specific properties A few tricky design choices (lists vs sets, how to store names, …) A couple of axioms remaining (reduction correctness) short-term goal: correctness of various strategies Longer term: reconfiguration and execution 60

61. Non-functional Component Structure Non-functional aspects as a composition of components (inside a membrane)  A component structure for the membrane  New kind of interfaces and bindings  An API for reconfiguring the membrane  Non-functional code is l components or objects l distributed or not Towards autonomic components 61 A Component Platform for Experimenting with Autonomic Composition Françoise Baude, Ludovic Henrio, and Paul Naoumenko. Autonomics 2007.

62. Can we do more dynamicity? YES: ProActive/ASP! Active objects:  newActive  Reference to active objects passed as request parameters or replies What formal models/properties  ASP(old): confluence and determinacy (POPL 2004, I&C) l future updates can occur at any time, execution uniquely characterised by the order of request senders  Fault tolerance (Europar’05, PPoPP’07)  ASP fun : in Isabelle/HOL, typed, proved progress, studied binder representations (SCP 2011)  Behavioural specification: possible but much more restricted (need strong static analysis) 62  

63. Support Constructs and Lemmas Operators to manipulate components inside hierarchies  E.g., Traverse hierarchy: find/replace, (recursive) set of subcomponents: cpSet To manipulate states  Retrieve referenced futures, all computed results, etc Properties on component structure  “Well-defined” and “correct” Components Support lemmas, e.g. 63

64. Component re-configuration Transformation of component structure or content at runtime  Two reconfiguration primitives formalised (remove and replace)  Illustrates the flexibility of the approach Basic properties on reconfiguration primitives Ludovic Henrio, Florian Kammüller, and Muhammad Uzair Khan A Framework for Reasoning on Component Composition - FMCO 2009, Springer (2010) 64

65. GCM-like Component with Eager-home 65