1. Eric MADELAINE ---- OASIS1 E. Madelaine Oasis team INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis ECNU Dept of Software Engineering October 2010 Specification and Verification for distributed applications running on heterogeneous infrastructures

2. Eric MADELAINE ---- OASIS2 Motivations … Specification and Verification for distributed applications Distributed Systems Components Services Adaptation Message passing Clouds Grids Self Healing Reconfiguration Self Optimizing Load Balancing Interfaces MultiCores

3. Eric MADELAINE ---- OASIS3 Motivations… (2) Services Adaptation Message passing Clouds Grids Self Healing Reconfiguration Self Optimizing Load Balancing Components Interfaces MultiCores Distributed Systems - Hard to program ? - Correct Behaviour ? - Correct Assembly ?

4. Eric MADELAINE ---- OASIS4 Do we need formal methods for developing component-based software ? Safe COTS-based development => Behaviour Specifications A B C ??? Safe management for complex systems (e.g. replacement at runtime) C2 A B C

5. Eric MADELAINE ---- OASIS5 Is it more difficult for distributed, asynchronous, evolving components ? Yes ! Asynchrony creates race-conditions, dead-locks, etc. Dynamicity creates new challenges: Correct (DYNAMIC) Adaptation, quiescent states, …

6. Eric MADELAINE ---- OASIS6 Motivations … (3) Heterogeneous Resources for Distributed Applications Windows CCS Cluster Storage Server GPUs Cloud Cluster 47+ linux nodes 2-proc/4-core PacaGrid cluster P2P LAN network Mobile terminal s

7. Eric MADELAINE ---- OASIS7 Heterogeneous Resources for Distributed Applications Windows CCS Cluster Storage Server GPUs Cloud Cluster 47+ linux nodes 2-proc/4-core PacaGrid cluster P2P LAN network Mobile terminal s -Heterogeneity: OS/processor/architecture/communications -High performance / Dynamicity / Mobility / Safety / Security… =  ProActive = seamless programming, deployment, scheduling, and execution middelware, strong semantic guaranties The Challenge: - Provide optimisations that depends on the underlying platform, and may evolve dynamically, while keeping simplicity and correctness

8. Eric MADELAINE ---- OASIS8 Motivation Building safe applications Heterogeneous Infrastructures: Multi-Cores, P2P, Grids, Clouds Context Oasis team,,MCorePhP collaborative project Active Objects, Distributed Components Safe Distributed Components Behavioural Semantics Model generation Checking Properties Specification and Verification Tools, Case Study VerCors platform Case-Study Conclusion & Perspectives Agenda

9. Eric MADELAINE ---- OASIS9 Challenges :  Guarantee safety and security for software systems  Develop and master future infrastructures of networks and services ASP Calculus Properties Model- Checking ProActive Components The OASIS team: Propose fundamental principles, techniques and tools for design, development, analysis, and verification of reliable distributed systems PLATFORMSPLATFORMS

10. Eric MADELAINE ---- OASIS10 ANR (french ministry of research) International “Blanc” project. Partners: University of Tsinghua, Beijing (Pr. Yongwei Wu) INRIA Oasis team (Pr. Denis Caromel, Dr. Eric Madelaine) Research Tasks: Programming Model for Multi-Core Infrastructure with ChinaGrid and CGSP Application and User Case in Bioinformatics MCorePhP: A collaborative project building the basis for safe programming of heterogeneous applications

11. Eric MADELAINE ---- OASIS11 Task 1: Programming Model for Multi-Core 1.1 New Basic Programming Model for Multi-Core Extensions of the Active Object programming model: - Sharing memory (efficiently) between activities - Multi-active (multi-threaded) activities 1.2 Legacy Support and Integration 1.3 Safe Code Generation: From model-level specification and analysis of properties, to “correct by construction” executable code. This presentation 1.4 Monitoring MCorePhP:

12. Eric MADELAINE ---- OASIS12 Asynchronous and Deterministic Objects [Denis Caromel – Ludovic Henrio] ASP (Asynchronous Sequential Processes) = Distributed Active Objects Asynchronous method calls Futures and Wait-by-necessity  Determinism/Confluence properties è Programming abstractions è Formal Basis for Verification

13. Eric MADELAINE ---- OASIS13 Active Objects (very short…) -Runnable (mono-threaded) objects -Communicating by remote method call -Asynchronous computation -Request queues (user-definable policy) -No shared memory -Futures Client obj. A Server obj. B

14. Eric MADELAINE ---- OASIS14 ProActive Parallel Suite Public domain library Object Web Consortium Spin-off company 2007 :

15. Eric MADELAINE ---- OASIS15 Fractal hierarchical model : Attribute Controller Binding Controller Lifecycle Controller Content Controller Content Controller / membrane composites encapsulate primitives, which encapsulates code Provided/Required Interfaces Hierarchy Separation of concern: functional / non-functional ADL Extensible

16. Eric MADELAINE ---- OASIS16 Grid Component Model (GCM): Grid aware extension to Fractal Targetting GRIDS requires to handle: Scalability => hierarchy, parallelism Volatility, heterogeneity => adaptation, dynamicity, autonomicity… Collective interfaces Multicast, gathercast, gather-multicast, MxN parallel communications Opportunity to use GCM for parallel computing Component non-functional concerns (membrane) as a Fractal system Controllers as objects or Fractal/GCM components Fractal extension for properly exposing the non-functional part, including non- functional client interfaces Non-functional ADL and associated APIs Opportunity to use GCM for autonomic computing

17. Eric MADELAINE ---- OASIS GCM Scopes and Objectives: Grid Codes that Compose and Deploy No programming, No Scripting, … No Pain Innovation: Abstract Deployment Composite Components Multicast and GatherCast MultiCast GatherCast

18. Eric MADELAINE ---- OASIS18 Opportunity to use GCM for autonomic computing Dynamic to Autonomic component based system reconfiguration Architecture of GCM membranes How to plug autonomous strategies to drive all non- functional concerns EU BIONETS IP project, P. Naoumenko BDO PACA ½ funded PhD: A GCM-based framework for autonomic and evolvable service compositions along bio-inspired strategies Distributed and reconfigurable service compositions

19. Eric MADELAINE ---- OASIS19 Motivation Building safe applications Heterogeneous Infrastructures: Multi-Cores, P2P, Grids, Clouds Context MCorePhP collaborative project Active Objects, Distributed Components Safe Distributed Components Behavioural Semantics Model generation Checking Properties Specification and Verification Tools, Case Study VerCors platform Case-Study Conclusion & Perspectives Agenda

20. Eric MADELAINE ---- OASIS20 Applications : Check behavioural compatibility between sub-components Check correctness of component deployment Check correctness of the transformation inside a running application. Behaviour specification and Safe composition Aim : Build reliable components from the composition of smaller pieces, using their formal specification. Component paradigm : only observe activity at interfaces. Behavioural properties: Deadlock freeness, progress/termination, safety and liveness.

21. Eric MADELAINE ---- OASIS21 pNets : Hierarchical and Parameterized LTSs [Arnold, Nivat 92] Synchronization networks [Lin 92] symbolic graphs with assignments [Lakas 96] semantics of Lotos open expressions Value-passing, Dynamic architectures, etc. But close to code structure Instantiation to finite structures (through abstract interpretation) [Forte’04: T. Barros, R. Boulifa, E. Madelaine] [Annals of Telecomunications’08: T. Barros, A. Cansado, L. Henrio, E. Madelaine]

22. Eric MADELAINE ---- OASIS22Eric MADELAINE22 pNets : generalized parallel operator PhiloNET : k  [1:n] A g = { Think(k), TakeL(k), … } with synchronisation vectors : Philo[k] Fork[k] Take Drop TakeL TakeR DropR DropL Think Eat PhiloNET

23. Eric MADELAINE ---- OASIS23 Building Behavioural Models : Principles For a given language/framework, define an operational semantics that builds pNets from the program structure. For Fractal or GCM components: Reason separately at each composition level Primitive components : functional behaviour is known Given by the user (specification language) Obtained by static analysis (primitive components, e.g. ProActive active objects) Composites : Computed from lower level Structure and NF behaviour automatically added from the component’s ADL

24. Eric MADELAINE ---- OASIS24 (1) Program semantics ==> Behaviour Model (parameterized) user-specified abstract interpretation (2) Behaviour Model ==> Finite Model Value Passing case : d efine an abstract representation from a finite partition of the value domains, on a per-formula basis  Preservation of safety and liveness properties [Cleaveland & Riely 93] Families of Processes : no similar generic result (but many results for specific topologies). Counter-example : on parameterized topologies of processes, reachability properties require induction reasoning. Practical approach : explore small finite configurations in a “bug search” fashion, use “infinite systems” techniques for decidable domains when available Abstractions and Correctness Interesting research subject here…

25. Eric MADELAINE ---- OASIS25 - Various temporal logics (CTL, ACTL, …) -Regular μ-calculus (Mateescu’2004) : the assembly language of temporal logics -Specification patterns (Dwyer’199x) Verification of Properties Deadlock Freeness No Error during deployment : [ (not √ ) *. O E ] false

26. Eric MADELAINE ---- OASIS26 Functional properties under reconfiguration (respecting the topology) Future update (asynchronous result messages) independent of life-cycle or binding reconfigurations Build a model including life-cycle controllers, with the reconfiguration actions visible: Then we can prove: Verification of Properties [ true*.Req_Get() ]  X. ( true  [  Resp_Get() ] X ) -Adapt model generation to the properties you want to prove

27. Eric MADELAINE ---- OASIS27 Motivation Building safe applications Heterogeneous Infrastructures: Multi-Cores, P2P, Grids, Clouds Context MCorePhP collaborative project Active Objects, Distributed Components Safe Distributed Components Behavioural Semantics Model generation Checking Properties Specification and Verification Tools, Case Study VerCors platform Case-Study: dating agreement protocol with group communication Conclusion & Perspectives Agenda

28. Eric MADELAINE ---- OASIS28 The Vercors Specification and Verification Platform (middle term) JDC Specification Graphical Editor (Eclipse Plugin) Vercors JDC Formula G C M / ProActi ve Code Generator ADL/IDL (final) Java Skeletons Business code Runtime pNets/ Fiacre Model Generator Finite model Formula Compiler Prover

29. Eric MADELAINE ---- OASIS29 Graphical Specifications : VCE tool

30. Eric MADELAINE ---- OASIS30 Verification Tools CADP toolset (INRIA Rhône-Alpes, VASY team) Generic Front-end (Lotos, BCG, Sync-vectors) Model generation: distributed on a cluster Large RAM space: Up to billions of states On-the-fly, Tau-reduction, Constrained… Verification Evaluator tool: Deadlock search / Regular μ-calculus Bisimulation ckecking, minimizing Less optimized than “classical” US model-checkers (Spin, etc) But scales better

31. Eric MADELAINE ---- OASIS31 Case-study: dating agreement protocol with group communication ProActive-based application, with: Active objects (single thread, no shared memory) Asynchronous (bounded) request queues Group communication No component structure Initiator Participant A B

32. Eric MADELAINE ---- OASIS32Eric MADELAINE ---- OASIS32 Generated Model: the full picture This is a small system: 10 pLTS 6 int. parameters 1 array parameter 11 pNets 19 synch vectors, including 3 broadcast and 2 collectors.

33. Eric MADELAINE ---- OASIS33Eric MADELAINE33 State generation 1: BRUTE FORCE -With no hierarchical minimization: the generation of a stand-alone group of 3 participants would be impossible -It is essential to build sub-systems in the correct context (= behavioural contract) => e.g. Projector tool of CADP. Brute forceMinimizedTotal time Single participant1 801 / 5 33890 / 3768 " Initiator3 163 / 152 08154 / 1 48911 " Full system, queue of length 2 170 K / 1 646 K458 / 1 284406 " Group of 3 participants > 10^11 states-- 3 participants Data ∈ { d1,d2 } Res ∈ Bool 15 visible labels Machine: Fedora 10, 4Go RAM 2 dual-core proc@2,4Ghz

34. Eric MADELAINE ---- OASIS34Eric MADELAINE34 State generation 2: distributed Principles: State space partitioned on a cluster by a static hash function. No shared memory. The state space is merged before other tools (minimization, model- checking) can be applied. Distributed MC is planned in future versions of CADP. On the fly partial order reduction available: Tau-compression (collapsing tau-chains) Tau-confluence (selecting only representatives of confluent-sets)

35. Eric MADELAINE ---- OASIS35Eric MADELAINE35 State generation 2: distributed -Distributed state generation has a (fixed) overhead, but allow for very large RAM configurations. The bottleneck is the merge phase. -On-the-fly partial-order reduction techniques may help to save memory space, at a high price. It may also fail… generationBrute forceTotal time Full system with 3 participants (8x4 cores) Brute force Tau-compression Tau-confluence 170 K / 1 646 K 170 K / 607 K 5 K / 14 K 6’45 11’48 30’ Group of 2 participants (15x8 cores) Brute force Tau-confluence 13 M / 48 M 392 K / 1 354 K 11’32 19h 10’55 Group of 3 participants Brute force Tau-confluence (estimated 125 G states) Out of memory during local computation

36. Eric MADELAINE ---- OASIS36Eric MADELAINE36 State generation 3: hierarchical / compositional Classical compositional state generation: Split the application into smaller pieces, minimize each with (branching) bisimulation before combining them. Distributed verification architecture: Define the verification activities as a workflow, and use a generic scheduler on the cloud infrastructure. Some of the workflow nodes are multinode (distributed) tasks.

37. Eric MADELAINE ---- OASIS37 Task 1 Book nodes; Prepare nodes; Build GCF Task 2.1 Compile client; Generate state space Task 2.2 Compile server; Generate state space Task 2.3 Merge sources Task 3 Rename Participants Task 4 Build product; Minimization Config1.gcf Config2.gcf InitiatorOptim.fcr Participant.fcr InitiatorOptim.bcgParticipant.bcg Participant$K[i].svl Participant$K[i].bcg SystemMin.bcg Flac + Distributor Flac + Distributor SVL BCG tools System.exp

38. Eric MADELAINE ---- OASIS38Eric MADELAINE38 State generation 3: hierarchical Classical compositional state generation: Split the application into smaller pieces, minimize each with (branching) bisimulation before combining them.  The biggest intermediate structure has ~ 3000 states before reduction.  A group of 3 (reduced) participants would be 90^3 = 800 000 states. Distributed verification architecture: Define the verification activities as a workflow, and use a generic scheduler on the cloud infrastructure. Some of the workflow nodes are multinode (distributed) tasks. => Open questions: build formalism and tool support to specify - the structural splitting - the mapping to verification tasks.

39. Eric MADELAINE ---- OASIS39 Motivation Building safe applications Heterogeneous Infrastructures: Multi-Cores, P2P, Grids, Clouds Context MCorePhP collaborative project Active Objects, Distributed Components Safe Distributed Components Behavioural Semantics Model generation Checking Properties Specification and Verification Tools, Case Study VerCors platform Case-Study Conclusion & Perspectives Agenda

40. Eric MADELAINE ---- OASIS40 Conclusions Starting Point: the pNETs model for behavioural semantics Semantic model for hierarchical, parameterized asynchronous systems Flexible, expressive and compact. We have define the semantics of: active objects, fractal components, asynchronous components, group communication, component reconfiguration… Tool support for (graphical) architecture specification, bridges to model-checking tool sets. This presentation summarizes our recent work on extensions for group communication Ongoing work on component reconfiguration… Papers, Use-cases and Tools, Position Offers at : http://www-sop.inria.fr/oasis/Vercors

41. Eric MADELAINE ---- OASIS41 Perspectives Extensions : Parameterized components in the specification language Tool support for abstraction specification Platform, Engines, Languages for Distributed Model-Checking Specialized model-checking engines for decidable classes of problems: –unbound fifo channels –Counters + presburger arithmetics Code Generation : From Architecture and Behaviour Diagrams … to ADL descriptions and GCM/ProActive code skeletons

42. Eric MADELAINE ---- OASIS42 “Safe by construction” Code Generation Generate code from high-level specification Generate behaviour model Generate GCM/ProActive code: Isolate the code that the developer should not modify Guaranteed properties (typically deadlock freedom, reachability) More precise than static analysis ! Perspectives ► User specification environment:  Eclipse + Graphical Editor (Component architecture, FSM-based behaviour)

43. Eric MADELAINE ---- OASIS43 Thank you 谢谢 Papers, Use-cases and Tools, Position Offers at : http://www-sop.inria.fr/oasis/Vercors

44. Eric MADELAINE ---- OASIS GCM : Grid Component Model Objective Extension of Fractal for programming Grids Innovations: Abstract Deployment Multicast and GatherCast Controller (NF) Components Standardization By the ETSI TC-GRID

45. Eric MADELAINE ---- OASIS45 Opportunity to use GCM+ProActive for agile enterprise grid computing Autonomic to agile enterprise and business services SOA @ IT Level Infrastructure Exploit Open Architecture of GCM membranes + GCM inherent grid awareness deployment SOA @ Business Level

46. Eric MADELAINE ---- OASIS46 CPER: IC2D on Blades+P2P (see demo)

47. Eric MADELAINE ---- OASIS47 My Definition : Software modules, composable, reconfigurable, with well-defined interfaces, and well-defined black box behaviour Our interests : 1. Encapsulation Black boxes, offered and required services 2. Composition Design of complex systems, hierarchical organization into sub-systems 3. Separation of concerns Architecture Description Language (ADL), management components 4. Distribution (e.g. Computational Grid) Interaction at interfaces through asynchronous method calls Software Components

48. Eric MADELAINE ---- OASIS48 The pNET model Parameterized Networks of Synchronised Automata Specification language Source code pNets system Abstraction Instantiation Verification tools Constraint: domains in pNets are “simple types”. The data domains in the source language have to be abstracted beforehand.

49. Eric MADELAINE ---- OASIS49 Parameterized LTSs : definition Given : A set of parameters V (with domains in first order “simple types”) An many-sorted term algebra ∑ V, with a distinguished Action sort A parameterized LTS is in which: Each state s  S has free variables fv(s)  V Labels l  L have the form e B  ∑ V,Bool a guard α  ∑ V,Action a parameterized action x j := e j an assignment of the state variables i,x i,y y=x-1