Denis Caromel 1

2

Denis Caromel 3 Clouds

Denis Caromel 4 Franca Perrina (Italy) Florin Alexandru Bratu (Romania) Denis Caromel (France, Nice) 3 Speakers/Demos

Denis Caromel 5 3 Strong Faiths 1.Parallel Computing will make it mainstream with Java 1.A need for a unified Parallel Abstraction: Multi-Core + Distributed 3. Java will make it possible to connect Enterprise Grids and Clouds

Denis Caromel 6 Agenda Effective SOA + GRIDs with 1.Background: INRIA, OASIS, ActiveEon 2.Programming, Optimizing 3. Scheduling 4. SOA: WS, WSDL, BPEL (Franca Perrina) 5. Enterprise Grids, Clouds: Amazon EC2 6. Java EE EJB (Florin Alexandru Bratu)

Denis Caromel 7 1. Background

Denis Caromel 8 OASIS Team & INRIA Computer Science and Control 8 Centers all over France Workforce: Strong in standardization committees: – IETF, W3C, ETSI, … Strong Industrial Partnerships Foster company foundation: 90 startups so far - Ilog (Nasdaq, Euronext) - … - ActiveEon A joint team between: INRIA, Nice Univ. CNRS Now about 40 persons 2004: First ProActive User Group 2008: 5 th one, Acad./Indus. User Presentations ProActive 4.0.1: Distributed and Parallel: From Multi-cores to Enterprise GRIDs

Denis Caromel 9 Startup Company Born of INRIA Co-developing, Providing support for Open Source ProActive Parallel SuiteProActive Parallel Suite Worldwide Customers (EU, Boston USA, etc.)

Denis Caromel Programming Optimizing Parallel Acceleration Toolkit in Java: Parallelism: Multi-Core+Distributed Used in production by industry

Denis Caromel 11

Denis Caromel 12

Denis Caromel 13 ProActive Parallel Suite

Denis Caromel 14 ProActive Parallel Suite

Denis Caromel 15 Distributed and Parallel Active Objects

Denis Caromel 16 A ProActive : Active objects Proxy Java Object A ag = newActive (“A”, […], VirtualNode) V v1 = ag.foo (param); V v2 = ag.bar (param);... v1.bar(); //Wait-By-Necessity V Wait-By-Necessity is a Dataflow Synchronization JVM A Active Object Future Object Request Req. Queue Thread v1 v2 ag WBN!

Denis Caromel 17 Standard system at Runtime: No Sharing NoC: Network On Chip Proofs of Determinism

Denis Caromel 18 TYPED ASYNCHRONOUS GROUPS

Denis Caromel 19 A Creating AO and Groups Typed Group Java or Active Object A ag = newActiveGroup (“A”, […], VirtualNode) V v = ag.foo(param);... v.bar(); //Wait-by-necessity V Group, Type, and Asynchrony are crucial for Composition JVM

Denis Caromel 20 Broadcast and Scatter JVM ag cg ag.bar(cg); // broadcast cg ProActive.setScatterGroup(cg) ; ag.bar(cg); // scatter cg c1 c2 c3 c1 c2 c3 c1 c2 c3 c1 c2 c3 c1 c2 c3 c1 c2 c3 s c1 c2 c3 s Broadcast is the default behavior Use a group as parameter, Scattered depends on rankings

Denis Caromel 21 Optimizing

Denis Caromel 22

Denis Caromel 23

Denis Caromel 24 IC2D

Denis Caromel 25 ChartIt

Denis Caromel 26 Pies for Analysis and Optimization

Denis Caromel 27 Video 1: IC2D Optimizing Monitoring, Debugging, Optimizing

Denis Caromel Scheduling

Denis Caromel 29

Denis Caromel 30 Scheduler and Resource Manager: User Interface

Denis Caromel 31 Scheduler: User Interface

Denis Caromel 32 Video 2: Scheduler, Resource Manager

Denis Caromel SOA Integration: Web Services, BPEL Workflow Franca Perrina OASIS Team - INRIA

Denis Caromel 34 Active Objects as Web Services Why ? Access Active Objects from any language How ? HTTP Server SOAP Engine (Axis) Usage: ProActive.exposeAsWebService(); ProActive.unExposeAsWebService(); JVM Web Service Client Web Services

Denis Caromel 35 ProActive + Services + Workflows Principles: 3 kinds of Parallel Services 3. Domain Specific Parallel Services (e.g. Monte Carlo Pricing) 2. Typical Parallel Computing Services (Parameter Sweeping, D&C, …) 1.Basic Job Scheduling Services (parallel execution on the Grid)

Denis Caromel 36 3 kinds of Parallel Services 3. Domain Specific Parallel Services: providing business functionalities executed in parallel 2. Parallelization services: typical parallel computing patterns (Parameter Sweeping, D&C, …) 1. Job Scheduling service: Schedule and Run jobs in parallel on the Grid. Parallel Services Operational Services … 1. Job Scheduling Service 2. Parameter Sweeping Service 2. Divide & Conquer Service Other Operational Service … Domain Specific Parallel Service … Other Basic Service High level Business ProcessGrid 3. Domain Specific Parallel Service

Denis Caromel 37 A sample pattern: Parameter Sweeping I1 I2 … In O1 O2 … On Exec Logic parameter sweeping I1 I2 … In O1 O2 … On Parameter Sweeping Service, customized with an Exec logic X Param Sweeping Service Process using parameter sweeping service … All the running instances of the Exec logic X are executed on the grid as a whole PA Scheduler & PA Resource Manager

Denis Caromel 38 Demo SOA Integration: Web Services, BPEL Workflow

Denis Caromel Enterprise Grids, Clouds: Standards & Amazon EC2

Denis Caromel 40 Deploy on Various Kinds of Infrastructures Internet Clusters Parallel Machine Large Equipment Internet Job management for embarrassingly parallel application (e.g. SETI) Internet Servlets EJBsDatabases

Denis Caromel 41 GCM Fractal Standardization Fractal Based Grid Component Model Overall, the standardization is supported by industrials: BT, FT-Orange, Nokia-Siemens, Telefonica, NEC, Alcatel-Lucent, Huawei …

Denis Caromel 42 Protocols and Scheduler in GCM Deployment Standard Protocols: Rsh, ssh Oarsh, Gsissh Scheduler, and Grids: GroupSSH, GroupRSH, GroupOARSH ARC (NorduGrid), CGSP China Grid, EEGE gLITE, Fura/InnerGrid (GridSystem Inc.) GLOBUS, GridBus IBM Load Leveler, LSF, Microsoft CCS (Windows HPC Server 2008) Sun Grid Engine, OAR, PBS / Torque, PRUN Soon available in stable release: Java EE Amazon EC2

Denis Caromel J2EE Integration Florin Alexandru Bratu OASIS Team - INRIA

Denis Caromel 44 J2EE Integration with Parallelism + Grids/Clouds Performing Grid & Cloud Computing From & In an Application Servers 1.Delegating heavy computations outsides J2EE Applications 2.Using Deployed J2EE Nodes as Computational Resources

Denis Caromel 45 ProActive – J2EE Integration (1) 1.Delegating heavy computations outsides J2EE Applications

Denis Caromel 46 ProActive – J2EE Integration (2) 2. Using Deployed J2EE Nodes as Computational Resources Objective Being able to deploy active objects inside the JVMs of application servers Implementation Based on a Sun standard – Java Connector Architecture JSR112 Deployment module: resource adapter (RAR)‏ Works with all J2EE-compliant Application Servers

Integration(2)‏

Denis Caromel 48 Grids & Clouds: Amazon EC2 Deployment

Denis Caromel 49 Big Picture + Clouds

Denis Caromel 50 Clouds: ProActive Amazon EC2 Deployment Principles & Achievements: ProActive Amazon Images (AMI) on EC2 So far up to 128 EC2 Instances (Indeed the maximum on the EC2 platform, … ready to try AMI) Seamless Deployment: no application change, no scripting, no pain Open the road to : In house Enterprise Cluster and Grid + Scale out on EC2

Denis Caromel 51 ProActive Deployment on Amazon EC2 Video

Denis Caromel 52 On Going AGOS Grid Architecture for SOA

Denis Caromel 53 AGOS Grid Architecture for SOA AGOS Solutions Building a Platform for Agile SOA with Grid In Open Source with Professional Support

Denis Caromel 54 AGOS Generic Architecture for SOA with GRIDs OS, HW OS VirtualizationGrid Utility interface ESB Enterprise Service Bus SCA Service Component Architecture Resource Manager Task & Services Scheduling Parallel Programming SPMD, workflow Agent, Master/Worker Fork and Join In memory db cache (JSR / JPI / javaspaces) SOA BPEL Exec Repository, Registry, Orchestration SOA Monitoring Reporting, Notifications, alarms Business Intelligence BI Monitoring SLM Service Level Management

Denis Caromel 55 Summary

Denis Caromel 56 Conclusion: Effective SOA + GRIDs in Java with An Acceleration Toolkit : Concurrency+Parallelism Multi-Core+Distributed Grid & SOA: J2EE, WS, BEPL, EG + Amazon EC2

Q&A

Denis Caromel 58