A Lightweight Platform for Integration of Mobile Devices into Pervasive Grids Stavros Isaiadis, Vladimir Getov University of Westminster, London {s.isaiadis, HPCC 2005, Sorrento, Italy

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 2 Outline Overview & Motivation Platform Architecture Current Implementation Status Future Plans Conclusion & Questions

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 3 Overview - Goals A proxy-based architecture that will allow the integration of mobile/limited devices into Grid systems in an efficient way A lightweight components-based platform that can be deployed in limited devices without draining resources A set of middleware components built on top of the core platform, including (among other): –A lightweight monitoring framework –Failure resilience components that will provide a virtually reliable environment –Parallelization components to support automatic task extraction and distribution (assuming suitable parallel/distributed programming model)

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 4 Motivation Future Grid systems should be truly pervasive and ubiquitous While access to the Grid and job submission from mobile/limited devices is generally supported (e.g. through portals), contribution of resources is very difficult Resource constraints make installation of Grid middleware prohibiting Mobile and pervasive computing attract a big part of current research and industry funds and hold a big share of the market pie Small and/or mobile devices increasingly offer functionality not available in traditional Grid nodes –multimedia equipment, sensors, global positioning systems etc.

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 5 Outline Overview & Motivation Platform Architecture Current Implementation Status Future Plans Conclusion & Questions

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 6 Integration Problems: Dynamicity & Limited Resources Dynamicity of small devices may penalize overall Grid performance and is not acceptable in complex and heavily loaded Grid systems Solution: –Hide devices behind a proxy and delegate finer control (scheduling, monitoring, recovery etc) to a local community system Resource limitations make contribution of resources difficult and inefficient Solution: –Similar services are aggregated and published through a single interface at the proxy –A single consistent interface to all aggregated resources is presented regardless of the internal state of the cluster –Devices now form a virtual cluster and are presented as a single entity to the Grid.

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 7 Service Aggregation SERVICE A Virtual cluster PROXY The Grid SERVICE A Aggregation CLIENT Service invocation Forward request (single, parallel or mirrored)

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 8 Providing a Virtually Stable and Flexible Environment When aggregating services that provide controlled access to raw resources (like CPU cycles, memory, storage etc.), we dont present the total aggregate but only a fraction This way we can mask internal failures by reallocating resources from within the cluster without notifying (and thus placing extra burden on) higher level Grid components A monitoring and failure recovery framework based on agents, heartbeats and an orchestrating gateway will help predict, detect, diagnose and recover from failures Forecasting components to support the community scheduling system providing recommendations on the best possible resource usage plans

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 9 COMPONENT IMPLEMENTED Platform Architecture VIRTUALIZATION LAYER (AGGREGATOR SERVICE INTERFACES) PLATFORM KERNEL COMPONENT NOT IMPLEMENTED YET MONITORING FAILURE RESILIENCE RESOURCE MANAGEMENT INDEXING REGISTRATION PARALLEL PROGRAMMING MODEL HIGHER LEVEL SERVICES LAYER MIDDLEWARE SERVICES LAYER CORE LAYER

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 10 Virtual Clustering VIRTUALIZATION LAYER (AGGREGATOR SERVICE INTERFACES) PLATFORM KERNEL MONITORING FAILURE RESILIENCE RESOURCE MANAGEMENT INDEXING REGISTRATION PARALLEL PROGRAMMING MODEL Proxy Virtual Cluster The Grid

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 11 Outline Overview & Motivation Platform Architecture Current Implementation Status Future Plans Conclusion & Questions

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 12 Implementation Status Pure Java implementation to ensure a high degree of interoperability Currently following a service-based approach –Both Grid and Web Services are supported A CCA compliant version is under way as a more lightweight alternative to the current service-based implementation Currently implemented components: –Registration/aggregation –Indexing –Resource management (partially) Registration and Indexing components are available both as Web Services and as Active RMI Objects registered in the proxy

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 13 HOSTING DEVICE Aggregation Operational Overview Aggregator WSDL Generator WSDL Parser Aggregator Service Generator Deploy Descriptors Generator Service Archive Generator Service Deployer WEB/GRID SERVICE CONTAINER SERVICE AGGREGATION SERVICE SERVICE WSDLMETA- DATA 1.Service provider sends the WSDL description of the service. Static meta data is automatically collected and forwarded. 2.The WSDL Parser extracts all necessary service parameters INFO 3.These are then fed to a WSDL generator that will generate the WSDL description for the aggregator service INFO WSDL 4.From the WSDL we generate the actual Java interface and implementation classes. We also add our own interfaces for mirrored or parallel execution and for administering the service and querying for meta-data INTF WSDD IMPL 5.All the resulting classes along with the deployment descriptors are used in order to create the service archive JAR/ GAR Deploy 6.The service archive is then deployed to the Web/Grid service container of choice META- DATA MAP Update 7.Finally, the indexing components are updated to include the new hosting device meta-data and the new aggregator-to-original service mappings INDEXING SERVICE Resource Meta-data Repository Aggregation Map Dynamic Index

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 14 Invocation Operational Overview SERVICE BSERVICE A SERVICE DSERVICE C WEB/GRID SERVICE CONTAINER SERVICE CLIENT SERVICE INDEXING SERVICE Resource Meta-data Repository Aggregation Map Dynamic Index MONITORING COMPONENTS Availability Forecasting Job Meta- Data Repository Monitoring Gateway AGGREGATOR SERVICE IMPLEMENTATION 1. A 2. C 3. … Virtual Cluster Proxy The Grid Service request 1.A client forwards a request to the aggregator service. Get available services 2.The aggregator service will query the index service and get back a prioritized list of services that match the requirements Execute 3.The aggregator will forward the request to the highest prioritized service )or services in the case of mirrored or parallel execution) Updates The index is updated in a push fashion from the monitoring framework components

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 15 Outline Overview & Motivation Platform Architecture Current Implementation Status Future Plans Conclusion & Questions

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 16 Future Plans Design and implement a lightweight monitoring framework –Small monitoring agents in the cluster devices –Monitoring gateway to collect information and orchestrate actions –Forecasting facilities to provide recommendations on the best resource usage plan, based on monitoring information statistics like usage and failure history Support for roaming between Virtual Clusters Design and implement a failure recovery framework Adopt a suitable parallel/distributed programming model –Provide suitable interfaces for parallelization parameters, selection requirements and querying executing jobs

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 17 Outline Overview & Motivation Platform Architecture Current Implementation Status Future Plans Conclusion & Questions

European Research Network on Foundations, Software Infrastructures and Applications for large scale distributed, GRID and Peer-to-Peer Technologies 18 Conclusion The Virtual Cluster architecture will allow the efficient integration of mobile and limited devices into the Grid The aggregation technique coupled with a suitable parallel programming model can provide the foundation for enhanced performance The monitoring and failure recovery framework can provide a virtually stable and reliable environment, along with relatively high service availability

Questions? …or comments, suggestions, ideas… Contact: You can find this presentation here: