Utility-Oriented Cloud & Grid Computing: A Vision, Hype, and Reality

Slides:



Advertisements
Similar presentations
Convergence Characteristics for Clusters, Grids, and P2P networks
Advertisements

Symantec 2010 Windows 7 Migration EMEA Results. Methodology Applied Research performed survey 1,360 enterprises worldwide SMBs and enterprises Cross-industry.
Symantec 2010 Windows 7 Migration Global Results.
AP STUDY SESSION 2.
1
Nimrod/G GRID Resource Broker and Computational Economy
Grid Business Symposium 2005, Seoul, Korea
Libra: An Economy driven Job Scheduling System for Clusters Jahanzeb Sherwani 1, Nosheen Ali 1, Nausheen Lotia 1, Zahra Hayat 1, Rajkumar Buyya 2 1. Lahore.
Nimrod/G and Grid Market A Case for Economy Grid Architecture for Service Oriented Global Grid Computing Rajkumar Buyya, David Abramson, Jon Giddy Monash.
Pricing for Utility-driven Resource Management and Allocation in Clusters Chee Shin Yeo and Rajkumar Buyya Grid Computing and Distributed Systems (GRIDS)
Grid Economics and Business Models: A Gridbus Perspective Rajkumar Buyya Grid and Distributed Systems (GRIDS) Laboratory Dept. of Computer Science and.
Major Sponsors/Supporters
Distributed Systems Architectures
Processes and Operating Systems
Evaluating Caching and Storage Options on the Amazon Web Services Cloud Gagan Agrawal, Ohio State University - Columbus, OH David Chiu, Washington State.
OGF19 -- NC 1 Service Level Agreements and QoS: what do we measure and why? Omer F. Rana School of Computer Science, Cardiff.
Enterprise Computing Strategies This gig is bigger than grid: from HPC to service orientation John Barr Research.
David Burdett May 11, 2004 Package Binding for WS CDL.
CALENDAR.
Research Councils ICT Conference Welcome Malcolm Atkinson Director 17 th May 2004.
Cloud Resource Broker for Scientific Community By: Shahzad Nizamani Supervisor: Peter Dew Co Supervisor: Karim Djemame Mo Haji.
Hello i am so and so, title/role and a little background on myself (i.e. former microsoft employee or anything interesting) set context for what going.
AN INGENIOUS APPROACH FOR IMPROVING TURNAROUND TIME OF GRID JOBS WITH RESOURCE ASSURANCE AND ALLOCATION MECHANISM Shikha Mehrotra Centre for Development.
Break Time Remaining 10:00.
Virtualization & Disaster Recovery
PP Test Review Sections 6-1 to 6-6
The Platform as a Service Model for Networking Eric Keller, Jennifer Rexford Princeton University INM/WREN 2010.
© 2010 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 1 Taiwan ITQ.
CS 6143 COMPUTER ARCHITECTURE II SPRING 2014 ACM Principles and Practice of Parallel Programming, PPoPP, 2006 Panel Presentations Parallel Processing is.
Cloud Computing for Education & Cloud Learning Minjuan Wang to BT Research Center (Abu Dhabi) Educational Technology San Diego State University
Operating Systems Operating Systems - Winter 2010 Chapter 3 – Input/Output Vrije Universiteit Amsterdam.
Sample Service Screenshots Enterprise Cloud Service 11.3.
Copyright © 2012, Elsevier Inc. All rights Reserved. 1 Chapter 7 Modeling Structure with Blocks.
SLP – Endless Possibilities What can SLP do for your school? Everything you need to know about SLP – past, present and future.
MaK_Full ahead loaded 1 Alarm Page Directory (F11)
Center on Knowledge Translation for Disability and Rehabilitation Research Information Retrieval for International Disability and Rehabilitation Research.
Before Between After.
1 Chapter 11: Data Centre Administration Objectives Data Centre Structure Data Centre Structure Data Centre Administration Data Centre Administration Data.
1 hi at no doifpi me be go we of at be do go hi if me no of pi we Inorder Traversal Inorder traversal. n Visit the left subtree. n Visit the node. n Visit.
Mushroom Softech Pvt. Ltd.1 eCRM-Presentation Security Administration Home/Login page General Masters Customer Registration Customer Wise Search Fill up.
Speak Up for Safety Dr. Susan Strauss Harassment & Bullying Consultant November 9, 2012.
Converting a Fraction to %
Clock will move after 1 minute
Physics for Scientists & Engineers, 3rd Edition
Select a time to count down from the clock above
Introduction Peter Dolog dolog [at] cs [dot] aau [dot] dk Intelligent Web and Information Systems September 9, 2010.
1 Cloud Computing Prof. Ravi Sandhu Executive Director and Endowed Chair April 12, © Ravi Sandhu World-Leading.
SLA-Oriented Resource Provisioning for Cloud Computing
INTRODUCTION TO CLOUD COMPUTING CS 595 LECTURE 6 2/13/2015.
Aneka: A Software Platform for .NET-based Cloud Computing
CCGrid 2014 Panel: Architect Cloud and HPC for Big Data Era Rajkumar Buyya Cloud Computing and Distributed Systems (CLOUDS) Lab Dept. of Computer Science.
An Introduction to Cloud Computing. The challenge Add new services for your users quickly and cost effectively.
Plan Introduction What is Cloud Computing?
Clouds on IT horizon Faculty of Maritime Studies University of Rijeka Sanja Mohorovičić INFuture 2009, Zagreb, 5 November 2009.
QoS-based Scheduling of e-Research Application Workflows on Global Grids Dr. Rajkumar Buyya Grid Computing and Distributed Systems (GRIDS) Laboratory Dept.
Gridbus Resource Broker for Application Service Costs-based Scheduling on Global Grids: A Case Study in Brain Activity Analysis Srikumar Venugopal 1, Rajkumar.
PhD course - Milan, March /09/ Some additional words about cloud computing Lionel Brunie National Institute of Applied Science (INSA) LIRIS.
Cloud computing is the use of computing resources (hardware and software) that are delivered as a service over the Internet. Cloud is the metaphor for.
GCC 2006 Panel: Grid Research and Engineering Vs Standards Dr. Rajkumar Buyya Grid Computing and Distributed Systems (GRIDS) Laboratory Dept. of Computer.
Gridbus Toolkit for Belle Analysis Data Grid and Utility Computing Rajkumar Buyya Grid Computing and Distributed Systems (GRIDS) Lab. Dept. of Computer.
Nimrod/G GRID Resource Broker and Computational Economy David Abramson, Rajkumar Buyya, Jon Giddy School of Computer Science and Software Engineering Monash.
Grid Resource Management: Challenges, Approaches, & Solutions Dr. Rajkumar Buyya Cloud Computing and Distributed Systems (CLOUDS) Lab. The University of.
1 520 Student Presentation GridSim – Grid Modeling and Simulation Toolkit.
Vision, Hype, and Reality for delivering IT Services as Computing Utilities By Rajkumar Buyya Chee Shin Yeo Srikumar Venugopal.
Economic and On Demand Brain Activity Analysis on Global Grids A case study.
Aneka Cloud ApplicationPlatform. Introduction Aneka consists of a scalable cloud middleware that can be deployed on top of heterogeneous computing resources.
1 TCS Confidential. 2 Objective : In this session we will be able to learn:  What is Cloud Computing?  Characteristics  Cloud Flavors  Cloud Deployment.
INTRODUCTION TO GRID & CLOUD COMPUTING U. Jhashuva 1 Asst. Professor Dept. of CSE.
Cloud Computing Dr. Sharad Saxena.
Presentation transcript:

Utility-Oriented Cloud & Grid Computing: A Vision, Hype, and Reality Dr. Rajkumar Buyya Grid Computing and Distributed Systems (GRIDS) Lab Dept. of Computer Science and Software Engineering The University of Melbourne, Australia www.gridbus.org www.buyya.com www.manjrasoft.com Gridbus Sponsors

The GRIDS Lab @ Melbourne R & D Education Youngest and one of the rapidly growing research labs in our School/University: Founded in 2002 Houses 20+ researchers consisting of: Research Fellows/PostDocs Software Engineers PhD candidates Honours/Masters students Funding National and International organizations Australian Research Council & DEST Many industries (Sun, StorageTek, Microsoft, IBM, Microsoft) University-wide collaboration: Faculties of Science, Engineering, and Medicine Many national and international collaborations. Academics Industries Software: Widely in academic and industrial users. Publication: My research team produces over 20% of our Dept’s research output. + Community Services: e.g., IEEE TC for Scalable Computing

Agenda Introduction Global Grids and Challenges Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services Global Grids and Challenges Security, resource management, pricing models, … Service-Oriented Grid Architecture and Gridbus Solutions Market-based Management, GMD, Grid Bank, Aneka Grid Service Broker Architecture, Design and Implementation Performance Evaluation: Experiments in Creation and Deployment of Applications on Global Grids A Case Study in High Energy Physics Summary and Conclusion

“Computer Utilities” Vision: Implications of the Internet 1969 – Leonard Kleinrock, ARPANET project “As of now, computer networks are still in their infancy, but as they grow up and become sophisticated, we will probably see the spread of ‘computer utilities’, which, like present electric and telephone utilities, will service individual homes and offices across the country” Computers Redefined 1984 – John Gage, Sun Microsystems “The network is the computer” 2008 – David Patterson, U. C. Berkeley “The data center is the computer. There are dramatic differences between of developing software for millions to use as a service versus distributing software for millions to run their PCs” 2008 – “Cloud is the computer” – Buyya!

Attributes/Capabilities Computing Paradigms and Attributes: Realizing the ‘Computer Utilities’ Vision } ? Web Data Centres Utility Computing Service Computing Grid Computing P2P Computing Market-Oriented Computing Cloud Computing … + -Ubiquitous access -Reliability Scalability Autonomic Dynamic discovery Composability -QoS -SLA - … Trillion $ business Who will own it? Paradigms Attributes/Capabilities

* Since Grids have been around for sometime (early 2000), do we have a unified vision of what Grids can do? * And did we make sufficient advances to turn vision of “computer utilities” into a reality? - Let us take a look at views of “industrial” practitioners & “academics”

“Industrial” vision of Grid computing IBM On Demand Computing Microsoft .NET Oracle 10g Sun N1 – Sun Grid Engine HP Adaptive Enterprise Amazon Elastic Compute Cloud Services Manjrasoft Aneka for building enterprise Grids and Clouds.

Most academics view: Cyberinfrastructure for conducting collaborative (e-)Science

How do Grids look like? A Bird Eye View of a Global Grid Grid Information Service Grid Resource Broker database Application R2 R3 R4 R5 RN Grid Resource Broker R6 R1 Resource Broker Grid Information Service

How do Grids look like? A Bird Eye View of a Global Grid Grid Information Service Grid Resource Broker database Application R2 R3 R4 R5 RN Grid Resource Broker R6 R1 Resource Broker Grid Information Service

E-Business E-Science How Are Grids Used? Utility computing High-performance computing Collaborative design Financial modeling Collaborative data-sharing High-energy physics E-Business Life sciences Drug discovery Data center automation E-Science Natural language processing Business Intelligence (Data Mining)

Agenda Introduction Global Grids and Challenges Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services Global Grids and Challenges Security, resource management, pricing models, … Service-Oriented Grid Architecture and Gridbus Solutions Market-based Management, GMD, Grid Bank, Aneka Grid Service Broker Architecture, Design and Implementation Performance Evaluation: Experiments in Creation and Deployment of Applications on Global Grids A Case Study in High Energy Physics Summary and Conclusion

Grid Challenges Application Construction Security Uniform Access Security System Management Computational Economy Resource Discovery Resource Allocation & Scheduling Data locality Network Management Application Construction

Some Grid Initiatives Worldwide Australia Nimrod-G Gridbus DISCWorld GrangeNet. APACGrid ARC eResearch Brazil OurGrid, EasyGrid LNCC-Grid + many others China ChinaGrid – Education CNGrid - application Europe UK eScience EU Grids.. and many more... India Garuda Japan NAREGI Korea... N*Grid Singapore NGP USA Globus TeraGrid Cyberinfrasture AutoMate and many more... Industry Initiatives IBM On Demand Computing HP Adaptive Computing Sun N1 Microsoft - .NET Oracle 10g Amzon – Elastic Compute Cloud Infosys – Enterprise Grid Satyam – Business Grid Manjrasoft – enterprise Clouds and Grids and many more Public Forums Open Grid Forum Conferences: CCGrid Grid HPDC E-Science 1.3 billion – 3 yrs 27 million 2? billion 120million – 5 yrs 450million – 5 yrs 486million – 5 yrs 1.3 billion (Rs) 1 billion – 5 yrs http://www.gridcomputing.com

Open-Source Grid Middleware Projects OurGrid Slide by Hiro

Driving Theme: Community vs. Utility Grids Type Feature Community Grids Utility Grids (Now Clouds) User QoS Best effort Contract/SLA Service Pricing Not considered / free access Usage, QoS level, Market supply and demand Example Middleware Globus, Condor, OMII, Unicore Nimrod-G, Gridbus, & many inspired efforts (IBM Business Grid, Sun Grid Market) .. Amazon EC2.. “So what’s difference between scheduling tasks on utility grids and community grids.” This table shows some differences in terms of availability, Quality of Service and pricing. In utility Grids, users can make a reservation with a service in advance to ensure the service availability. For the Quality of Service, users can negotiate with service providers on service level agreements for their required quality of service. “Compared with utility services.” Services availability and quality of service in community grids may not be guaranteed. Community grids provide free access but in utility grids users need to pay for service access. In general, service pricing is based on the quality of service. Users don’t always prefer to use services with higher qos, since it charges more. So in a utility grid scheduler needs to allocate services to tasks based on users’ QoS requirements. For example, if a user want to process a task, there are multiple choices, one service can complete task in 4 min, the price is 100, second service can complete in 10 mins but only charge 20, a task also can be completed in just one min, but service provider charge 200. If user don’t need results immediately, see the deadline is 15 mins. Then the second service is best, it can complete task execution within the deadline but the cost is minimum.

Pushes Grid computing into mainstream computing The Gridbus Project @ Melbourne: Enable Leasing of ICT Services on Demand WWG Gridbus Pushes Grid computing into mainstream computing

Agenda Introduction Global Grids and Challenges Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services Global Grids and Challenges Security, resource management, pricing models, … Service-Oriented Grid Architecture and Gridbus Solutions Market-based Management, GMD, Grid Bank, Aneka Grid Service Broker Architecture, Design and Implementation Performance Evaluation: Experiments in Creation and Deployment of Applications on Global Grids A Case Study in High Energy Physics Summary and Conclusion

What do Grid players want & require? Grid Service Consumers (GSCs): - minimize expenses, meet QoS How do I express QoS requirements ? How do I trade between timeframe & cost ? How do I discover services and map jobs to meet my QoS needs? How do I manage Grid dynamics and get my work done? … Grid Service Providers (GSPs):– maximise ROI How do I decide service pricing models ? How do I specify them ? How do I translate them into resource allocations ? How do I enforce them ? How do I advertise & attract consumers ? How do I do accounting and handle payments? They need mechanisms, tools and technologies that help them in value expression, value translation, and value enforcement.

Service-Oriented Grid Architecture Data Catalogue Grid Bank Grid Market Services Information Service Sign-on Health Monitor Info ? Grid Explorer Grid Node N … … Programming Environments Secure Job Control Agent Applications Schedule Advisor Grid Node1 QoS Pricing Algorithms Trade Server Trading Trade Manager Accounting Resource Reservation Misc. services … Deployment Agent JobExec Resource Allocation Grid Resource Broker Storage R1 R2 … Rm Core Middleware Services Grid Service Consumer Grid Service Providers

Market-Oriented Grid Software: A union of Gridbus and other technologies … Grid Applications Science Commerce Engineering Collaboratories Grid Portals … APIs/Tools: MPI ExcellGrid Workflow APIs Task, Parametric, and Components Programming User-Level Middleware Grid Workflow Engine Grid Scheduling: Gridbus Resource Broker Globus Unicore … Grid Storage Economy Grid Exchange & Federation Grid Market Directory Core Grid Middleware Aneka Cloud (WS-based access + SLA Grid Bank NorduGrid XGrid Grid Economy JVM Condor PBS SGE Libra Tomcat Grid Fabric Software .NET Windows Solaris Linux AIX IRIX OSF1 Mac CDB Grid Fabric Hardware PDB Worldwide Grid

On Demand Assembly of Services in Market-Oriented Grid Environments Visual Application Composer Application Code Explore data 1 Results+ Cost Info 10 Grid Resource Broker 2 Bill 12 Data Catalogue 5 ASP Catalogue Grid Info Service Grid Market Directory 4 6 3 Job 8 Results 9 7 GSP (e.g., Microsoft) PE (e.g., Amazon) (e.g., IBM) CPU or PE Grid Service (GS) (Globus) Aneka EC2 GTS Resource Allocation GSP (Accounting Service) Gridbus GridBank 11

On Demand Assembly of Services in Market-Oriented Grid Environments

Enterprise/Private Clouds Public/Internet Clouds Cloud Services Infrastructure as a Services (IaaS) CPU, Storage: Amazon.com et. al Platform as a Services (PaaS) Google App Engine, Microsoft Azure, and Manjrasoft Aneka Software as a Service (SaaS) SalesForce.Com Enterprise/Private Clouds Clouds Public/Internet Clouds

Layered view of services within a Cloud stack Software as a Service (SaaS) e.g., ..SalesForce.com Platform as a Service (PaaS) e.g., ..Aneka Infrastructure as a Service (IaaS) e.g., Amazon, Nirvanix

Aneka A Software Platform for Building and Managing “Enterprise” Grids and Clouds

Aneka: A 3rd Gen enterprise Grid Technology  Cloud model Generation Properties Technologies First (-2001) Application-specific platforms distributed.net, SETI@Home Second (2002-2006) Single programming model, rigid architecture, no QoS UD, XtremeWeb, Alchemi, Digipede, DataSynapse, BOINC Third (2007-2012?) SOA, extensible architecture, multiple programming models, multi-tenancy, enterprise QoS, SLAs, market-based resource allocation Aneka

ANEKA – Product Overview (Alpha) .NET based service-oriented platform for grid / cloud computing Development and Run Time Environment Includes Development and Management Tools Suitable for Development of Enterprise Grid / Cloud Applications Grid / Cloud enabling legacy applications Ideal for Corporate Developers, Software, SaaS, Hosting Vendors and Application / System Integrators ANEKA Product Architecture

Aneka Deployment Models Enterprise/Private Harness LAN connected resources Application Development, Testing, Execution Teaching and Learning Sensitive applications Public Hosted by a 3rd party service provider owning a large Data Center (1000s of servers) Offers subscription-based services to their shared infrastructure on “pay-as go” model.to many users from different organisations. Amazon.com, Microsoft Azure Aneka SDK + Execution Manger Enterprise/Private Clouds Aneka Public Clouds

Programming / Deployment Model FIRST PRODUCT Aneka: components public DumbTask: ITask { … public void Execute() …… } Programming / Deployment Model Aneka enterprise Cloud for(int i=0; i<n; i++) { … DumbTask task = new DumbTask(); app.SubmitExecution(task); } Executor Executor work units Client Agent Executor Scheduler internet work units Executor Aneka Worker Service Aneka Manager Client Agent internet Aneka Users

How does it solve the problem? Divide the problem in to multiple small tasks and distribute them run in parallel on multiple computers within a Cloud. An Illustratioin Executor Application Manager Manager / Executor GThreads/Tasks

User scenario: GoFront (unit of China Southern Railway Group) Application: Locomotive design CAD rendering Raw Locomotive Design Files (Using AutoDesk Maya) Using Maya Graphical Mode Directly Case 1: Single Server 4 cores server Aneka Maya Renderer Use private Aneka Cloud GoFront Private Aneka Cloud LAN network (Running Maya Batch Mode on demand) Case 2: Aneka Enterprise Cloud Aneka utilizes idle desktops (30) to decrease task time from days to hours Time (in hrs) Single Server Aneka Cloud

Aneka: How can get it? Available to Download: Teaching material Software: www.manjrasoft.com Manual: Setting up Cloud using your LAN-network computers Teaching material parallel and distributed computing and programming, List of possible assignments for students Possible Projects for Final year students.. Price – highly affordable  = Fee you charge to 1 student (each year) and all students/teachers in entire college/university can use it! Applications Other Departments (Physics, Chemistry, Biology, Finance, Engineering) can use it for their applications.

On Demand Assembly of Services in Market-Oriented Grid Environments

Agenda Introduction Global Grids and Challenges Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services Global Grids and Challenges Security, resource management, pricing models, … Service-Oriented Grid Architecture and Gridbus Solutions Market-based Management, GMD, Grid Bank, Aneka Grid Service Broker Architecture, Design and Implementation Performance Evaluation: Experiments in Creation and Deployment of Applications on Global Grids A Case Study in High Energy Physics Summary and Conclusion

Grid Service Broker (GSB) A resource broker for scheduling task farming data Grid applications with static or dynamic parameter sweeps on global Grids. It uses computational economy paradigm for optimal selection of computational and data services depending on their quality, cost, and availability, and users’ QoS requirements (deadline, budget, & T/C optimisation) Key Features A single window to manage & control experiment Programmable Task Farming Engine Resource Discovery and Resource Trading Optimal Data Source Discovery Scheduling & Predications Generic Dispatcher & Grid Agents Transportation of data & sharing of results Accounting

App, T, $, Optimization Preference workload Gridbus User Console/Portal/Application Interface App, T, $, Optimization Preference Gridbus Broker Gridbus Farming Engine Schedule Advisor Record Keeper Trading Manager Grid Dispatcher Grid Explorer TM TS $ GE GIS, NWS Core Middleware Grid Info Server RM & TS G $ Data Catalog Data Node C $ U G Globus enabled node. L A Amazon EC2/S3 Cloud.

Gridbus Broker: Separating “applications” from “different” remote service access enablers and schedulers Application Development Interface Home Node/Portal Gridbus Broker fork() batch() -PBS -Condor -SGE -Aneka -XGrid Single-sign on security Alogorithm1 Scheduling Interfaces AlogorithmN Data Catalog Plugin Actuators Aneka -PBS -Condor -SGE Globus Job manager fork() batch() Gridbus agent Data Store AMI Amazon EC2 -PBS -Condor -SGE -XGrid SSH fork() batch() Gridbus agent Access Technology SRB Grid FTP

Gridbus Services for eScience applications Application Development Environment: XML-based language for composition of task farming (legacy) applications as parameter sweep applications. Task Farming APIs for new applications. Web APIs (e.g., Portlets) for Grid portal development. Threads-based Programming Interface Workflow interface and Gridbus-enabled workflow engine. … Grid Superscalar – in cooperation with BSC/UPC Resource Allocation and Scheduling Dynamic discovery of optional computational and data nodes that meet user QoS requirements. Hide Low-Level Grid Middleware interfaces Globus (v2, v4), SRB, Aneka, Unicore, and ssh-based access to local/remote resources managed by XGrid, PBS, Condor, SGE.

Click Here for Demo Drug Design Made Easy!

s

Agenda Introduction Global Grids and Challenges Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services Global Grids and Challenges Security, resource management, pricing models, … Service-Oriented Grid Architecture and Gridbus Solutions Market-based Management, GMD, Grid Bank, Aneka Grid Service Broker Architecture, Design and Implementation Performance Evaluation: Experiments in Creation and Deployment of Applications on Global Grids A Case Study in High Energy Physics Summary and Conclusion

Case Study: High Energy Physics and Data Grid The Belle Experiment KEK B-Factory, Japan Investigating fundamental violation of symmetry in nature (Charge Parity) which may help explain “why do we have more antimatter in the universe OR imbalance of matter and antimatter in the universe?”. Collaboration 1000 people, 50 institutes 100’s TB data currently High Energy Physics (HEP) is the study of the fundamental constituents of matter and the forces between these constituents. It is called High Energy Physics as using high energies enables us to probe smaller distances and structures within matter, and also allows us to study matter as it was in the early universe, the history of matter. It is also called Particle Physics as we deal with quanta of matter and forces and the properties associated with these. The study of HEP is broken into two main disciplines, theoretical and experimental. Theoretical HEP propose theories and models to describe matter, forces, their properties, actions, and interactions. Experimental HEP construct experiments or detectors and accelerators to investigate matter interactions and behaviour under high energy conditions. Experimental HEP can be roughly broken into 3 separate activities. The boundaries of these activities, in time and responsibility, are often indistinct. The activities are the construction of detectors which typically takes many years, the measurement or collection of data, and the analysis of this data. We will focus on the using data grids for the analysis of data within HEP.

Case Study: Event Simulation and Analysis B0->D*+D*-Ks Simulation and Analysis Package - Belle Analysis Software Framework (BASF) Experiment in 2 parts – Generation of Simulated Data and Analysis of the distributed data Analyzed 100 data files (30MB each) that were distributed among the five nodes within Australian Belle DataGrid platform.

Australian Belle Data Grid Testbed VPAC Melbourne

Belle Data Grid (GSP CPU Service Price: G$/sec) NA G$4 G$6 VPAC Melbourne G$2 Data node

Belle Data Grid (Bandwidth Price: G$/MB) 32 33 36 G$4 31 30 34 NA 38 31 G$4 G$6 VPAC Melbourne G$2 Data node

Deploying Application Scenario A data grid scenario with 100 jobs and each accessing remote data of ~30MB Deadline: 3hrs. Budget: G$ 60K Scheduling Optimisation Scenario: Minimise Time Minimise Cost Results:

Time Minimization in Data Grids 10 20 30 40 50 60 70 80 1 2 3 4 5 6 7 8 9 11 12 13 14 15 16 17 18 19 21 22 23 24 25 26 27 28 29 31 32 33 34 35 36 37 38 39 41 42 Time (in mins.) Number of jobs completed fleagle.ph.unimelb.edu.au belle.anu.edu.au belle.physics.usyd.edu.au brecca-2.vpac.org

Results : Cost Minimization in Data Grids 10 20 30 40 50 60 70 80 90 100 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 Time(in mins.) Number of jobs completed fleagle.ph.unimelb.edu.au belle.anu.edu.au belle.physics.usyd.edu.au brecca-2.vpac.org

Observation Organization Node details Cost (in G$/CPU-sec)   Node details Cost (in G$/CPU-sec) Total Jobs Executed Time Cost CS,UniMelb belle.cs.mu.oz.au 4 CPU, 2GB RAM, 40 GB HD, Linux N.A. (Not used as a compute resource) -- Physics, UniMelb fleagle.ph.unimelb.edu.au 1 CPU, 512 MB RAM, 40 GB HD, Linux 2 3 94 CS, University of Adelaide belle.cs.adelaide.edu.au 4 CPU (only 1 available) , 2GB RAM, 40 GB HD, Linux ANU, Canberra belle.anu.edu.au 4 CPU, 2GB RAM, 40 GB HD, Linux 4 Dept of Physics, USyd belle.physics.usyd.edu.au 4 CPU (only 1 available), 2GB RAM, 40 GB HD, Linux 72 VPAC, Melbourne brecca-2.vpac.org 180 node cluster (only head node used), Linux 6 23

Agenda Introduction Global Grids and Challenges Utility Networks and Grid Computing Application Drivers and Various Types of Grid Services Global Grids and Challenges Security, resource management, pricing models, … Service-Oriented Grid Architecture and Gridbus Solutions Market-based Management, GMD, Grid Bank, Aneka Grid Service Broker Architecture, Design and Implementation Performance Evaluation: Experiments in Creation and Deployment of Applications on Global Grids A Case Study in High Energy Physics Summary and Conclusion

Summary and Conclusion Grids exploit synergies that result from cooperation of autonomous entities: Resource sharing, dynamic provisioning, and aggregation at global level Great Science and Great Business! Grids have emerged as enabler for Cyberinfrastructure that powers e-Science and e-Business applications. SOA + Market-based Grid Management = Utility Grids Grids allow users to dynamically lease Grid services at runtime based on their quality, cost, availability, and users QoS requirements. Delivering ICT services as computing utilities. Clouds are rapidly emerging, but more work is required Federation of Clouds, Cloud Exchange, and Application Scaling

Convergence of Competing Paradigms/Communities Needed } ? Web Data Centres Utility Computing Service Computing Grid Computing P2P Computing Cloud Computing Market-Oriented Computing … + Ubiquitous access Reliability Scalability Autonomic Dynamic discovery Composability QoS SLA … Trillion $ business Who will own it? Paradigms Attributes/Capabilities

Thanks for your attention! Are there any Questions? Comments/ Suggestions We Welcome Cooperation in R&D and Business! http:/www.gridbus.org | www.Manjrasoft.com rbuyya@unimelb.edu.au | raj@manjrasoft.com