2 nd December 2002James Magowan - Surrey e-Science Day1 James Magowan IT Specialist Dynamic e-business, IBM UK, Hursley Lab

2 nd December 2002James Magowan - Surrey e-Science Day2 IBM Hursley - Background  3000 Employees  Largest Software Development Lab (of any organisation) outside of the USA  IBM Software Group – Middleware CICS MQ Series Java Technology Centre Advanced Technology & eSolutions 

2 nd December 2002James Magowan - Surrey e-Science Day3 Computing: The Grid Distributed Networking: TCP/IP Information: World Wide Web Communications: Internet Evolution

2 nd December 2002James Magowan - Surrey e-Science Day4  Virtual, collaborative organizations sharing applications and data in an open heterogeneous environment  A vast aggregation of geographically dispersed computing resources Virtual Servers, Storage and Instruments Grid Middleware Distributed Physical Servers and Storage Grid Computing Distributed Computing Over the Internet Using Open Standards

2 nd December 2002James Magowan - Surrey e-Science Day5 Storage Data Applications ProcessingI/OOperating System Microcosm – Pre-Internet “System” Grid Computing

2 nd December Grid Computing....a single unified image Storage Data Applications ProcessingI/OOperating System Macrocosm – Distributed Resources and Applications

2 nd December 2002James Magowan - Surrey e-Science Day7 Motivations for Grid Computing  Increase Capacity Exploit distributed resources to provide capacity for high-demand applications –Existing applications that cannot be run effectively on a single processor –New large scale application that provide strategic business advantages

2 nd December 2002James Magowan - Surrey e-Science Day8 Motivations for Grid Computing  Increase Capacity Exploit distributed resources to provide capacity for high-demand applications  Improve Efficiency / Reduce Costs Reduce infrastructure cost associated with over-provisioned resources Reduce the cost of manpower to manage and configure resources

2 nd December 2002James Magowan - Surrey e-Science Day9 Motivations for Grid Computing  Reduce “Time to Results” Exploit opportunities for parallel computing to allow business critical computation to be completed in a timely fashion Gain competitive advantage by allowing computation to be executed more frequently and on customer demand March 29 March 28 March 27 Serial Execution Parallel Execution

2 nd December 2002James Magowan - Surrey e-Science Day10 Motivations for Grid Computing  Reduce “Time to Results” Exploit opportunities for parallel computing to allow business critical computation to be completed in a timely fashion Gain competitive advantage by allowing computation to be executed more frequently and on customer demand  Enable Collaborations Enable collaboration across applications to integrate results Support large multi-disciplinary collaborations Both within a single organization and between partners Simulation Pricing Design Design Analytics

2 nd December 2002James Magowan - Surrey e-Science Day11  Provide Reliability / Availability Use distributed resources Monitor work progress Restart failed jobs Motivations for Grid Computing Job Scheduler TIMEOUT ! JOB 1 JOB 2 JOB 3 JOB 1 Recovery / Restart

2 nd December 2002James Magowan - Surrey e-Science Day12  Provide Reliability / Availability Use distributed resources Monitor work progress Restart failed jobs  Support Heterogeneous systems Different hardware, system platforms, and available middleware Specialized equipment Motivations for Grid Computing Linux / Z-OS AIX / Linux

2 nd December 2002James Magowan - Surrey e-Science Day13 Uses of Grid Technology 4 Models, Unique Value Propositions Increased Processing “Aggregate processing power from a distributed collection of heterogeneous systems” Customer Values:  Productivity  Flexibility  Resource use  Reliability/ Availability  Complexity  Total cost of ownership Decreased

2 nd December 2002James Magowan - Surrey e-Science Day14 Increased Processing “Aggregate processing power from a distributed collection of heterogeneous systems” Data “Secure access and sharing of distributed data & information in a collaborative fashion” Customer Values:  Productivity  Flexibility  Resource use  Reliability/ Availability  Complexity  Total cost of ownership Decreased Uses of Grid Technology 4 Models, Unique Value Propositions

2 nd December 2002James Magowan - Surrey e-Science Day15 Increased Processing “Aggregate processing power from a distributed collection of heterogeneous systems” Resiliency “Improve the quality of service of distributed systems, despite unplanned events” Data “Secure access and sharing of distributed data & information in a collaborative fashion” Customer Values:  Productivity  Flexibility  Resource use  Reliability/ Availability  Complexity  Total cost of ownership Decreased Uses of Grid Technology 4 Models, Unique Value Propositions

2 nd December 2002James Magowan - Surrey e-Science Day16 Increased Processing “Aggregate processing power from a distributed collection of heterogeneous systems” Resiliency “Improve the quality of service of distributed systems, despite unplanned events” Data “Secure access and sharing of distributed data & information in a collaborative fashion” On Demand “Access data & processing capabilities in a utility-like fashion…….. Make vs. Buy” Customer Values:  Complexity  Total cost of ownership Decreased Uses of Grid Technology 4 Models, Unique Value Propositions  Productivity  Flexibility  Resource use  Reliability/ Availability

2 nd December 2002James Magowan - Surrey e-Science Day17 1. Intra-Grids Grid NAS/SAN Grid NAS/SAN Grid Deployment Options A Function of Business Need, Technology and Organizational Flexibility

2 nd December 2002James Magowan - Surrey e-Science Day18 1. Intra-Grids 2. Extra-Grids Grid NAS/SAN Grid NAS/SAN VPN Grid Deployment Options A Function of Business Need, Technology and Organizational Flexibility

2 nd December 2002James Magowan - Surrey e-Science Day19 1. Intra-Grids 2. Extra-Grids 3. Inter-Grids Grid NAS/SAN Grid NAS/SAN VPN Cactus NTG (SF) Express Project MFG Fin. Services Grid Deployment Options A Function of Business Need, Technology and Organizational Flexibility

2 nd December 2002James Magowan - Surrey e-Science Day20 Grid & Autonomic Computing Self-healing Discover, diagnose, and react to disruptions Self-healing Discover, diagnose, and react to disruptions Self-optimizing Monitor and tune resources automatically Self-optimizing Monitor and tune resources automatically Self-protecting Anticipate, detect, identify, and protect against attacks from anywhere Self-protecting Anticipate, detect, identify, and protect against attacks from anywhere Self-configuring Adapt automatically to the dynamically changing environments Self-configuring Adapt automatically to the dynamically changing environments Self- Configuring Self- Configuring Self- Healing Self- Healing Self- Optimizing Self- Optimizing Self- Protecting Self- Protecting

2 nd December 2002James Magowan - Surrey e-Science Day21 IBM and Grid  GGF - Global Grid Forum Working Groups Approves GRID Open Standards  OGSA - Open Grid Services Architecture Standard proposed by Globus and IBM Reference Implementation (Globus-3 Toolkit)  Open Source Software

2 nd December 2002James Magowan - Surrey e-Science Day22 Applications Middleware Systems Management and Automation Workload / Performance Management Security Availability / Service Management Logical Resource Management Clustering Services Connectivity Management Physical Resource Management OS Open Grid Services Architecture

2 nd December 2002James Magowan - Surrey e-Science Day23 Professional Services Network OGSA Enabled Storage OGSA Enabled Servers OGSA Enabled Messaging OGSA Enabled Directory OGSA Enabled File Systems OGSA Enabled Database OGSA Enabled Workflow OGSA Enabled Security OGSA Enabled General Middleware Web Services OGSI – Open Grid Services Infrastructure Grid Services System Management Sevices Open Grid Services Architecture (OGSA) Applications Autonomic Capabilities Architecture Framework OGSA Structure

2 nd December 2002James Magowan - Surrey e-Science Day24 OGSA Structure – OGSI Architecture Framework Web Services HandleMapNotificationFactory ManagementRegistry LifecycleDiscovery OGSI – Open Grid Services Infrastructure Grid Services System Management Services  Exploits existing web services properties Interface abstraction (WSDL) Protocol, language, hosting platform independence  Enhancement to web services State Management Event Notification Referenceable Handles Lifecycle Management Service Data Extension

2 nd December 2002James Magowan - Surrey e-Science Day25 Architecture Framework Web Services HandleMapNotificationFactory ManagementRegistry LifecycleDiscovery OGSI – Open Grid Services Infrastructure Grid Services System Management Services Open source Reference implementation Hosting platform (Java) Other Possible Hosting Platforms (environments) Microsoft (C#) Globus (C/C++) (Python) OGSA Structure – Open Hosting

2 nd December 2002James Magowan - Surrey e-Science Day26 Web Services Websphere Application Server Architecture Framework HandleMapNotificationFactory ManagementRegistry LifecycleDiscovery OGSI – Open Grid Services Infrastructure Websphere OGSI Service Collections Job Scheduling File Transfer Data Replication Provisioning Logging Problem Determination Resource Management Cluster Management Policy Grid Services System Management Services Value-added Grid Middleware and management applications Tivoli OGSA Structure – Software Evolution Open architecture for “Grid Services” Some “open source” reference implementations Vendor provider “value added” implementations WebSphere

2 nd December 2002James Magowan - Surrey e-Science Day27 European DataGrid

2 nd December 2002James Magowan - Surrey e-Science Day28 European DataGrid  Geographically dispersed people and resources  Petabytes of data per year (100 PB by 2010)  Challenge: Coordinated Use of Distributed computing resources Remote software development and physics analysis Communication and collaboration at a distance

2 nd December 2002James Magowan - Surrey e-Science Day29 OGSA-DAI  Open Grid Services Architecture  Database Access and Integration  Provide access to large scientific databases via the Grid.  Open Source and Standards

2 nd December 2002James Magowan - Surrey e-Science Day30 OGSA-DAI  Data Access and Integration Services XML and Relational Database Management Systems Design for any JDBC-enabled RDBMs Test with Oracle, DB2, MySQL  Languages Java  Handle Large Datasets –Optimised Performance BLOB’s and/or rows Multiple Protocols for Data Transport (SOAP/HTTP and GridFTP)  Globus Globus-2 (GSI & GridFTP) and Globus-3

2 nd December 2002James Magowan - Surrey e-Science Day31 e-Diamond - UK Digital Mammography Archive  Support for Breast Cancer Diagnosis, Treatment, Teaching and Epidemiological Studies  National eScience Centre Initiative

2 nd December 2002James Magowan - Surrey e-Science Day32 e-Diamond - UK Digital Mammography Archive  “One of the pilot e-Science projects is to develop a digital mammography archive, together with an intelligent medical decision support system for breast cancer diagnosis and treatment. An individual hospital will not have supercomputing facilities, but through the Grid it could buy the time it needs. So the surgeon in the operating theatre will be able to pull up a high-resolution mammogram to identify exactly where the tumour can be found”  Tony Blair (speech to the Royal Society – 23 May 2002)

2 nd December 2002James Magowan - Surrey e-Science Day33 James Magowan IT Specialist Dynamic e-business, IBM UK, Hursley Lab