The Grid, Grid Services and the Semantic Web: Technologies and Opportunities Dr. Carl Kesselman Director Center for Grid Technologies Information Sciences Institute University of Southern California

Outline l What are Grids? l Grid technology -Globus and the Open Grid Services Architecture l Grids and the Semantic Web

How do we solve problems? l Communities committed to common goals -Virtual organizations l Teams with heterogeneous members & capabilities l Distributed geographically and politically -No location/organization possesses all required skills and resources l Adapt as a function of the situation -Adjust membership, reallocate responsibilities, renegotiate resources

The Grid Vision “ Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations” -On-demand, ubiquitous access to computing, data, and services -New capabilities constructed dynamically and transparently from distributed services “When the network is as fast as the computer's internal links, the machine disintegrates across the net into a set of special purpose appliances” (George Gilder)

Biomedical Informatics Research Network (BIRN) l Evolving reference set of brains provides essential data for developing therapies for neurological disorders (multiple sclerosis, Alzheimer’s, etc.). l Today -One lab, small patient base -4 TB collection l Tomorrow -10s of collaborating labs -Larger population sample -400 TB data collection: more brains, higher resolution -Multiple scale data integration and analysis

National Virtual Observatory Xray (ROSAT) theme Change scale Change theme from Caltech CACR Caltech Astronomy Microsoft Research Optical (DPOSS) Coma cluster Virtual Sky has 140,000,000 tiles 140 Gbyte

Living in an Exponential World (1) Computing & Sensors Moore’s Law: transistor count doubles each 18 months Magnetohydro- dynamics star formation

Living in an Exponential World: (2) Storage l Storage density doubles every 12 months l Dramatic growth in online data (1 petabyte = 1000 terabyte = 1,000,000 gigabyte) -2000~0.5 petabyte -2005~10 petabytes -2010~100 petabytes -2015~1000 petabytes? l Transforming entire disciplines in physical and, increasingly, biological sciences; humanities next?

An Exponential World: (3) Networks (Or, Coefficients Matter …) l Network vs. computer performance -Computer speed doubles every 18 months -Network speed doubles every 9 months -Difference = order of magnitude per 5 years l 1986 to Computers: x 500 -Networks: x 340,000 l 2001 to Computers: x 60 -Networks: x 4000 Moore’s Law vs. storage improvements vs. optical improvements. Graph from Scientific American (Jan- 2001) by Cleo Vilett, source Vined Khoslan, Kleiner, Caufield and Perkins.

The Grid World: Current Status l Dozens of major Grid projects in scientific & technical computing/research & education l Considerable consensus on key concepts and technologies -Open source Globus Toolkit™ a de facto standard for major protocols & services -Far from complete or perfect, but out there, evolving rapidly, and large tool/user base l Industrial interest emerging rapidly l Opportunity: convergence of eScience and eBusiness requirements & technologies

Globus Toolkit l Globus Toolkit is the source of many of the protocols described in “Grid architecture” l Adopted by almost all major Grid projects worldwide as a source of infrastructure l Open source, open architecture framework encourages community development l Active R&D program continues to move technology forward l Developers at ANL, USC/ISI, NCSA, LBNL, and other institutions

User process #1 Proxy Authenticate & create proxy credential GSI (Grid Security Infrastruc- ture) Gatekeeper (factory) Reliable remote invocation GRAM (Grid Resource Allocation & Management) Reporter (registry + discovery) User process #2 Proxy #2 Create process Register The Globus Toolkit in One Slide l Grid protocols (GSI, GRAM, …) enable resource sharing within virtual orgs; toolkit provides reference implementation ( = Globus Toolkit services) l Protocols (and APIs) enable other tools and services for membership, discovery, data mgmt, workflow, … Other service (e.g. GridFTP) Other GSI- authenticated remote service requests GIIS: Grid Information Index Server (discovery) MDS-2 (Meta Directory Service) Soft state registration; enquiry

The Next Step l Globus leverages standard protocols -TLS, LDAP, X.509, HTTP -Only TCP in common l Is there a better foundation for Grid functions -More unified protocol stack (common base) -Better support for virtualization -Leverage commodity infrastructure

“Web Services” l Increasingly popular standards-based framework for accessing network applications -W3C standardization; Microsoft, IBM, Sun, others l WSDL: Web Services Description Language -Interface Definition Language for Web services l SOAP: Simple Object Access Protocol -XML-based RPC protocol; common WSDL target l WS-Inspection -Conventions for locating service descriptions l UDDI: Universal Desc., Discovery, & Integration -Directory for Web services

Transient Service Instances l “Web services” address discovery & invocation of persistent services -Interface to persistent state of entire enterprise l In Grids, must also support transient service instances, created/destroyed dynamically -Interfaces to the states of distributed activities -E.g. workflow, video conf., dist. data analysis l Significant implications for how services are managed, named, discovered, and used -In fact, much of our work is concerned with the management of service instances

OGSA Design Principles l Service orientation to virtualize resources -Everything is a service l From Web services -Standard interface definition mechanisms: multiple protocol bindings, local/remote transparency l From Grids -Service semantics, reliability and security models -Lifecycle management, discovery, other services l Multiple “hosting environments” -C, J2EE,.NET, …

The Grid Service = Interfaces + Service Data Service data element Service data element Service data element GridService… other interfaces … Implementation Service data access Explicit destruction Soft-state lifetime Notification Authorization Service creation Service registry Manageability Concurrency Reliable invocation Authentication Hosting environment/runtime (“C”, J2EE,.NET, …)

Given a set of Services? l How do we do a better job of finding out what services we want to use l How do we do a better job of configuring services l How do we do a better job of composing and nesting services l Answer: Do a better job of representing services

Deeper representation of services l Information is captured via structure -X.509 certificates, MDS models, CIM schema, Metadata l Knowledge expresses relationships between entities -Concepts and relationships -Logical framework to inference over relationships

Vision “The Semantic Web is an extension of the current Web in which information is given a well-defined meaning, better enabling computers and people to work in cooperation. It is the idea of having data on the Web defined and linked in a way that it can be used for more effective discovery, automation, integration and reuse across various applications. The Web can reach its full potential if it becomes a place where data can be processed by automated tools as well as people” From the W3C Semantic Web Activity statement

Resource Description Framework

Attributes Semantics Knowledge Information Data Ingest Services ManagementAccess Services (Model-based Access) (Data Handling System ) MCAT/HDF Grids XML DTD SDLIP Information Repository XTM DTD Rules - KQL Attribute- based Query Fields Containers Folders Storage (Replicas, Persistent IDs) Feature-based Query Relationship s Between Concepts Knowledge Repository for Rules Knowledge or Topic-Based Query / Browse Knowledge-based Data Grid Roadmap (Regan Moore: SDSC)

Ontologies Everywhere l What happens if knowledge permeates the Grid -Data elements -Service descriptions (service data elements) - Protocols (e.g. policy, provisioning) l More dynamic and general model then Semantic Web -OGSA lifetime model -OGSA SDE model

Cognative Grid l Grid Services + Ontologies + Knowledge Driven Services l Examples -Knowledge driven matchmaking -Agent based service composition -High-level planning and resource discovery -Knowledge based provisioning l Some people are using term “semantic grid” to discribe Grid Services+Knowlege

SCEC Modeling Environment Knowledge Base Ontologies Curated taxonomies, Relations & constraints Pathway Models Pathway templates, Models of simulation codes Code Repositories Data & Simulation Products Data Collections FSM RDM AWM SRM Storage GRID Pathway Execution Policy, Data ingest, Repository access Grid Services Compute & storage management, Security DIGITAL LIBRARIES Navigation & Queries Versioning, Topic maps Mediated Collections Federated access KNOWLEDGE ACQUISITION Acquisition Interfaces Dialog planning, Pathway construction strategies Pathway Assembly Template instantiation, Resource selection, Constraint checking KNOWLEDGE REPRESENTATION & REASONING Knowledge Server Knowledge base access, Inference Translation Services Syntactic & semantic translation Pathway Instantiations Computing Users

DOCKER: Publishing SHA Code SCEC ontologies AS97 msg types AS97 ontology constrs docs User specifies: l Types of model parameters l Format of input messages l Documentation l Constraints User Interface Constraint Acquisition Model Specification DOCKER Web Browser Wrapper Generation (WSDL, PWL) AS97 (Y. Gil, USC/ISI)

Recommends other models Yes Did you know that [Sadigh97] is a good model for dist >80 miles?

Automatically Generates Interface

Automatically Generates KR Description

myGrid Project - bioinformatics l Imminent ‘deluge’ of genomics data -Highly heterogeneous, Highly complex and inter-related l Convergence of data and literature archives 1.Database access from the Grid 2.Process enactment on the Grid 3.Personalisation services 4.Metadata services Grid Services + Ontologies Carol Gobel, U. Manchester

Resource selection: Matchmaking l Providers and requesters describe themselves -Synactic description >Structured or Semi-structured l A Matchmaker matches compatible classads -Match based on attribute name, simple prioritization l Semantic matchmaking -Inference based matching (e.g. CIM+relations) -Automatic classification (e.g. description logic) -Leverage domain specific ontologies

Pegasus: Planning for Execution in Grids l Create workflow to create virtual data -Domain specific and generic rules l Map Workflow unto Grid resources -System state via Grid services (MDS, RLS,…) -Global and local optimization criteria

Summary l Technology exponentials are changing the shape of scientific investigation & knowledge -More computing, even more data, yet more networking l The Grid: Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations l Many potential opportunities for application of semantic web technologies to Grid services -OGSA

Partial Acknowledgements l Open Grid Services Architecture design -Karl USC/ISI -Ian Foster, Steve -Jeff Nick, Steve Graham, Jeff IBM l Semantic/Cognitive Grid -Yolanda Gil, Ewa Deelman, Jim Blythe, Tom Russ, Hans Chalupsky -Conversations with Jim Hendler, Carol Gobel, David DeRoure l Strong links with many EU, UK, US Grid projects l Support from DOE, NASA, NSF, Microsoft

For More Information l Grid Book - l The Globus Project™ - l OGSA - l Global Grid Forum -