EGC2005 European Grid Conference,Amsterdam, Feb 2005 (Semantic Grid) Services + Semantic (Grid Services) Professor Carole Goble The University of Manchester, UK e-Science North West Regional Centre my Grid, OntoGrid, Knowledge Web GGF Semantic Grid Research Group

EGC2005 European Grid Conference, Amsterdam, Feb 2005 “The ongoing convergence between Grids, Web Services and the Semantic Web is a fundamental step towards the realisation of a common service-oriented architecture empowering people to create, provide, access and use a variety of intelligent services, anywhere, anytime, in a secure, cost-effective and trustworthy way.” Next Generation Grids 2 Requirements and Options for European Grids Research and Beyond EU Expert Group Report July 2004

EGC2005 European Grid Conference, Amsterdam, Feb 2005 “To realise the Next Generation Grid requires semantically rich information representation, the exploitation of knowledge, and co-ordination and orchestration that is aware of context and task” David Snelling, NextGRID Building Intelligent Grid Services

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Knowledge everywhere already…its called metadata State properties of a resource –Data in a purchase order –Current usage agreement for resources on a grid –Metrics associated with work load or performance on a Web server Declarative descriptions of data sets, codes, services, workflows –Typing and classifying service or workflow inputs, outputs, goals, … –Access rights to resources Declarative descriptions for, and records of, service interactions –event notification topics, provenance trails, monitoring records Policy and profile encoding –personal profiles and security groupings Used in –job control; workflow composition, semantic dataset integration, resource brokering, resource scheduling, problem solving selection, intelligent portals… GGF WG-CMM, CIM, GIS, MDS, ….

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Knowledge and the knowledge producing & consuming protocols & patterns are already in Grid Middleware and Grid Applications. Embedded in middleware code, in schemas, in catalogues, in applications and in practice.

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Bringing knowledge into the light Managing and operating a Grid intelligently requires: 1. Knowledge –Knowledge about the state and properties of Grid components, and their configurations –Mechanisms for interpreting that knowledge 2. Intelligently acquiring and refreshing knowledge 3. Use it practically in decision making.

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Convergence Semantic Web Technologies Semantic Web itself Grid services Semantic Web Services Semantics for the Grid Grid services for Semantic Web Plumbers Developers Web Services Grid Semantic Web and Agents Semantic Grid Engineers Aesthetics Theoreticians

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Semantic Web mechanisms Metadata Annotation RDF Ontologies OWL/RDFS Web XML, URI, UniCode Deep web PHP, WS* Rules SWRL  p ->  a; p=a Trust Search engines and filters Applications Uniform naming scheme. Metadata – descriptions of properties and content Metadata – glue linking resources together Ontologies – interpretation of metadata for people and processes.

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Making Knowledge Explicit OWL Web Ontology Language RDF Resource Description Framework

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Make knowledge explicit. Make knowledge protocols explicit. Describe some of these declaratively so they might be exchanged and machine processed. Metadata data – here is what it is and/or how it relates to something else Ontologies / controlled vocabularies – we understand each other Ontology Metadata assertion Object

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Knowledge Stakeholders Computer Scientists Scientific Applications Grid Middleware Grid platform and resources Security policies standards Scientists Service Providers Knowledge for Grid Applications Knowledge for the operation of the Grid Sources of Knowledge

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Upper domain generic services Web Service Resource Framework Web Service-Notification WS-I+ Web Services Grid Domain Applications Collective services Base services System services “Plumbing” Operational Knowledge Application Knowledge knowledge worker's applications and tools

EGC2005 European Grid Conference, Amsterdam, Feb 2005 The Semantic Grid is an extension of the current Grid in which information and services are given well-defined and explicitly represented meaning, better enabling computers and people to work in cooperation Semantics in and on the Grid

EGC2005 European Grid Conference,Amsterdam, Feb 2005 Time to move beyond slogans.

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Semantic Grid roadmap Exploit the languages from the Semantic Web and other. Specifying and developing the architectural components and tools forming the infrastructure of the Semantic Grid and define the architecture of the (Semantic) Grid. Prototyping applications using the languages, the components and defining the content necessary. Developing in parallel, yet are interdependent. A maelstrom of research coupled concurrently with standards activity, and early experiments and prototypes running alongside (some) commercial developments.

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Combe Chem Semantic Grid trajectory Time Efforts Implicit Semantics 1 st generation SRB Implicit Semantics OGSA generation GGF Semantic Grid Research Group Many workshops Systematic Investigation Phase Specific experiments Part of the Architecture Dagstuhl Schloss Seminar Grid Resource Ontology Many projects Pioneering Phase Ad-hoc experiments, early pioneers SDK Demonstration Phase

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Knowledge Aware Grid Services KAGS Knowledge Aware Grid Services KAGS Grid Compliant Knowledge Services GCKS Grid Compliant Knowledge Services GCKS Grid Aware Knowledge Services GAKS Grid Aware Knowledge Services GAKS Three strands (Semantic Grid) Services Semantic (Grid Services) And how all these services play together Profiles, Protocols, Patterns, Policies P4

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Middleware Functionality: Existing operations for interaction with a knowledge service Metadata: How fast? What language is supported? Lifetime Management: Factory methods, creation of resources Knowledge: Additional port types relating to knowledge, for example discovery. Use of Grid infrastructure within the implementation of the service. Three strands Knowledge Aware Grid Services KAGS Knowledge Aware Grid Services KAGS Grid Compliant Knowledge Services GCKS Grid Compliant Knowledge Services GCKS Grid Aware Knowledge Services GAKS Grid Aware Knowledge Services GAKS

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Grid Compliant Knowledge Services Take today’s knowledge services from the Semantic web and other worlds What does it mean for them to be Grid Services? What are the state properties of an ontology grid service? What are the lifetime management properties of an ontology grid service? What is a virtualised and dynamically provisioned ontology service, (metadata store, metadata annotator, reasoner …) ? How will an ontology grid service and a metadata grid service play together?

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Grid Compliant Ontologies Resource A distinguishable unique identity and lifetime (usually static) Maintains a specific state that can be materialized May be accessed through one or more Web Services Artifact - a file, XML document, database, usually real (could be virtual). Could be compound. Service Base interface for inspecting and manipulating an ontology A well defined “Ask-Tell” API: getSubConcepts(concept), getSuperConcepts(concept), classify, checkSatisfiability(concept), put(conceptExpression) … Resource – a connection to the Ontology Service An ontology might be just a file. Or an application. Or embedded in an application after a community has thought about it for a bit.

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Ontology as an OGSA-DAI Realization WS-DAI Message Patterns Behavioural Properties WS-DAIO Ontology WS-DAIX XML WS-DAIR Relational WS-DAIO-RDF RDF Specific WS-DAIO-OWL OWL specific Provide a realization of WS-DAI with specific ontology messages (activities)

EGC2005 European Grid Conference, Amsterdam, Feb 2005 RDF Annotation store as an OGSA-DAI Realization WS-DAI Message Patterns Behavioural Properties WS-RDFWS-DAIX XML WS-DAIR Relational JenaSesame DB2mySQL Provide a realization of WS-DAI for RDF

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Data -> Ontology Access Data Access collects together messages that access and/or modify a resource Note: the messages are ignorant of the query other than its class. OSGA-DIAO –The message patterns & the behavioural properties –The API for the ontology querying –The realisation mapping to the ontology language – OWL, RDF, RDFS, DAG

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Knowledge Aware Grid Services Take a Grid service and see how it might take advantage of a knowledge service or knowledge resource. Might be a base Grid service or an Application Service or a high level Grid service. What are the generic and specific knowledge services required for Grid? Two starting points: –Discovery. Registry/Brokering – shared semantics; resource annotation; painless knowledge recovery. –Debugging – shared semantics; knowledge collection; knowledge recovery.

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Semantic Web – describing data Semantic Web Services – describing processes. –WSMO, OWL-S Web Service API Web Interface Web Service API Generic Schema for Web (Grid) Services Specific Application Ontology Semantic Web Services Thierry’s observations about Web Service abstractions

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Discovery in Taverna workflow workbench Taverna currently ships with access to >1000 publicly available bioinformatics services Bioinformatican chooses services when forming workflows, with assistance. A common ontology is used to annotate and query any my Grid object including services. Discover workflows and services described in the registry via Taverna. Look for all workflows that accept an input of semantic type “nucleotide sequence”

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Low level descriptions WSDL, Scufl Feta importer UDDI registry my Grid domain classification Reasoner PeDRo annotator Feta semantic discovery engine Feta GUI Taverna workbench clients KAVE provenance Ontology editor Annotator Resource match make Knowledge Engineer Ontologist builds myGrid Domain Ontology Feta skeletons generated by mining low level descriptions Skeletal descriptions are annotated Descriptions are loaded and engine initiated Search requests User interacts with GUI to discover resources Semantic Discovery Annotated descriptions are stored Metadata discovery Ontology Acquisition Resource discovery

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Intelligent Debugging Architecture Acklin

EGC2005 European Grid Conference, Amsterdam, Feb AC Homo sapiens BAC clone CTA-315H11 from 7, complete sequence AC Homo sapiens BAC clone RP11-622P13 from 7, complete sequence AL Human DNA sequence from clone RP11-553N16 on chromosome 1, complete sequence AL Homo sapiens chromosome 21 segment HS21C AL Human chromosome 14 DNA sequence BAC R-775G15 of library RPCI-11 from chromosome 14 of Homo sapiens (Human), complete sequence BX Homo sapiens mRNA; cDNA DKFZp686G08119 (from clone DKFZp686G08119) AC Homo sapiens 12q22 BAC RPCI11-256L6 (Roswell Park Cancer Institute Human BAC Library) complete sequence AK Homo sapiens cDNA FLJ45040 fis, clone BRAWH AC Homo sapiens chromosome 17, clone RP11-104J23, complete sequence AL Human DNA sequence from clone RP4-715N11 on chromosome 20q Contains two putative novel genes, ESTs, STSs and GSSs, complete sequence AC Homo sapiens BAC clone RP11-731I19 from 2, complete sequence AC Homo sapiens chromosome 15, clone RP11-342M21, complete sequence AL Human DNA sequence from clone RP11-461K13 on chromosome 10, complete sequence AC Homo sapiens PAC clone RP3-368G6 from X, complete sequence AC Homo sapiens chromosome 4 clone B200N5 map 4q25, complete sequence AF Homo sapiens chromosome 21q22.3 PAC 171F15, complete sequence >gi| |gb|AC | Homo sapiens BAC clone CTA-315H11 from 7, complete sequence AAGCTTTTCTGGCACTGTTTCCTTCTT CCTGATAACCAGAGAAGGAAAAGATC TCCATTTTACAGATGAG GAAACAGGCTCAGAGAGGTCAAGGCT CTGGCTCAAGGTCACACAGCCTGGGA ACGGCAAAGCTGATATTC AAACCCAAGCATCTTGGCTCCAAAGC CCTGGTTTCTGTTCCCACTACTGTCAG TGACCTTGGCAAGCCCT GTCCTCCTCCGGGCTTCACTCTGCAC ACCTGTAACCTGGGGTTAAATGGGCT CACCTGGACTGTTGAGCG urn:lsid:taverna:datathing:15..BLAST_Report rdf:type urn:lsid:taverna:datathing:13..similar_sequences_to.. nucleotide_sequence rdf:type service invocation..created_by workflow invocation workflow definition experiment definition project person group service description organisation..described_by..run_during..invocation_of..part_of..works_for..part_of..author..run_for AB..masked_sequence_of..filtered_version_of Relationship BLAST report has with other Other classes of information related to BLAST report Keeping track Jun Zhao, Chris Wroe, Carole Goble, Robert Stevens, Dennis Quan, Mark Greenwood, Using Semantic Web Technologies for Representing e-Science Provenance in Proc 3 rd International Semantic Web Conference, Hiroshima, Japan, Nov 2004

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Grid Aware Knowledge Services What is the architecture of distributed knowledge services? Can Grid platforms realistically provide a robust distributed stateful computing platform for agent systems? OGSA-DAIS for RDF repositories. Replica location service for replicated knowledge services. Secure file transfer for metadata. Event notification for metadata or ontology updates. Authentication and authorisation for updates. Metadata updated by workflows; Security and RDF! Distributed reasoning !! Depends on the availability of these Grid services.

EGC2005 European Grid Conference, Amsterdam, Feb 2005 WS-Notification and Semantic Integrity Subscriber – an Annotation Service - indicates interest in a particular (semantic) topic – Ontology Version change - by issuing a subscribe request Subscriptions are WS-Resources –Various subscriptions are possible Notification may be triggered by: –WS Resource Property value changes –Other “situations” Broker examines current subscriptions Brokers may –“Transform” or “interpret” topics <- knowledge! Broker Subscriber Publisher subscribe SS S notify Metadata service Ontology Service Adapted from Dr. Daniel Sabbah, IBM, Globus World 2004.

EGC2005 European Grid Conference, Amsterdam, Feb 2005 resource discovery, intelligent debugging, provenance mining Car repair settlement, satellite data configuration. Grid Application and Application Services OGSA plumbing services OGSA OntoKit semantic grid services OGSA OntoKit knowledge Generation services Base services: annotation management ontology access and integration, annotation access, reasoning, ontology alignment GRID PROPERTIES Resources Resources: Ontology, Knowledge Base, Registry, Database DOMAIN & MIDDLEWARE OGSA OntoKit plumbing services Patterns & Upper Services: Semantic broker, semantic registry, semantic logging, semantic workflow management, vocabulary management PATTERNS OF INTERACTION Yet Another Stack

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Obstacles to Overcome Semantic what? Compelling use cases –“Revolution is only possible when it becomes inevitable” –Niche activity. No content or hard to get the content! –Ontology acquisition. Pain-free metadata acquisition. Baggage of communities –Different agendas –Hendler Principle: “A little semantics goes a long way”. –Failure to mainstream – agents Instability of both platforms –Middleware hard to use and incomplete –Off putting to “the other side” –Deployment, research, development, applications and standardisation all happening together Whither Grid Architecture?

EGC2005 European Grid Conference, Amsterdam, Feb 2005 MDA and the Grid Where is grid? –current grids are on a platform level –grids compatible with service oriented architectures are on ASM level Challenge: –should grids do better than SOA based on Web Services? –automatic transformation of PIM models into a grid specific ASMs and PSMs Opportunity: –transform a business level architectures to Web Services, Grid, whatever- comes-next platform Computation Independent Model Platform Independent Model Architecture Specific Model Platform Specific Model working system e.g. OGSA e.g. GT4, gLite manual automatic semi automatic Prof.dr. Žiga Turk

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Map concepts between ontologies Unicore and GLUE have different philosophies for describing resources :-( In Unicore, the resources are described in terms of resource requests In GLUE, resources are described in terms of the availability of resources.

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Use Assertion Implicit Explicit Shared human consensus Text descriptions Type systemsSchemata Ontologies Rules Non-embedded metadata Embedded metadata Not all knowledge will use separate services

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Source of metadata and knowledge Grid Resource Ontology Activation Energy Metadata mining The network effect – service providers rule Return on investment for service providers and users Applications keep it real: listen to users to take short cuts.

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Semantic proportions speculation – no empirical foundation at all Resource Generic Grid Application

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Knowledge aware grid services GridKnowledge, Agents & the Semantic Web Overcoming community divisions Growing pains of middleware Make it easier not harder or more “interesting” A little semantics goes a long way Evolution not revolution Technology push

EGC2005 European Grid Conference, Amsterdam, Feb 2005 “WSRF is the instruction set of the Grid” Thierry Priol WSRFWS-I+ Grid service behaviour Semantic Grid Services

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Whither Grid Architecture?

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Applications Use Cases Semantic Grid Architecture Grid Architecture Semantic Architecture InteliGrids ProvenanceSIMDAT NextGRID WSRF UniGrids WS-I+ K-WfGrid SDK

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Summary What existing technologies can we harness and what needs to be done that is new? Semantic SOA – what are the resources, services, profiles, patterns and policies? What are the appropriate abstractions for a Semantic Grid based architecture? (or a Grid Architecture?) How will semantics make the Grid more flexible and simpler – and how do we avoid making it more complicated! How do we ensure close cooperation with design and development of next generation Grid research and next generation knowledge research?

EGC2005 European Grid Conference, Amsterdam, Feb 2005 Thanks my Grid consortium, esp. Phil Lord, Pinar Alper, Chris Wroe, Luc Moreau OntoGrid project members Norman Paton, OGSA-DAI Prof.dr. Žiga Turk, InteliGrids John Brooke, UniGrids Stephane Viali Thierry Pioli, CoreGrid David de Roure, GGF Sem-Grd RG