Proprietary Data Services and Ontology Driven Applications (ODA) 2nd SOA for E-Government Conference October 2006 Presented by: Atif Kureishy October 31, 2006

2 Service Oriented Architectures (SOA) Benefits of the SOA Architectural Model Benefits Features Platform and language independent Leverage existing IT investments Standards Based and Highly Interoperable Plug and play components independent of platform Enable redundancy for survivability Emphasis on business logic and less on plumbing Loosely Coupled and Decentralized Discoverable Faster decision cycles Increase cross-organizational information visibility Enterprise agility Promote reuse of infrastructure services Promote development of discrete, modular functions for reuse Promote reuse of services, not reuse of code Reusable

3 Web Services is not the complete answer Standards Based and Highly Interoperable Loosely Coupled and Decentralized Discoverable Reusable Web Services and their related standards (SOAP, WSDL, UDDI, WS-*) provides an implementation framework for several key features of SOA Web Services technologies do not provide all the requirements for Dynamic USE of Discoverable Services Discovery – Yes – UDDI Use – No – requires service consumers and providers to agree on a pre-defined standard interface for the service Service Oriented Architectures (SOA)

4 Approaches for Dynamic use of Discoverable Services The ability to dynamically use a newly discovered services is enabled by one of two general approaches –Service Mediation –Service Polymorphism Service Mediation is a process for translating a service definition to a “known” interface –Functionality typically provided by ESBs –Still requires some level of service integration Service Polymorphism –Using the same methods for different service instances that work across different domains –Requires advanced agreement to the Polymorphic specification, but then enables truly dynamic service integration –Polymorphism is very relevant for specific types of service, including a “Data Service”

5 Anatomy of a Data Service Definition: Data Services are a type (class) of service specifically built to ask questions about and manipulate data Can utilize a polymorphic approach if the following design principles are followed: –Separation of Service Spec (methods) from the Data Types being services (Ontology) –Generalized (Polymorphic) Data Service Methods Ontology & Introspection interfaces - understand the data provided by a data service instance Search, Retrieve, and Modify interfaces - (CRUD)

6 Dynamic Data Service Consumption enabled by Ontology Driven Applications Once Data Services can be defined in a domain independent polymorphic construct, it is possible to develop Ontology Driven Applications –Adapted from Model Driven Architectures (MDA) concepts, but not necessarily UML based –Appropriate for a specific sub-class of applications: search, forms, reports, business intelligence, visualization (is not the silver bullet for every data oriented application) –Enabled via Ontology/Taxonomy representations (RDF/OWL/XSD) and service exposure of Ontology/Taxonomy introspection methods

7 The need for information sharing/disseminating systems are becoming a vital part of inter agency communication, intelligence analysis, and making mission critical decisions There are several key challenges in building information sharing systems: –Many disparate data sources, and no definition of how one may be related or associated with another –No common data model or information model has been defined or established –Varying levels of security classifications –Varying forms of data (structured, semi-structured, un-structured) –Robust sets of links, associations, and relationships within and across data sources –New data sources continue to be implemented Therefore, in order to make the greatest impact, information sharing systems must: –Separate the management and aggregation of data from the applications and/or services that use the data –Establish a common Information Model that applies to all the aggregated content –Provide a unified or normalized view (semantic mediation) of all the underlying data sources –Efficiently aggregate data from disparate sources –Provide the ability to easily and readily add on and incorporate new data sources –Capture relationships and associations across the different data sources to generate federated queries –Allow for the secure dissemination of the content stored to be leveraged by external mission systems

8 A key factor in designing such a ODA is to define and establish a comprehensive Information Model A comprehensive Information Model requires two components: –A robust Data Model that represents the data itself –A Metadata Model that provides the necessary information to drive the applications/systems Building the comprehensive Information Model involves: –Selecting a technology to define your model (XML Schema, RDF, OWL, …) –Leveraging a layered OO approach in defining your model –Defining applicable metadata information about the data in your model in order to make it easier to consume and interpret the data represented –Capturing relationship and link information using RDF, OWL or an RDF type approach –Aggregating and normalizing common information from disparate data sources

9 A good Data Model must have a solid base model A solid abstract base model serves as a core foundation for the rest of your data model, which can be leveraged for any new data that may get incorporated into the system Leverage OO principles of Inheritance, Polymorphism, and Data Encapsulation –Create root abstract data types that serve as base types for all other data types –Should be generic and not domain specific –Must be robust enough to support model abstractions such as inheritance, relationships (peer to peer and hierarchical), associations, and cardinality Resource Information Object ComponentRelationship inherits Field contains relates inherits

10 Defining your layered data model Base Model: Definition of your abstract InformationObjectType, ComponentType, and RelationshipType Data Source Model: Aggregation of the data source building blocks that establishes the Information Objects, their Relationships, and their Components Domain Model: Definition of the common fields, components, and relationships across all data sources Data Source Building Blocks Model: Definition of the fields, components, and relationships that apply to each data source

11 Defining your Metadata Model The metadata model is what provides semantic meaning to the data being represented and is used to drive your application or system –Does not need to be a part of the data output, but rather used internally by the system Define your metadata model for each base model type, InformationObject, Component, Field, and Relationship, which could include, but is not limited to: –Categorization information –Classification level –Data source name and/or provider –Formatting –Query information –Relationship information –What are key fields Essentially, any important piece of information that is not the data itself would be defined in this metadata model

12 Realizing relationships and links between data types in your Information Model Each relationship is captured as a relation between two InformationObjects, one is the subject, and the other the object The relationships themselves may have attributes that need to be captured or quantified within your relationship model definition The information about the relationship should be defined in you metadata model for the relationship type

13 A representative abstract base model Information Objects are defined using three basic concepts –The Information Object: Contains a set of fields and components to describe its properties and attributes –Field: A simple property or attribute of an object –Component: A set of fields that can be nested and repeating to describe more complex object properties or attributes –Relationship: Establishes a relationship between any two Information Objects including itself Information Object Component Relationship Information Object Field 1 Field Field N

14 Illustrative Data Source Example Information Object Types Fields and Components RelationshipRelated Object FacilityFields: Name, Location, etc. Components: Facility Site UsesCommsLinkCommsLink IsOnRouteRoute CommunicatesWithFacility RouteFields: Name, etc.RunsThroughFacility IsComposedOfCommsLink Fields: Name, nature, etc. Component: Media Type ConnectsFacility IsPartOfRouteRoute

15 Graphical Representation of an Information Object

16 Example User Interface Metadata Model Field Name Name Other Text Identification false

17 Enhance the usability of your model by incorporating a query and result set model In order to support dynamic query building that is not tied to your data, define a query model that applies to your defined base model and leverages information captured in your metadata model Include definitions in your metadata model to support querying capabilities –For example, include a queryable attribute in your field metadata model to provide information on what fields may be queryable Include definitions to capture different query types supported, input parameters, output result sets, … Establish definitions for the different types of data transfer objects that may be used by the query interface By defining a model that captures all parts of the query or data access interface, the system can more readily be extended to support a web services interface

18 Example ODA Architecture that leverages your Information Model Let the models drive your architecture through introspection, more specifically, the metadata models Support dynamic queries by designing a data access layer that understand how to query the data by interrogating the metadata model Leverage an Data Access Service (EII) tool that can utilize your model to serve as a data aggregation and querying layer Support extensibility and the addition of new content through changes in the model, without affecting the rest of the architecture

19 Extend your ODA to a Service Oriented Architecture by leveraging Web Services SOA provides: –Exposes enterprise assets as services that creates a network of services –Services can be discovered and consumed in a secure manner across organizational boundaries –Standards and specifications such as UDDI, WSDL, SOAP, and WSRP promote interoperability, application reuse, loosely-coupled components leading to the commoditization of these services A SOA based on Web Services utilizes standards to increase the availability of enterprise assets and reduce dependency on proprietary vendor integration schemes

20 Summary – Implementing an Information Sharing Architecture Growing need for data management, aggregation, and dissemination systems These systems need to remain flexible, adaptable, and extensible to support the dynamic content being handled An architecture driven by an ontology can achieve this; a key part of the models is that the metadata is defined and captured There are emerging technologies and standards to help support defining an Information Model An architecture driven by XML, RDF, or OWL models can also be more easily extended to support a Service Oriented Architecture In order to provide a SOA to meet the dissemination challenges of data management systems, web services are an ideal technical solution to securely share information across organizational boundaries