Red Sky Data Inc.1 Presentation # 402 Steve Jones, David Stanford Red Sky Data Inc. Meta Data – The Final Frontier IOUG Live! 2004

Red Sky Data Inc.2 Welcome A little about us A little about you Welcome to Toronto

Red Sky Data Inc.3 Weather Forecast

Red Sky Data Inc.4 Weather Forecast

Red Sky Data Inc.5 Weather Forecast

Red Sky Data Inc.6 Objectives Gain an in-depth understanding of meta data issues and requirements (in the context of data warehousing) Gain an appreciation for Oracle’s solution to meta data Provide a real world, implemented solution (a starting point) for your meta data solution

Red Sky Data Inc.7 Agenda Meta data – What Is It? Overall Requirements Common Warehouse Metamodel (CWM) Vendor Tools Today Oracle’s Meta Data Solution Shortcomings in the Marketplace Meta Data Design – A Sample Model Security in Meta Data Wrap Up

Red Sky Data Inc.8 Meta Data – What is it? More than “data about data” Collection of information describing the contents of the data warehouse environment Used to maximize the value from the data warehouse – both technical and business

Red Sky Data Inc.9 Benefits of Meta Data Improved data consistency Reduced data redundancy Easier re-use of data and applications that use it Enhanced self-service for a broader range of enterprise users

Red Sky Data Inc.10 Benefits of Meta Data Lower cost development, implementation, and maintenance of new applications and processes Personalized information Faster, smoother implementation of third-party applications and tools

Red Sky Data Inc.11 Categories of Meta Data  Data load statistics  Source and target table/column information  Transformation logic  Obtained from database catalogs, tool repositories, etc. Technical Meta Data

Red Sky Data Inc.12 Categories of Meta Data Business rules Source system names and descriptions Contact names Entity/attribute names and descriptions Report names and descriptions Obtained from business users Business Meta Data

Red Sky Data Inc.13 End to End Requirements Meta Data

Red Sky Data Inc.14 Essential Capabilities Collection Capture information from various sources Technical information (e.g. names, datatypes) Business information (e.g. definitions, rules) Transformation logic Data quality characteristics

Red Sky Data Inc.15 Available to all warehouse users Web-based Simple, intuitive interface Single repository for all meta data Third normal form Automated capture process Essential Capabilities Design Deployment

Red Sky Data Inc.16 Restrict access to sensitive information User/group-based Table-based May require row-level security (rare) Look for occurrences of keywords Results by data warehouse area Essential Capabilities Security Search

Red Sky Data Inc.17 Common Warehouse Metamodel (CWM) Specification that describes meta data interchange among data warehousing, business intelligence, knowledge management and portal technologies Product vendors build adaptors that facilitate the interchange Standard interfaces, protocols, language XML-based

Red Sky Data Inc.18 Meeting End to End Requirements CWM Metamodel (shared meta data) CWM Meta Data Interchange Meta Data Repository

Red Sky Data Inc.19 CWM Timeline * MDC releases OIM v1.0

Red Sky Data Inc.20 Tool Marketplace Remains immature Many tools are vendor-specific Lacks standards, although progress is being made Tools good at technical meta data, not so good at business meta data

Red Sky Data Inc.21 Selected Vendor Solutions Major players include: Computer Associates – Advantage Repository Data Advantage Group – MetaCenter IBM – Information Catalog Manager Informatica – Informatica SuperGlue Microsoft – Meta Data Services Oracle – Oracle Repository Many others...

Red Sky Data Inc.22 Data Advantage Group Primary focus is meta data Flagship product is MetaCenter Analytical data management platform Capture, analyze and manage meta data Tailorable to business needs Seemless integration with development and deployment tools (ETL, BI, Case)

Red Sky Data Inc.23 Meta Integration Model Bridge (MIMB) Standalone tool from Meta Integration Technology Inc. Allows meta data to be exchanged, or “bridged” between products (design tools, meta data repositories, databases, etc.) Company Product A Company ProductB MIMB Export From (import bridge) Import(export bridge) To Source: Meta Integration Technology, Inc.

Red Sky Data Inc.24 MIMB Features Implements/integrates IDEF1X Data Modelling and UML Object Modelling standards Meta data import from: Relational databases Data and object modelling design tools ETL, DW, BI environments OMG CWM/XMI compliant modelling tools Meta data repositories

Red Sky Data Inc.25 Oracle’s Meta Data Solution Meta data shared between Oracle database components Two primary components: Oracle Warehouse Builder (OWB) Oracle RDBMS

Red Sky Data Inc.26 Oracle Warehouse Builder (OWB) Meta Data Change Management Backup and history management Versioned meta data objects Applies to any individual object (e.g. table) or objects within objects (e.g. tables within a module) Support for MITI Bridges Import designs from over 40 3 rd party design tools (e.g. ErWin, PowerDesigner, Embarcadero ER Studio)

Red Sky Data Inc.27 OWB – Other Features Information about all ETL components is stored in the Oracle-based repository Documented public views of repository API available Adheres to OMG-CWM Version support Multi-user environment

Red Sky Data Inc.28 OWB – Other Features Object-level security policies Meta data accessible from OWB client application or web-based Design Browser Data lineage and impact analysis viewable from Design Browser Descriptive information can be added to components

Red Sky Data Inc.29 OWB – Other Features OWB meta data model is extendible, allowing addition of user-defined properties – user interfaces automatically extend Export bridges available: Oracle Discoverer and Oracle OLAP Server CWM format MITI

Red Sky Data Inc.30 Oracle RDBMS* Meta data specific features included in 9i and 10 g DBMS_METADATA package Retrieve meta data from repository as XML or DDL Re-create objects using XML / DDL

Red Sky Data Inc.31 DBMS_METADATA Subprograms

Red Sky Data Inc.32 DBMS_METADATA Subprograms

Red Sky Data Inc.33 DBMS_METADATA Tips To improve client performance, increase number of objects returned by FETCH_xxx functions by using SET_COUNT procedure Use SET_FILTER procedure to filter meta data objects

Red Sky Data Inc.34 Shortcomings In The Marketplace No one vendor fully supports a complete meta data solution Poor business meta data support

Red Sky Data Inc.35 Inmon’s Evaluation (2001)* D= Direct Support, I= Indirect Support

Red Sky Data Inc.36 A Sample Solution CWM is best method, but custom solution may be required to address gaps Following sample based on environment using: Informatica Cognos DB2

Red Sky Data Inc.37 Sample Model Logical divisions Business Meta Data Design Meta Data ETL Meta Data Reporting Tool Meta Data

Red Sky Data Inc.38 Sample Model*

Red Sky Data Inc.39 Sample Model – Business*

Red Sky Data Inc.40 Sample Model – Design*

Red Sky Data Inc.41 Sample Model – ETL*

Red Sky Data Inc.42 Sample Model – Reporting*

Red Sky Data Inc.43 Meta Data Load Stream Source Extracts End UsersData Warehouse Staging Area Data Mart Data Mart Data Warehouse Stream ERwin Extract DB2 Catalog Informatica Repository Cognos Report Listing MS Access Verification & Input Meta Data Repository Meta Data Data Mart Source System Document End User Interface Technical & Power Users Meta Data Stream

Red Sky Data Inc.44 Security in Meta Data End user access should be read-only Treat meta data components as any other database component: Create users and groups Grant access to groups Use Oracle's row-level security (or equivalent) if needed

Red Sky Data Inc.45 In Summary Meta data completes the architecture of the data warehouse environment It includes both technical and business information Vendor tools have improved, but still some work to do CWM is the standard protocol for meta data sharing Take advantage of Oracle's built-in meta data support

Red Sky Data Inc.46 A Q & Q U E S T I O N S A N S W E R S David Stanford Thank You! Steve Jones Session #402