1. Iterative Waterfall Case Study: Network Information Data On-line Analysis Alessandro Zorer alessandro.zorer@create-net.it

2. Agenda  Iterative Waterfall methodology (based on Sodalia SIMEP)  General approach  DWH ‘tailoring’  Case Study: Network Information Data On-line Analysis  Needs  Approach  Focus on System Architecture  Functional View  Process View  Development View  Physical View  Fifth View  Summary  Q&A

3. Iterative Waterfall Methodology & Process Iteration # 2 Iteration # 1 System Development Strategy Concept Exploration and Context Analysis Iteration # n System Requirements  Iterative approach  Multilayer  Multiperspective System Architecture. Component Design. Component Development. Component Design. Component Test. System Test.

4. Adaptation to DWH Requirement Manager Use Case Prioritization Context Analyst Use Case and Actor identification Architect DW Designer Data Mart Designer Use Case model Structuring Integration Analysis Data Warehouse design DW Analysis Interface Prototyping DW may be tailored to each specific project and domain Product impact Analysis Expressed as a UML activity diagram

5. Context Analysis Business Model Develop Business Model Stakeholder Identification Manage Dependencies Elicit Stakeholder Needs Activity Diagrams Glossary Find Actors and Use Cases Structure the Use-Case Model Requirement Attributes Use-Case Model Responsible for Capture a Common Vocabulary Context Analyst

6. Concept Exploration UML Requirement Definition and System Architecture DESMET Analysis Requirements Analysis Business Analysis Use case identification Data Sourcces identification Data Consumers identification System Architecture System Architecture Design Performance Scalability Capacity Architectural Qualities Tools Assessment & Evaluation

7. Architectural Views  Logical View A logical abstract view of system elements and of the services to be provided to the end user  Process View Analysis the dynamic aspect of the system through scenarios and other diagrams (e.g., sequence, collaboration and activity diagrams). The elements focused are: tasks with their workflow, processes with their dependency, synchronization and concurrence aspects.  Development View Organization of the actual software modules in the software-development environment. The modules may be packaged in components or subsystems (component diagram) which may be organized in a hierarchy of layers, each layer providing a narrow well-defined interface to other layers.  Physical View Provide the deployment configurations in terms of Hardware and Software Components. This view shows the System Topology, a network of processing nodes with the software running on them. Capacity issues are addressed.  Fifth View Orthogonal view. Issues addressed are: Potential Software Reuse Analysis, Requirements allocation on Components, Performance Analysis, Functionality Categorization and Ranking.

8. DESMET Methodology for evaluating COTS based on:  Functional Qualities  Architectural qualities (i.e. adaptability, Scalability)  Performance analysis  Business aspects Time to market System lifecycle Contractual constraints Support organization

9. DW Design System Architecture Design UML E-R Physical Architecture Design Data Modeling MD Schema Data Flow Design Metadata Management Design Data MartsInput Design Output Design

10. DM Design Data Mart Construction TestingTraining Deployment Iteration CustomizationUnit Testing Data FlowDatabaseHardware

11. Support tools infrastructure Business Process Modeling Object / Component Modeling Logical / Physical Database Modeling Enterprise Integrated DW Modeling

12. Case Study: Network Information Data On-line Analysis Business needs:  Definition and development of a DataWarehouse Framework for Multidimensional Analysis based on:  Call Data (Network Management)  Fault Data (Problem Management)  Performance Data (End-to-End Analysis)  Optimization of network performances through gathering and analysis  High integrability of new data sources  Optimization and extension of on-line analysis functionalities  Quick creation of reports and flexibility for the end user (through custom Data Marts)  Extension of capabilities in term of historical data management.

13. Solution  A specialized and adaptable Data Warehouse solution to support Network Traffic Management and Call Behavior Analysis through a smart data correlation among CDR and configuration, performance and trouble tickets  Highly scalable to adapt from small to large business needs  Based an a mix of COTS and developed components  Flexible to accomadate a variety of different sources and Call Data Record formats  Detailed data analysis capabilities to support different DSS customer organizations  Predefined “good example” analysis library to quikly develop and deliver QoS monitors and Service Level Management functions

14. System Framework Approach  Simplify the design, implementation, and management of data warehousing solutions  An open architecture that allows easy integration with and extended by third party vendors  Heterogeneous data import, export, validation and cleansing services with optional data lineage  Integrated metadata for warehouse design, data extraction/transformation, server management, and end-user analysis tools  Core management services for scheduling, storage management, performance monitoring, alerts/events, and notification

15. Logical View  UML Domain and System Modeling  describes system concepts in a formal way  drives data modeling  drives components design  drives dynamic modeling  Standard-based Object Information Model (OIM) from Microsoft and Metadata Coalition

16. Layered Modeling Organization Data Transformation Layer Data Warehouse Layer Data Analysis Layer MetaData ManagementWorkFlow Management

17. Generic Record-oriented Model Element DeployedRecordLogicalRecord 88Level Field Record Column Attribute Classifier ModelElement Group RecordFormat 0..* +Type +DeployedCatalogs SummaryInformationTransformableObject GroupDef DeployedGroupLogicalGroupDeployedFieldLogicalField RecordItem

18. Generic Call Data Record Model NetworkElementServiceType CallDataRecord DeployedRecord SourceIDDestinationIDElapsedMeasure NEType

19. Generic OLAP Model Package DeployedOLAPDatabaseLogicalOLAPDatabase OLAPDatabase Dimension DimHierarchy Cube DataSourceCatalogStore Connection OLAP Server Connection ModelElement 0..* 1..* +Cubes +Dimensions +Data Sources +DimHierarchies +DeployedCatalogs

20. OLAP and DSS  Fast  five seconds or less.  Analysis  Performs basic numerical and statistical analysis of the data, predefined or ad hoc  Shared  Implements the security requirements across a large user population  Multidimensional  Is the essential characteristic of OLAP  Information  Accesses all the data and information wherever it may reside and not limited by volume.

21. Metadata Management 1. The link between the DSS system and the business analysts. 2. Critical for maintaining, controlling, and expanding the DSS system. Reduces the cost and cycle time of problem resolution. Technical Users (Developers & Analysts) Business Users (Executives & Business Analysts) Data Administrator METADATA

22. Metadata Consumer  Business Users  Less technical  Use predefined queries & reports  DSS navigation and definition  Power Business Users  More technical  Ad-hoc  Technical Users  Acquisition & access developers, analysts, data modelers, architects  Need users access patterns & frequency  Transformation rules

23. Metadata Management Business Meta Data Technical Meta Data Technical Users (Developers & Analysts ) Business Users (Executives & Business Analysts) Transformation Rules Attribute Names Domain Values Access Patterns Entity Relationships Attribute Business Definitions Entity Business Definitions Aggregation Rules Report Business Descriptions List of Available Reports Power Business Users Data Administrator

24. Data transformation u Finding the right data to satisfy end users needs u Moving the right data to the target u Scheduling and monitoring u Providing visual access u Linking transformations and movement metadata with all other metadata activity

25. Workflow Management Process integration DataintegrationDataintegration Metadata Operational Data DW

26. Sequence Diagram

27. Functional Architecture Operational Systems Data Trasformation Data Transformation Tools Load Validation Tools Warehouse Management Tools Data Warehouse Operational DB Applications Meta Data Sources Data Cleansing Tools Metadata Repository Enterprise Reference Data CASE & Modeling Tools Source Data Extract Tools Database Utilities Data Quality Assessment Tools DWData Marts Data Mining & Simulation Tools OLAP Data Query, Reporting and Visualization Tools Query Meta Data Movement & Replication Tools Meta Data Access Tools Meta Data Administration Utilities Project Deliverables Generator Meta Data Management Change Management Tools

28. Layered Architecture Data Transformation Enterprise Warehouse Replication & Propagation Dependent Data Mart Knowledge Discovery / Data Mining Information Access / Applications Business Users IT Users Data Warehouse Middleware Network ManagementDatabase ManagementSystems Management Network Admin Operations Manager Database Admin Data Analyst Applic. Developer Power Analyst Knowledge Worker Executive/ Manager Customer Contact Application Server Source Data (Internal and/or External) Metadata Logical Data Model Physical Data base Design Data Dictionaries Workflow Management Data Capture Data Management Data Analysis Support Infrastructure Staging Area

29. Components Integration Data Analysis Integrated Support Infrastructure Data Management Data Capture

30. Components Integration Data Analysis & DSS Data capture Integrated Support Infrastructure Summarization Data Cleaning & Maintenance Communication Service Infrastructure Query Data Management Schedule-driven acquisition Transformation chain Asynchronous acquisition Data Browser Workflow Change Management Report Sched WEB Services System Management

31. Physical View UnixMVS Server Platform Client Platform Windows NT Directory Services DW Intranet Windows 95/98/NT Unix WS OLAP Corporate Data

32. System Scalability System Sizing  Small Size ( <= 10 M CDR/day )  Medium Size ( >= 10 M <= 50 M CDR /day )  Large Size ( >= 50 M <= 200 M CDR / day ) Solutions :  Process distribution (divide et impera)  Different COTS choice (performance and TCO)  Hardware platform

33. Architectural Qualities  Performance (Canned queries, MD Analysis, Ad hoc, Min. Impact on Operational System)  Flexibility (MD Flex, Ad hoc, Change data structure)  Scalability (No. of Users, Volume of Data)  Ease of Use (Location, Formulation, Navigation, Manipulation)  Data Quality (Consistent, Correct, Timely, Integrated)  Connection to the Detail Business Transactions

34. Summary  Iterative waterfall approach for large projects …  Architecture as a CENTRAL activity for the success of projects  Scalability as a driving factor in this case  Standard adoption (Metadata Coalition OIM Model)  COTS + developed components to meets Time to market and Best-in-class solution  Flexibility in data capturing and high modularity to improve the level of integration with already in place systems Q&A