1. THE ARCHITECTURAL COMPONENTS

2. CHAPTER OBJECTIVES Understand data warehouse architecture
Learn about the architectural components
Review the distinguishing characteristics of data warehouse architecture
Examine how the architectural framework supports the flow of data
Comprehend what technical architecture means
Study the functions and services of the architectural components
1/16/2019
Data Warehouse

3. Architecture: Definitions
The structure that brings all the components of a data warehouse together is known as the architecture.
1/16/2019
Data Warehouse

4. For example Take the case of the architecture of a school building.
The architecture of the building is not just the visual style. It includes the various classrooms, offices, library, corridors, gymnasiums, doors, windows, roof, and a large number of other such components. When all of thesec ompo-nents are brought and placed together, the structure that ties all of the components Toge-ther is the architecture of the school building.
1/16/2019
Data Warehouse

5. Architecture in Three Major Areas
As you already know, the three major areas in the data warehouse are:
Data acquisition
Data storage
Information delivery
1/16/2019
Data Warehouse

6. Architectural components in the three major areas.
1/16/2019
Data Warehouse

7. DISTINGUISHING CHARACTERISTICS
The architecture has distinguishing characteristics worth considering in detail.
Different Objectives and Scope
Data Content
Complex Analysis and Quick Response
Flexible and Dynamic
Metadata-driven
1/16/2019
Data Warehouse

8. Different Objectives and Scope
There are several sets of factors to consider.
you must consider the number and extent of the data sources.
How many legacy systems are you going to extract the data from?
What are the external sources?
Are you planning to include departmental files,
spreadsheets, and private databases?
What about including the archived data?
1/16/2019
Data Warehouse

9. Data Content
The “read-only” data in the data warehouse sits in the middle as the primary component in the architecture.
In an operational system, although the database is important, this importance does not measure up to that of a data warehouse data repository.
1/16/2019
Data Warehouse

10. Complex Analysis and Quick Response
Your data warehouse architecture must support complex analysis of the strategic information by the users.
Information retrieval processes in an operational system dwindle in complexity when compared to the use of information from a data warehouse.
1/16/2019
Data Warehouse

11. Flexible and Dynamic
You have to make sure your data warehouse architecture is flexible enough to accommodate additional requirements as and when they surface.
1/16/2019
Data Warehouse

12. Metadata-driven
Metadata surrounds the entire movement as the data moves from the source systems to the end-users as useful, strategic information.
1/16/2019
Data Warehouse

13. ARCHITECTURAL FRAMEWORK
In this section, we grouped the architectural components as :
building blocks in the three distinct areas
of data acquisition,
data storage,
and information delivery.
1/16/2019
Data Warehouse

14. Architecture Supporting Flow of Data
1/16/2019
Data Warehouse

15. Manajemen dan Kontrol Module
This component has two major functions:
to constantly monitor all the ongoing operations
to step in and recover from problems when things go wrong.
1/16/2019
Data Warehouse

16. Technical Architecture
The technical architecture of a data warehouse is, therefore, the complete set of functions and services provided within its components.
1/16/2019
Data Warehouse

17. Data Acquisition
This area covers the entire process of extracting data from the data sources, moving all the extracted data to the staging area, and preparing the data for loading into the data warehouse repository.
1/16/2019
Data Warehouse

18. Data Flow
Flow. In the data acquisition area, the data flow begins at the data sources and pauses at the staging area. After transformation and integration, the data is ready for loading into the data warehouse repository.
Data Sources. For the majority of data warehouses, the primary data source consists of the enterprise’s operational systems. Many of the operational systems at several enterprises are still legacy systems.
1/16/2019
Data Warehouse

19. Intermediary Data Stores
Intermediary Data Stores. As data gets extracted from the data sources, it moves through temporary files.
Staging Area. This is the place where all the extracted data is put together and prepared for loading into the data warehouse.
1/16/2019
Data Warehouse

20. Functions and Services
List of Functions and Services :
Data Extraction
Data Transformation
Data Staging
1/16/2019
Data Warehouse

21. List of Functions and Services :
Data Extraction
Select data sources and determine the types of filters to be applied to individual sources
Generate automatic extract files from operational systems using replication and other techniques
Create intermediary files to store selected data to be merged later
Transport extracted files from multiple platforms
Provide automated job control services for creating extract files
Reformat input from outside sources
Reformat input from departmental data files, databases, and spreadsheets
Generate common application code for data extraction
Resolve inconsistencies for common data elements from multiple sources
1/16/2019
Data Warehouse

22. List of Functions and Services :
Data Transformation
Map input data to data for data warehouse repository
Clean data, deduplicate, and merge/purge
Denormalize extracted data structures as required by the dimensional model of the data warehouse
Convert data types
Calculate and derive attribute values
Check for referential integrity
Aggregate data as needed
Resolve missing values
Consolidate and integrate data
1/16/2019
Data Warehouse

23. Data Staging Provide backup and recovery for staging area repositories
List of Functions and Services :
Data Staging
Provide backup and recovery for staging area repositories
Sort and merge files
Create files as input to make changes to dimension tables
If data staging storage is a relational database, create and populate database
Preserve audit trail to relate each data item in the data warehouse to input source
Resolve and create primary and foreign keys for load tables
Consolidate datasets and create flat files for loading through DBMS utilities
If staging area storage is a relational database, extract load files
1/16/2019
Data Warehouse

24. Data Storage
This area covers the process of loading the data from the staging area into the data warehouse repository.
All functions for transforming and integrating the data are completed in the data staging area.
1/16/2019
Data Warehouse

25. Data Flow
Flow. For data storage, the data flow begins at the data staging area. The transformedand integrated data is moved from the staging area to the data warehouse repository.
Data Groups. Prepared data waiting in the data staging area fall into two groups. Thefirst group is the set of files or tables containing data for a full refresh. The other group of data is the set of files or tables containing ongoing incremental loads.
1/16/2019
Data Warehouse

26. The Data Repository. Almost all of today’s data warehouse databases are relational databases. All the power, flexibility, and ease of use capabilities of the RDBMS become available for the processing of data.
1/16/2019
Data Warehouse

27. Functions and Services
List of Functions and Services :
Load data for full refreshes of data warehouse tables
Perform incremental loads at regular prescribed intervals
Support loading into multiple tables at the detailed and summarized levels
Optimize the loading process
Provide automated job control services for loading the data warehouse
Provide backup and recovery for the data warehouse database
Provide security
Monitor and fine-tune the database
Periodically archive data from the database according to preset conditions
1/16/2019
Data Warehouse

28. Information Delivery
This area spans a broad spectrum of many different methods of making information available to users.
1/16/2019
Data Warehouse

29. Data Flow
Flow. For information delivery, the data flow begins at the enterprise-wide data warehouse and the dependent data marts when the design is based on the top-down technique. When the design follows the bottom-up method, the data flow starts at the set of conformed data marts.
Service Locations. In your information delivery component, you may provide query services from the user desktop, from an application server, or from the database itself.
1/16/2019
Data Warehouse

30. Data stores for standard reporting
Data Stores. For information delivery, you may consider the following intermediary data stores:
Proprietary temporary stores to hold results of individual queries and reports for repeated use
Data stores for standard reporting
Proprietary multidimensional databases
1/16/2019
Data Warehouse

31. Functions and Services
List of Functions and Services :
Provide security to control information access
Monitor user access to improve service and for future enhancements
Allow users to browse data warehouse content
Simplify access by hiding internal complexities of data storage from users
Automatically reformat queries for optimal execution
Enable queries to be aware of aggregate tables for faster results
Govern queries and control runaway queries
Provide self-service report generation for users, consisting of a variety of flexible options to create, schedule, and run reports
1/16/2019
Data Warehouse

32. Functions and Services
List of Functions and Services :
Store result sets of queries and reports for future use
Provide multiple levels of data granularity
Provide event triggers to monitor data loading
Make provision for the users to perform complex analysis through online analytical processing (OLAP)
Enable data feeds to downstream, specialized decisions support systems such as EIS and data mining
1/16/2019
Data Warehouse

33. CHAPTER SUMMARY
Architecture is the structure that brings all the components together.
Data warehouse architecture consists of distinct components with the read-only data repository as the centerpiece.
The architectural components support the functioning of the data warehouse in the three major areas of data acquisition, data storage, and information delivery.
Data warehouse architecture is wide, complex, expansive, and has several distiguishing characteristics.
The architectural framework enables the flow of data from the data sources at one end and the user’s desktop at the other.
The technical architecture of a data warehouse is the complete set of functions and services provided within its components. It includes the procedures and rules needed to perform the functions and to provide the services. It encompasses the data stores needed for each component to provide the services.
1/16/2019
Data Warehouse