1. Database and Data Warehouse
October 8, 2015

2. LEARNING GOALS Explain basic concepts of data management.
Describe traditional file systems and identify their problems.
Define database management systems and describe their various functions.
Explain how the relational database model works.
Explain Object-Oriented databases.
Explain Data Warehouse, Data Mart 2

3. Mini case: Bank Accounts’ data
What basic data/info are required for opening bank account? 3

4. Access and Management tools
What is a database?
Collection of related files containing records on entities like people, places, events, things, etc.
Databases make data easy to access and manage.
Customers Info
Accounts Info
Employees Info
Access and Management tools 4

5. Basic Concepts of Data Management
Table 1
Table 2
Table 3
Report
Form 1 Acc #:_______ Name:_______
Database: Collection of data organized in different containers 5

6. Basic Concepts of Database systems
Accounts table
AccountID
Customer
Type
Balance
660001
John Smith
Checking
$120.00
660002
Linda Martin
Saving $
660003
Paul Graham $
Each table has:
Fields
Records
1 Primary key
Table
Two-dimensional structure composed of rows and columns
Field
Like a column in a spreadsheet
Field name
Like a column name in a spreadsheet
Examples: AccountID, Customer, Type, Balance
Field values
Actual data for the field
Record
Set of fields that describe an entity (a person, an account, etc.)
Primary key
A field, or group of fields, that uniquely identifies a record 6

7. Basic Concepts in Data Management
A Primary key could be a single field like in these tables
Primary key
AccountID
Customer
Type
Balance
660001
John Smith
Checking
$120.00
660002
Linda Martin
Saving $
660003
Paul Graham $
Primary key could be a composite key, i.e. multiple fields 7

8. Traditional File Systems
Early attempt to computerize manual filling system
System of files that store groups of records used by a particular software application
Simple but with a cost
Inability to share data
Inadequate security
Difficulties in maintenance and expansion
Allows data duplication (e.g. redundancy)
Application 1
Program 1
File 1
File 2
File 3
Program 2
Application 2
Program 1
File 1
File 2
File 3
Program 2 8

9. Traditional File System Anomalies
Insertion anomaly
Data needs to be entered more than once if located in multiple file systems
Modification anomaly
Redundant data in separate file systems
Inconsistent data in your system
Deletion anomaly
Failure to simultaneously delete all copies of redundant data
Deletion of critical data 9

10. Database Advantages
Database advantages from a business perspective include:
Ease of data insertion
Example: can insert a new address once; and the address is updated in all forms, reports, etc.
Increased flexibility
Handling changes quickly and easily
Increased scalability and performance
Scalability: how the DB can adapt to increased demand
Reduced information redundancy & inconsistency
Increased information integrity (quality)
Can’t delete a record if related info is used in other container
Increased information security
All of the above are discussed in the following slides:
A good way to explain databases is to compare them to spreadsheets
What are the limitations when using a spreadsheet?
Limited number of rows and columns (Excel ,536 rows by 256 columns) Once you use more than 65,536 rows you have outgrown your spreadsheet
Only one users can access the spreadsheet
Users can view all information in the spreadsheet
Users can change all information in the spreadsheet
All of the disadvantages associated with a spreadsheet are fixed when using a database
These advantages are discussed in detail over the next several slides 10

11. Common Database software
All of the above are discussed in the following slides:
A good way to explain databases is to compare them to spreadsheets
What are the limitations when using a spreadsheet?
Limited number of rows and columns (Excel ,536 rows by 256 columns) Once you use more than 65,536 rows you have outgrown your spreadsheet
Only one users can access the spreadsheet
Users can view all information in the spreadsheet
Users can change all information in the spreadsheet
All of the disadvantages associated with a spreadsheet are fixed when using a database
These advantages are discussed in detail over the next several slides 11

12. Types of DBMSs Desktop Designed to run on desktop computers
Server / Enterprise
Handheld
Desktop
Designed to run on desktop computers
Used by individuals or small businesses
Requires little or no formal training
Does not have all the capabilities of larger DBMSs
Examples: Microsoft Access, FileMaker 12

13. Types of DBMSs (Cont.) Server / Enterprise
Designed for managing larger and complex databases by large organizations
Typically operate in a client/server setup
Either centralized or distributed
Centralized – all data on one server
Easy to maintain
Prone to run slowly when many simultaneous users
No access if the one server goes down
Distributed – each location has part of the database
Very complex database administration
Usually faster than centralized
If one server crashes, others can still continue to operate.
Examples: Oracle Enterprise, DB2, Microsoft SQL Server 13

14. Types of DBMSs (Cont.) Handheld Designed to run on handheld devices
Less complex and have less capabilities than Desktop or Server DBMSs
Example: Oracle Database Lite, IBM’s DB2 Everywhere. 14

15. Database Management System (DBMS)
Combination of software and data for
Collecting, storing and managing data in a database environment.
A DBMS includes:
Database
Database engine (for accessing and modifying the DB content)
Data Manipulation Language
Application 1 Program-1 Program-2
Application 2 Program-1 Program-2
DBMS 15

16. Database Management System (DBMS)
Software through which users and application programs interact with a database
Discuss the two primary forms of user interaction with a database
Direct interaction
The user interacts directly with the DBMS
The DBMS obtains the information from the database
Indirect interaction
User interacts with an application (i.e., payroll application, manufacturing application, sales application)
The application interacts with the DBMS 16

17. ID Name Amt 01 John 23.00 02 Linda 3.00 03 Paul 53.00
DBMS Functions
Create database structure (tables, relationships, schema, etc.)
Transform data into information (reports, ..)
Provide user with different logical views of actual database content
Provide security: password authentication, access control
DBMSs control who can add, view, change, or delete data in the database
Logical views
Physical view
ID Name Linda
ID Name Amt 01 John Linda
ID Name Amt 01 John Linda Paul
Name Amt Paul 17

18. DBMS Functions (cont.) Allowing multi-user access with control
Control concurrency of access to data
Prevent one user from accessing data that has not been completely updated
When selling tickets online, Ticketmaster allows you to hold a ticket for only 2 minutes to make your purchase decision, then the ticket is released to sell to someone else – that is concurrency control 18

19. Database Models
Database model = a representation of the relationship between structures (e.g. tables) in a database
Common database models
Flat file model
Relational model (the most common, today)
Object-oriented database model
Hierarchical model
Network model 19

20. Flat File Database model
Stores data in basic table structures
No relationship between tables
Used on PDAs for address book 20

21. Relational Database Model
Multiple two-dimensional tables related by common fields
Uses controlled redundancy to create fields that provide linkage relationships between tables in the database
These fields are called foreign keys – the secret to a relational database
A foreign key is a field, or group of fields, in one table that is the primary key of another table
Handles One-to-Many and One-to-One relationships 21

22. Object-Oriented Database model
Needed for multimedia applications that manage images, voice, videos, graphics, etc.
Used in conjunction with Object-oriented programming languages
Slower compared to relational DBMS for processing large volume of transactions
Hybrid object-relational Databases are emerging 22

23. Hierarchical Database Model
Data is organized into a tree-like structure using parent-child relationships.
Created in the 1960s by IBM
Limited to storing data in One-to-Many relationships
One parent segment to many child segments
Not very flexible
Examples: IBM’s Information Management System (IMS) and Windows registry.

24. Network Database model
Developed in 1969
Many-to-Many relationships between entities
Any record may be linked to any other record
Highly flexible but also highly complex
Hard to maintain
Rarely used today

25. Data Warehouse
a logical collection of information gathered from many different data sources
Stores current and historical data
supports business analysis activities and decision-making tasks
The primary purpose of a data warehouse is to aggregate information throughout an organization into a single repository for decision-making purposes
What is the primary difference between a database and data warehouse?
The primary difference between a database and a data warehouse is that a database stores information for a single application, whereas a data warehouse stores information from multiple databases, or multiple applications, and external information such as industry information
This enables cross-functional analysis, industry analysis, market analysis, etc., all from a single repository
Data warehouses support only analytical processing (OLAP) 25 25

26. Data Warehouse Fundamentals
Many organizations need internal, external, current, and historical data
Data Warehouse are designed to, typically, store and manage data from operational transaction systems, Web site transactions, external sources, etc. 26

27. Data Warehouse: A Multi-Tiered Architecture
Operational
DBs
Other
sources
Monitor
&
Integrator
OLAP Server
Metadata
Extract
Transform
Load
Refresh
Analysis
Query
Reports
Data mining
Serve
Data
Warehouse
Data Marts
Data Sources
Data Storage
OLAP Engine
Front-End Tools

28. Multidimensional Analysis
Data mining – the process of analyzing data to extract information not offered by the raw data alone
Data-mining tools use a variety of techniques (fuzzy-logic, neural networks, intelligent agents) in order to
find patterns and relationships in large volumes of data
and infer rules that predict future behavior and guide decision making
Other analytical tools: query tools, statistical tools, etc. used to
Analyze data, determine relationships, and test hypotheses about the data
Data mining can begin at a summary information level (coarse granularity) and progress through increasing levels of detail (drilling down), or the reverse (drilling up)
Data-mining tools include query tools, reporting tools, multidimensional analysis tools, statistical tools, and intelligent agents
Ask your students to provide an example of what an accountant might discover through the use of data-mining tools
Ans: An accountant could drill down into the details of all of the expense and revenue finding great business intelligence including which employees are spending the most amount of money on long-distance phone calls to which customers are returning the most products
Could the data warehousing team at Enron have discovered the accounting inaccuracies that caused the company to go bankrupt?
If the did spot them, what should the team have done? 28 28

29. Data Warehouse Fundamentals
The data warehouse modeled in the above figure compiles information from internal databases or transactional/operational databases and external databases through ETL
It then send subsets of information to the data marts through the ETL process
Ask your students to distinguish between a data warehouse and a data mart?
Ans: A data warehouse has an enterprisewide organizational focus, while a data mart focuses on a subset of information for a given business unit such as finance
Extraction, transformation, and loading (ETL) – a process that extracts information from internal and external databases, transforms the information using a common set of enterprise definitions, and loads the information into a data warehouse. 29

30. Information Cleansing or Scrubbing
Organizations must maintain high-quality data in the data warehouse
Information cleansing or scrubbing
a process that weeds out and fixes or discards inconsistent, incorrect, or incomplete information
first, occurs during ETL. Then, when the data is in the Data Warehouse using Information cleansing or scrubbing tools.
This is a an excellent time to return to the information learned in Chapter 6 on high-quality and low-quality information
What would happen if the information contained in the data warehouse was only about 70 percent accurate?
Would you use this information to make business decisions?
Is it realistic to assume that an organization could get to a 100% accuracy level on information contained in its data warehouse?
No, it is too expensive 30 30

31. Data Mart
Subset of data warehouses that is highly focused and isolated for a specific population of users
Example: Marketing data mart, Sales data mart, etc. 31

32. Database vs. Data Warehouse
Databases contain information in a series of two-dimensional tables
In a Data Warehouse and data mart, information is multidimensional, it contains layers of columns and rows
Total annual sales
of TV in U.S.A.
Each layer in a data warehouse or data mart represents information according to an additional dimension
Dimensions could include such things as:
Products
Promotions
Stores
Category
Region
Stock price
Date
Time
Weather
Why is the ability to look at information based on different dimensions critical to a businesses success?
Ans: The ability to look at information from different dimensions can add tremendous business insight
By slicing-and-dicing the information a business can uncover great unexpected insights
Date
Product
Country
All, All, All
sum TV
VCR PC
1Qtr
2Qtr
3Qtr
4Qtr
U.S.A
Canada
Mexico 32 32

33. Summary Questions
Notes
What is a database, a table, a field, a record, a primary key, a composite key?
2) What are the problems with traditional file systems?
3) What are the major functions of a DBMS?
(a) Name some Desktop DBMSs. (b) Name some Enterprise DBMSs. (c) Handheld DBMSs
Describe hierarchical database model, network model
What are the differences between Flat File, Relational, and Object-oriented database models?
What is Data warehouse? Data Mart?
What is Extraction, transformation, and loading (ETL)? What is data-mining? What is Information cleansing or scrubbing? 33