Foundations of Database Systems Class Introduction G. Green 1

Agenda Introductions Seating Chart Course Overview Syllabus Case Database Development Overview G. Green 2

Foundations of Database Systems Objectives Understand data-related activities of SDLC Implement data modeling, database design, and database implementation techniques CASE (Visio) Database (SQL Server) Course Contents Lectures, Examples, In-Class Exercises Individual Assignments (3) Team Project* (3 parts) Quizzes (3) Exams (2) *Can request teammates; see syllabus for Team Preferences deadline G. Green 3

Research Service Learning & Kolb’s Learning Cycle International and US Periodic Assessments Some NOT graded; others are G. Green 4

Learning  Participate : › Prepare --read & reread book, notes-- for each class › Attend, listen, be attentive, engaged › Ask and answer questions, & add to discussion › Do each assignment completely & in a timely and professional manner  Take PLENTY of notes in class: › Do NOT just rely on powerpoint  Explore : › Go beyond classroom material G. Green 5

Class Resources  Syllabus/Schedule, Grades, Attendance:   Schedule also contains links to all lecture slides, study guides, assignments and project write-ups  Other Resources:   NOTE: the syllabus/schedule on this website will NOT contain the links described above G. Green 6

Syllabus… G. Green 7

Introduction to Databases Chapter 1 G. Green 8

Topics Chapter 1 The Database Environment Database Development Process Chapter 9 (Pages 409 – 410) Big Data Chapter 10 (Pages 444 – 445, ) Master Data Management Data Federation Chapter 11 (Pages 464 – 472, 486, 499 – 506) Database Personnel Metadata Management (e.g., Data Dictionaries) Backup Facilities Overview of Tuning the Database for Performance G. Green 9

10 Evolution of Database Technologies 1960’s1970’s1980’s1990’s2000+ Federated MDDB XML NoSQL ……. Traditional Files Hierarchical Network Relational Object Object-Relational

11 Figure 1-3 Old file processing systems: Example Duplicate Data

Traditional File Processing Environment  Disadvantages: › Program-data dependence = “structural” & “data” › Limited data sharing = “islands of automation” › Duplication of data = “redundancy” › Lengthy development times › Excessive program maintenance G. Green 12

The Database Environment G. Green 13

Advantages of Databases Program-data independence Improved data sharing Minimal data redundancy Improved data accessibility/responsiveness Improved data consistency Faster application development Enforcement of standards Improved data quality Reduced program maintenance G. Green 14

Data and Database Administration Chapter 11 G. Green 15

Traditional Administration Definitions Data Administration Data Administration: A high-level function that is responsible for the overall management of data resources in an organization, including maintaining corporate-wide definitions and standards Database Administration Database Administration: A technical function that is responsible for physical database design and for dealing with technical issues such as security enforcement, database performance, and backup and recovery 16

Data People Involved in SDLC Data Administrators Data(base) Analysts/Designers  requirements elicitation, design Business (Intelligence) Analyst  BI requirements, design Data Architects  strategy, governance Data Stewards  quality, metadata, MDM Business Analytics Engineer  data analytics, statistics, mining Data Mining Engineer; Big Data  “big data” specialists Engineer; Data Scientist … Database Administrators (System) DBAs  implementation/maintenance Application DBAs Procedural DBAs  stored code e-DBAs  web-enabled DBMSs Data Warehouse Administrators  ETL, DW implementation G. Green 17

Growing Skillset Relational database design, implementation Database programming ETL (extract, translate, load) Data warehousing design (star schema) and implementation (MDDB) Data analysis, reporting, and mining techniques Cloud database implementations Statistical modeling with tools such as R, SAS, or SPSS Data visualization tools Technologies for structured and unstructured data Hadoop (Hadoop is an Apache project to provide an open-source implementation of frameworks for reliable, scalable, distributed computing and data storage.) NoSQL "NewSQL" ***See Big Data University for (mostly) free self-study trainingBig Data University 18 G. Gree n

Data Quality and Integration Chapter 10 G. Green 19

Metadata Management System Catalog Part of DBMS "Active" dictionary Data Dictionary Typically "passive" Extension of catalog metadata Information Repository (e.g., IRDS) Standards for data dictionaries Integrates dictionaries G. Green 20

Master Data Management "Ensuring the currency, meaning, and quality of reference data within and across various subject areas" (pg 444) Identify Common Data Subjects Common Data Elements Sources of "the truth" Cleanse Update applications to reference Master Data repository Ensures consistency of key data (not ALL data) throughout organization 21 G. Gree n

Database Development Process G. Green 22

Systems Development Life Cycle G. Green 23 Planning Analysis Design Implementation Enterprise Modeling* DB Scope, Requirements (Conceptual Data Model) DB Design (Logical DB Design) DB Design (Physical DB Design) DB Implementation (Load, Test, Eval, Op) DB Maintenance* DB Activities in SDLCSDLC for this class

Enterprise Data Modeling Determine organizational data requirements Build enterprise data model outcome is a very high-level Entity-Relationship Diagram see :

25 G. Gree n Source:

Conceptual Data Modeling  Determine user data requirements  Determine business rules  Build conceptual data model › outcome is an Entity-Relationship Diagram (conceptual schema) G. Green 26

Logical Database Design  Select database model › e.g., the Relational Model  Transform conceptual (ERD) into logical (relational) data model  Normalize data structures › Outcome is normalized, relational tables G. Green 27

Physical Database Design  Select database product (e.g., SQL Server)  Select storage device(s)  Design fields, records, files (physical schema) › outcomes are detailed, physical definitions for:  fields (data dictionary)  records (space requirements for physical structures)*  files (access methods) *Will not do in this class G. Green 28

Database Implementation Create database file/table structures Create views (external schema) Establish access rights Load test data Write/test programs that process data Install database (with production data) into production operations › outcomes are secured database tables loaded with data G. Green 29

Database Maintenance Maintain database structures Storage/space management Performance, tuning I/O Contention CPU Usage Application Tuning Data availability DBMS upgrades, "fixes" Backup, recovery …….

Database Maintenance, cont… Backup Full Incremental Differential Business Continuity Data Replication ("fallback") 31 G. Gree n

Data and Database Administration Chapter 11 G. Green 32

Cloud Computing Business Model Computing resources on demand Need-based architectures Internet-based delivery Pay as you go History (VERY high-level and approximate) 33 G. Gree n Time-sharing Virtual Machines Utility Computing WWW Personal Computers Grid Computing Cloud Computing 50's60's70's80's90's 2000's

Cloud Computing Services Impacts to Data(base) Administration See textbook page 469 G. Green 34

Summary Evolution of Data Management Disadvantages of file processing Database Concepts Components of a DBMS Environment Database Advantages Database Development: Overall SDLC Database Activities in the SDLC Data Models/Schemas What they represent People Involved in SDLC (esp. DB) Traditional job divisions and responsibilities Newer job titles G. Green 35