Overview Thanks to Dr. Raj and Dr. Liu for sharing course materials; also S. Miner from Gordon College Database Concepts

Course Overview Broad introduction to database management systems (DBMS) and the design, implementation, and applications of databases. Topics: – an overview of DBMS architectures; – concepts and implementations of the relational model; – SQL; – database design and modeling techniques; – As time allows, others, such as transaction management, indexing…

Who are you? Your name, program/year, hometown Why are you taking this course? Anything else??

Data Management What are some data management problems? Previous solutions: – Pen and Paper – Rolodex Cards Image Credit: ConcordSupplies.com

Data Management Why not a simple text file? An Excel spreadsheet? How about a more sophisticated structure: – Linked list – BST – Hash – Indexed file What are the problems with each of the structures mentioned above?

Problems Integrity constraints buried in application logic – hard to add to or change Atomicity problems – what happens when the system crashes during an important operation? Concurrency issues – when multiple users work with the same data at the same time Security issues – how to give someone access to some, but not all, of the data

Other Issues Text file gets big - Searches become slow Customer information changes Duplication Relationship between data elements “Must-exist” data and “must-not-access” data Ad hoc analysis and retrieval

Why can’t we just use files? What do you think?

File Systems for Data: Drawbacks Data redundancy and inconsistency – Multiple file formats, information duplication Difficulty in accessing data – Must write a new program to carry out each new task Integrity issues – Integrity constraints embedded in programs Data updates – may leave database in inconsistent state

File Systems for Data: Drawbacks Concurrent access by multiple users – Needed for performance, but can lead to inconsistency Security problems – Who gets to see the data and modify it Update inconsistencies if multiple copies Data may be in > 1 file, in > 1 location Different implementations and structures; now need people able to understand all in order to support …

Database Management System (DBMS) Decouples applications from the files on the file system Programs go through the database to access data stored in the underlying files Manages very large amounts of data – Collection of interrelated data about particular enterprise Access to very large amounts of data – Efficient, concurrent, secure and atomic access Two people withdrawing money from the same account at different ATMs Usable, convenient, and efficient environment Provides set of programs for data access

DBMS Applications Banking – All transactions Airlines – Reservations, schedules Universities – Registration, grades Payroll Management – Manage employees, pay, taxes, and so on …

DBMS Time Line Pre-1960 – Transition from punched card and tape 1960s – From file management to databases – Hierarchical Data Model – Multi-user access with network 1970s – Codd (IBM) Relational Model – Chen introduced Entity Relationship Model – Query languages developed (SQL)

DBMS Time Line (continued) 1980s – Client/Server DBs, Oracle, DB2 PC databases, DBase, Paradox, and more – SQL standard for definition and manipulation 1990s – Web-based information delivery – Trends Expert/object/distributed/XML DBMS 2000s – Add scale to 1990s – Data integration, mining, and privacy Current Problem: Big Data

DBMS Users (Loosely Based on Interaction) Application programmers – Interact with system through Data Manipulation Language (DML) calls Naïve users – Invoke one of the previously written “permanent” application programs Sophisticated users – Form requests in a database query language Specialized users – Write specialized database applications that do not fit into traditional data processing framework

Data Dictionary In addition to storing data, the DBMS also stores metadata – data about the data – in a data dictionary A standard name for each data item that applications use to access it Where the data item is stored (which file and where in the file) Security constraints – rules about who is allowed to access which data can be applied at the data item level; these are enforced by the DBMS Integrity constraints – which values are valid for data items; enforced by the DBMS

Abstraction Physical – where the data is actually stored (files) Logical (conceptual) – describes data and data relationships in the data View – targeted end-user interfaces to database that highlights some data, hides others, and may include virtual fields computed from the data. Data independence – changes at one abstraction layer should not impact other layers

Database Administrator (DBA) Coordinates all enterprise DBMS activities – Understands data resources and needs Database administrator's duties – Schema and physical definition, organization, and modification – Storage structure and access method – Specifying integrity constraints and security – Granting access – Monitors performance & maintenance DBA role – Varies from organization to organization

High-level View of DBMS Components Storage management – Data organization on secondary storage for processing queries efficiently Query processing component – Efficient query execution, algorithms for implementing relational operators Transaction management – Concurrency control; failure and recovery Application interface – APIs, query tools, administration tools

Application Architectures Two-tier architecture – Client programs communicate with DBMS server Three-tier architecture – Client programs communicate with middleware server – Middleware server communicates with DBMS server

Relational DBMS Relational database – Consist of a set of relations (tables) Languages – Programming languages – DBMS languages Data Definition Language (DDL) Data Manipulation Language (DML) Database application programming

A Sample Relational Database tagnameagemom 12Fido209 09Fifi807 07Fiona12null 13Frisky109 Dogaddresssizecolor 13 Elm bigred 4 Ash smallwhite 7 Oak biggreen DogHouseaddresstag 13 Elm 12 4 Ash 09 7 Oak 07 null13 HouseToDog

Data Definition Language (DDL) Notation to specify database schema – Includes storage space, usage, indexes, keys create table dog ( tag integer, name char(10), age integer, mom integer )

Data Manipulation Language (DML) Language for accessing and manipulating data – DMLs also known as query languages SQL, the most widely used query language Two classes of languages – Procedural or Low-level User specifies what data is required and how to get those data – Nonprocedural or High-level User specifies what data is required without specifying how to get the data SQL is predominantly non-procedural

Data Manipulation is CRUD Create Retrieve Update Delete insert into dog (tag,name,age) values (12, ‘Lady’, 5) select name from dog where age > 3 update dog set age = age + 1 where tag = 12 delete from dog where age > 1

DBMS Programming Application programs access DBMS via – Language extensions to allow embedded SQL – Application program interfaces (APIs) ODBC/JDBC that allow SQL queries to be sent to the DBMS

Drawbacks The additional DBMS software layer comes with some costs Each application incurs overhead by going through the database to access its data Applications cannot optimize access to data stored in the files on their own Designers and programmers need more (albeit standardized) knowledge of how a DBMS works Additional layer can lead to increased complexity (at least in the short term) Database and file systems are not “either / or” solutions, more like “both / and”

Recap - Concerns Data redundancy Data inconsistency Security constraints Integrity constraints Concurrency constraints

Problems Data redundancy - wasted space Update issues – every copy of the data needs to be modified Data inconsistency – sometimes every copy is not modified Data access issues (getting to just the right data) “There’s no program for that.” Data isolation – merging data from disparate sources together