Big Data Yuan Xue CS 292 Special topics on

Part I Relational Database Yuan Xue

Discussion  Did you ever encounter a data management problem?  Experimental data from a homework?  Personal data?  Other data?  How did you manage your data?

Database  Database: An integrated collection of related data  Usually stored on secondary storage (as files)  Also in-memory database  Examples of databases  Vanderbilt student database, course registration and grading database (backend of YES);  Amazon’s products and customer database; Ebay’s products and transaction database;  Facebook’s user and message database;  And more… Database Data

Database Management System (DBMS)  DBMS: A collection of software/programs  Designed to assist in creating, and managing database  Support defining, constructing, manipulating, sharing databases  Examples of DBMSs  Relational DBMSs: Commercial: Oracle, IBM (DB2, Informix), Microsoft (SQL Server, Access); Open source: MySQL, PostgreSQL  NoSQL and newSQL: BigTable/Hbase, Cassandra, Redis, Riak, MongoDB, Dynamo, DynamoDB, Spanner  Other: object-oriented database, etc

Database System Environment Database DBMS Users Data Application Data Without DBMS With DBMS

Benefit of DBMS  Development convenience  Reduce application development time  Data independence:  Application programs not dependent on data representation and storage details  Data integrity and consistency:  Enforce consistency constraints on data  Data sharing and Concurrency control  Data is better utilized (discovered and reused), redundancy of data is minimized  Avoid undesirable race conditions that arise with simultaneous access/updates to data  Centralized control  DBA tunes the database to balance user's needs  Security  Prevent unauthorized access.  Crash recovery  Ensure the integrity of data in the presence of failures

Example Application – MiniTwitter  What data do we need?  What capabilities on the data do we need?

Example Application – MiniTwitter  What data do we need?  User profile info: ID, password, , display name, picture, people I follow, people who follow me.  Tweets: author, time, content (topic), replies (author, time, content), favorite (author, time),  What capabilities on the data do we need?  Register a new user  Follow/unfollow a user (approve following request)  post/delete a tweet  Read/update in real-time all the tweets from the people I follow  Show the number of tweets I posted, #people following me, #people I follow  Trend information Information required to record System State Operations that update and retrieve System State

Three-Level Architecture  Key question: how to describe data? Conceptual Data Model Logic Data Model Physical Data Model Entities, attributes, relationships (entity-relationship model) Coming next Storage, data structure

Database Model  Logic Data Model: logical structure of data organization  Types of data model  Relational model:  table  Semistructured data model (XML/JSON)  tree  Various data models in NoSQL systems  key-value pair  column-family  graph  Object-oriented model  object, class, inheritance  a layer over relational model

Schema = structural description of relations in database Instance = actual contents at given point in time Schema – structural description of relations in database Instance – data in the database at a given point in time Relational Data Model IDName Password

Schema = structural description of relations in database Instance = actual contents at given point in time Database = set of named relations (or tables) Each relation has a set of named attributes (or columns) Each tuple (or row) has a value for each attribute Each attribute has a type (or domain) Relational Data Model IDName Password

Discussion  How to design relations (tables) for MiniTwitter  What are the aspects we need to consider?

Design – Version 0.1 IDName Password Aadf1234 com qwer6789 IDTimestampAuthorContent Alice00Hello Bob2013Nice weather Not sure.. User Tweet FolloweeFollowerTimestamp Alice00Bob Bob2013Cathy Alice00Cathy Cathy123Alice Bob2013Alice Follow Pretending to be md5 hashcode ;)

Key – attribute whose value is unique in each tuple Or set of attributes whose combined values are unique Relational Data Model IDName Password ail.com Aadf1234 l.com qwer6789 IDtimestampAuthorContent Alice00Hello Bob2013Nice weather Not sure.. User Tweet IDFollowertimestamp Alice00Bob Bob2013Cathy Alice00Cathy Cathy123Alice Bob2013Alice Follow

Key – attribute whose value is unique in each tuple Or set of attributes whose combined values are unique Relational Data Model IDName Password ail.com Aadf1234 l.com qwer6789 IDtimestampAuthorContent Alice00Hello Bob2013Nice weather Not sure.. User Tweet IDFollowertimestamp Alice00Bob Bob2013Cathy Alice00Cathy Cathy123Alice Bob2013Alice Follow

Foreign Key – attribute or set of attributes in one table that point to the primary key of another Relational Data Model IDName Password ail.com Aadf1234 l.com qwer6789 IDtimestampAuthorContent Alice00Hello Bob2013Nice weather Not sure.. User Tweet IDFollowertimestamp Alice00Bob Bob2013Cathy Alice00Cathy Cathy123Alice Bob2013Alice Follow

Foreign Key – attribute or set of attributes in one table that point to the primary key of another Relational Data Model IDName Password ail.com Aadf1234 l.com qwer6789 IDtimestampAuthorContent Alice00Hello Bob2013Nice weather Not sure.. User Tweet IDFollowertimestamp Alice00Bob Bob2013Cathy Alice00Cathy Cathy123Alice Bob2013Alice Follow

More on Relational Data Model  NULL – special value for “unknown” or “undefined”  Relational Model Constraint Summary  Domain constraints  Key constraints  Integrity contraints

Relational Data Model and Database  Relation Model  Simple representation  Efficient implementation  Driven by relational algebra and relational calculus  Up-front definition of schemas and types that the data will thereafter adhere to  High-level simple yet expressive query language  Relational databases  Proven success for both open source and proprietary systems  Provide full ACID guarantees.  SQL as widely used and standard way of database interaction

Creating and Using a Relational Database  Steps in creating and using a (relational) database 1. Design schema (using DDL – data definition language) 2. Initialization: “Bulk load” initial data 3. Operation: execute queries and modifications Data Meta-data: database definition