1. M.P. Johnson, DBMS, Stern/NYU, Spring 20081 C20.0046: Database Management Systems Lecture #12 M.P. Johnson Stern School of Business, NYU Spring, 2008

2. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 2 Agenda Last time: nulls, grouping & aggregation Today: creating tables, modifications, etc.

3. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 3 Null/logic review TRUE AND UNKNOWN = ? TRUE OR UNKNOWN = ? UNKNOWN OR UNKNOWN = ? X = NULL = ?

4. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 4 HAVING clauses Sometimes want to limit which rows may be grouped Q: How many mins. of film did each rich producer make?  Old = made movies before 1930 Q: Is HAVING necessary here? SELECT name, sum(length) total FROM Movie, MovieExec WHERE producerSsn = ssn GROUP BY name HAVING min(year) < 1930 SELECT name, sum(length) total FROM Movie, MovieExec WHERE producerSsn = ssn GROUP BY name HAVING min(year) < 1930

5. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 5 Review Examples from sqlzoo.netsqlzoo.net SELECT L FROM R 1, …, R n WHERE C SELECT L FROM R 1, …, R n WHERE C  L (  C (R 1 x … R n )

6. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 6 New topic: Modifications Three kinds of modifications 1. Insertions 2. Deletions 3. Updates Sometimes “update” used as a synonym for “modification”

7. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 7 Insertions General form: Missing attribute  NULL (or other default value) Example: Insert a new purchase to the database: INSERT INTO R(A1,…., An) VALUES(v1,….,vn) INSERT INTO Knights(name, britnatl, title) VALUES('Bill Gates', 'n', 'KBE')

8. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 8 Insertions If we’re sure we have all values in the right order, can just say: Only do this if you’re sure of order in which the table fields were defined INSERT INTO R VALUES(v1,….,vn) INSERT INTO Knights VALUES('R. Giuliani', 'n', 'KBE'); INSERT INTO Knights VALUES('Bernard Kerik', 'n', 'CBE'); INSERT INTO Knights VALUES('R. Giuliani', 'n', 'KBE'); INSERT INTO Knights VALUES('Bernard Kerik', 'n', 'CBE');

9. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 9 Insertion the result of a query Product(name, etc.) Purchase(buyerssn, prodName, etc.) Maybe some purchases name missing products  add those to the Product table Subquery replaces VALUES INSERT INTO R(As) (query)

10. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 10 Insertion example Product(name, listPrice, category) Purchase(prodName, buyerName, price) Premise: data corruption  lose some Product data  every product referred to in Purchase should exist in Product, but some are missing namelistPricecategory Canon D101000Camera Canon D202000Camera prodNamebuyerName Canon D10Bill Canon D10Hilary Canon D20George ProductPurchase

11. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 11 Insertion example namelistPricecategory Canon D10NULL Canon D20NULL namelistPricecategory ProductProduct’ prodNamebuyerName Canon D10Bill Canon D20Hilary Canon D20George Purchase Canon D20NULL Q: Or do we get: A: Depends on implementation! INSERT INTO Product(name) SELECT prodName FROM Purchase WHERE prodName NOT IN (SELECT name FROM Product) INSERT INTO Product(name) SELECT prodName FROM Purchase WHERE prodName NOT IN (SELECT name FROM Product)

12. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 12 Deletions General form: E.g.: As usual, WHERE can contain subqueries  Depending on the DBMS Q: How do you delete just one row with SQL simpliciter?  Oracle has the ROWID/ROWNUM pseudo-field… DELETE FROM Table WHERE condition DELETE FROM Table WHERE condition INSERT INTO Knights VALUES('R. Giuliani', 'n', 'KBE'); INSERT INTO Knights VALUES('Bernard Kerik', 'n', 'CBE'); DELETE FROM Knights WHERE name = 'Bernard Kerik'; INSERT INTO Knights VALUES('R. Giuliani', 'n', 'KBE'); INSERT INTO Knights VALUES('Bernard Kerik', 'n', 'CBE'); DELETE FROM Knights WHERE name = 'Bernard Kerik';

13. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 13 Updates General form: Example: As usual, WHERE can contain subqueries UPDATE Product SET field1 = value1, field2 = value2 WHERE condition UPDATE Product SET field1 = value1, field2 = value2 WHERE condition UPDATE Product SET price = price/2 WHERE Product.name IN (SELECT product FROM Purchase WHERE Date = DATE'Oct, 25, 1999') UPDATE Product SET price = price/2 WHERE Product.name IN (SELECT product FROM Purchase WHERE Date = DATE'Oct, 25, 1999')

14. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 14 Next: defining schemata So far, have done queries and data manipulation Now doing data definition Recall data types:  INT or INTEGER (variant: SHORTINT)  FLOAT or REAL: floating-point numbers DOUBLE PRECISION: DECIMAL(n,d):  E.g. decimal(5,2): five decimal digits, with the decimal point two positions from the right: e.g. 123.45  DATE and TIME  Character strings Fixed length: CHAR(n) Variable length: VARCHAR(n)

15. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 15 Creating tables Form: E.g.: CREATE TABLE Table-name ( field1 field-type, field2 field-type, … fieldn field-type ) CREATE TABLE Table-name ( field1 field-type, field2 field-type, … fieldn field-type ) No comma! CREATE TABLE People ( name VARCHAR(30), ssn CHAR(9), age INT, city VARCHAR(30), gender CHAR(1), dob DATE ) CREATE TABLE People ( name VARCHAR(30), ssn CHAR(9), age INT, city VARCHAR(30), gender CHAR(1), dob DATE )

16. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 16 Default Values Specify defaults when creating table: The default default: NULL CREATE TABLE People ( name VARCHAR(30), ssn CHAR(9), age SHORTINT DEFAULT 100, city VARCHAR(30) DEFAULT 'New York', gender BIT(1), dob DATE DEFAULT DATE '1900-01-01' ) CREATE TABLE People ( name VARCHAR(30), ssn CHAR(9), age SHORTINT DEFAULT 100, city VARCHAR(30) DEFAULT 'New York', gender BIT(1), dob DATE DEFAULT DATE '1900-01-01' )

17. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 17 Deleting and modifying schemata Delete data, indices, schema: Delete data and indices: Either way, exercise extreme caution! Add or delete attributes: Q: What’s put in the new fields? DROP TABLE Person TRUNCATE TABLE Person ALTER TABLE Person ADD phone CHAR(12) ALTER TABLE Person ADD phone CHAR(12) ALTER TABLE Person DROP age ALTER TABLE Person DROP age

18. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 18 Constraints in SQL Constraints in SQL: Keys foreign keys Attribute-level constraints Tuple-level constraints Global constraints: assertions The more complex the constraint, the harder it is to check and to enforce simplest Most complex

19. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 19 Constraints in SQL A constraint = a property that we’d like our database to hold The system will enforce the constraint by taking some actions:  forbid an update  or perform compensating updates

20. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 20 Primary Keys & Unique Keys There is at most one PRIMARY KEY; there can be many UNIQUE  Primary key v. candidate keys Key fields get indices automatically (why?) CREATE TABLE Product ( productID CHAR(10), name CHAR(30), category VARCHAR(20), price INT, PRIMARY KEY (productID), UNIQUE (name, category) ) CREATE TABLE Product ( productID CHAR(10), name CHAR(30), category VARCHAR(20), price INT, PRIMARY KEY (productID), UNIQUE (name, category) )

21. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 21 Keys Or: CREATE TABLE Product ( name CHAR(30) PRIMARY KEY, category VARCHAR(20) ) CREATE TABLE Product ( name CHAR(30) PRIMARY KEY, category VARCHAR(20) ) CREATE TABLE Product ( name CHAR(30), category VARCHAR(20) PRIMARY KEY (name) ) CREATE TABLE Product ( name CHAR(30), category VARCHAR(20) PRIMARY KEY (name) )

22. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 22 Keys with Multiple Attributes NameCategoryPrice Canon D10Camera1200 Canon D20Camera2000 iPod G4MP3 Player250 Canon D20Camera800 CREATE TABLE Product ( name CHAR(30), category VARCHAR(20), price INT, PRIMARY KEY (name, category) ) CREATE TABLE Product ( name CHAR(30), category VARCHAR(20), price INT, PRIMARY KEY (name, category) )

23. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 23 Foreign Key Constraints prodName is a foreign key to Product(name) name should be a key in Product Purchase ~ Product is many-one NB: referenced field specified with parens, not dot Referenced and referring fields should/must be indexed (why?) CREATE TABLE Purchase ( prodName CHAR(30) REFERENCES Product(name), date DATETIME ) CREATE TABLE Purchase ( prodName CHAR(30) REFERENCES Product(name), date DATETIME ) Referential integrity in SQL

24. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 24 NameCategory Canon D10Camera Canon D20Camera iPod 4GMP3 Player ProdNameStore Canon D10Wiz Canon D10Wiz Canon D20Best Buy ProductPurchase

25. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 25 Foreign Key Constraints Or: (name, category) must be a key (primary/unique) in Product (why?) CREATE TABLE Purchase ( prodName CHAR(30), category VARCHAR(20), date DATETIME, FOREIGN KEY (prodName, category) REFERENCES Product(name, category) CREATE TABLE Purchase ( prodName CHAR(30), category VARCHAR(20), date DATETIME, FOREIGN KEY (prodName, category) REFERENCES Product(name, category)

26. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 26 NameCategory Canon D10Camera Canon D20Camera iPod 4GMP3 Player ProdNameStore Canon D10Wiz Canon D10Wiz Canon D20Best Buy ProductPurchase What happens during updates? Types of updates: In Purchase: insert/update In Product: delete/update

27. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 27 What happens during updates? SQL has three policies for maintaining referential integrity: 1. Reject: violating modifications (default) 2. Cascade: after a delete/update do a delete/update 3. Set-null: set foreign-key field to NULL

28. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 28 Constraints on attributes and tuples Constraints on attributes:  NOT NULL-- obvious meaning... Can combine NOT NULL with foreign key constraints  CHECK condition-- any condition on row itself Some DBMS support subqueries here, but many don’t Constraints on tuples  CHECK condition

29. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 29 CREATE TABLE Purchase ( prodName CHAR(30) CHECK (prodName IN SELECT Product.name FROM Product), date DATETIME NOT NULL ) CREATE TABLE Purchase ( prodName CHAR(30) CHECK (prodName IN SELECT Product.name FROM Product), date DATETIME NOT NULL ) How is this different from a Foreign Key? Tuple-level constraint

30. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 30 General Assertions Defined in latest SQL standard, but isn’t actually implemented Approximated in MySQL and Oracle as stored procedures  PL/SQL in Oracle CREATE ASSERTION myAssert CHECK (NOT EXISTS( SELECT Product.name FROM Product, Purchase WHERE Product.name = Purchase.prodName GROUP BY Product.name HAVING count(*) > 200) ) CREATE ASSERTION myAssert CHECK (NOT EXISTS( SELECT Product.name FROM Product, Purchase WHERE Product.name = Purchase.prodName GROUP BY Product.name HAVING count(*) > 200) )

31. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 31 New: Indices Important for speeding up query processing Index on field(s) = data structure that makes searches/comparisons on those fields fast Suppose we have a relation Person (name, age, city) Sequential scan of the whole Person file may take a very long time SELECT * FROM Person WHERE name = 'Waksal, Sam' SELECT * FROM Person WHERE name = 'Waksal, Sam'

32. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 32 Creating Indices Syntax: Here: Now searching by name is much faster  How much faster?  Log-time, say  Base-what? doesn’t matter, but say 2 If all New Yorkers, #comparisons:  8000000  log 2 (8000000) ~= 23  (i.e., 2 23 ~= 8000000) CREATE INDEX index-name ON R(fields) CREATE INDEX nameIndex ON Person(name)

33. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 33 How do indices work? What the data structure?  Different possibilities 1 st intuition: index on field f is an ordered list of all values in the table’s f field  each item has address (“rowid”) of its row Where do we get the ordered list? 2 nd intuition: put all f values in a BST  searching BST takes log time (why?) DBMSs actually use a variant: B+Tree  Later…

34. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 34 Creating Indices Indexes can be useful in range queries too: CREATE INDEX ageIndex ON Person (age) SELECT * FROM Person WHERE age > 25

35. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 35 Using indices Indices can be created on multiple attributes: Helps in: And in: But not in: Idea: our sorted list is sorted on age;city, not city;age Q: In Movie, should index be on year;title or title;year? CREATE INDEX doubleNdx ON Person (lname, fname) SELECT * FROM Person WHERE fname='Sam' AND lname = 'Waksal' SELECT * FROM Person WHERE lname='Waksal' SELECT * FROM Person WHERE fname='Sam'

36. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 36 The Index Selection Problem Indices speed up queries dramatically, so… Why not just index all (sequences of) fields?  how does the list/B+Tree stay up to date? Given a workload: a set of SQL queries and their freqs Q: What indices will speed up the workload? Answers:  Attributes in WHERE clauses (queries)  favor an index  Attributes in INSERT/UPDATE/DELETEs  discourage an index  In many DBMSs: your primary/foregin key fields get indexed automatically (why?)

37. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 37 Next: Views Stored relations physically exist and persist Views are relations that don’t  in some texts, “table” = stored relation = “base table” Basically names/references given to queries  maybe a relevant subset of a table Employee(ssn, name, department, project, salary) Payroll has access to Employee, others only to Developers CREATE VIEW Developers AS SELECT name, project FROM Employee WHERE department = 'Dev' CREATE VIEW Developers AS SELECT name, project FROM Employee WHERE department = 'Dev'

38. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 38 A Different View Person(name, city) Purchase(buyer, seller, product, store) Product(name, maker, category) We have a new virtual table: NYCview(buyer, seller, product, store) CREATE VIEW NYCview AS SELECT buyer, seller, product, store FROM Person, Purchase WHERE Person.city = 'New York' AND Person.name = Purchase.buyer CREATE VIEW NYCview AS SELECT buyer, seller, product, store FROM Person, Purchase WHERE Person.city = 'New York' AND Person.name = Purchase.buyer

39. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 39 A Different View Now we can query the view: CREATE VIEW NYCview AS SELECT buyer, seller, product, store FROM Person, Purchase WHERE Person.city = 'New York' AND Person.name = Purchase.buyer CREATE VIEW NYCview AS SELECT buyer, seller, product, store FROM Person, Purchase WHERE Person.city = 'New York' AND Person.name = Purchase.buyer SELECT name, NYCview.store FROM NYCview, Product WHERE NYCview.product = Product.name AND Product.category = 'Camera' SELECT name, NYCview.store FROM NYCview, Product WHERE NYCview.product = Product.name AND Product.category = 'Camera'

40. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 40 What happens when we query a view? SELECT name, NYCview.store FROM NYCview, Product WHERE NYCview.product = Product.name AND Product.category = 'Camera' SELECT name, NYCview.store FROM NYCview, Product WHERE NYCview.product = Product.name AND Product.category = 'Camera' SELECT name, Purchase.store FROM Person, Purchase, Product WHERE Person.city = 'New York' AND Person.name = Purchase.buyer AND Purchase.poduct = Product.name AND Product.category = 'Camera' SELECT name, Purchase.store FROM Person, Purchase, Product WHERE Person.city = 'New York' AND Person.name = Purchase.buyer AND Purchase.poduct = Product.name AND Product.category = 'Camera'

41. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 41 Can rename view fields CREATE VIEW NYCview(NYCbuyer, NYCseller, prod, store) AS SELECT buyer, seller, product, store FROM Person, Purchase WHERE Person.city = 'New York' AND Person.name = Purchase.buyer CREATE VIEW NYCview(NYCbuyer, NYCseller, prod, store) AS SELECT buyer, seller, product, store FROM Person, Purchase WHERE Person.city = 'New York' AND Person.name = Purchase.buyer

42. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 42 Types of Views Views discussed here:  Used in databases  Computed only on-demand – slow at runtime  Always up to date Sometimes talk about “materialized” views  Used in data warehouses  Pre-computed offline – fast at runtime  May have stale data  Maybe more later…

43. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 43 Updating Views How to insert a tuple into a table that doesn’t exist? Employee(ssn, name, department, project, salary) If we make the following insertion: It becomes: CREATE VIEW Developers AS SELECT name, project FROM Employee WHERE department = 'Development' CREATE VIEW Developers AS SELECT name, project FROM Employee WHERE department = 'Development' INSERT INTO Developers VALUES('Bill', ‘Word') INSERT INTO Employee(ssn, name, dept, project, sal) VALUES(NULL, 'Bill', NULL, 'Word', NULL) INSERT INTO Employee(ssn, name, dept, project, sal) VALUES(NULL, 'Bill', NULL, 'Word', NULL)

44. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 44 Non-Updatable Views Person(name, city) Purchase(buyer, seller, product, store) How can we add the following tuple to the view? ('NYC', 'The Wiz') We don’t know the name of the person who made the purchase cannot set to NULL (why?) CREATE VIEW CityStore AS SELECT Person.city, Purchase.store FROM Person, Purchase WHERE Person.name = Purchase.buyer CREATE VIEW CityStore AS SELECT Person.city, Purchase.store FROM Person, Purchase WHERE Person.name = Purchase.buyer

45. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 45 Finally: R.A./SQL has limitations Can easily find Alice’s direct subordinates: But: find all of King’s underlings Cannot compute “transitive closure” Cannot express in R.A./SQL! SQL is not “Turing-Complete” NameJobBoss KingPresidentNULL JonesManagerKing BlakeManagerKing FordAnalystJones ScottAnalystJones

46. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 46 Midterm Know all steps of project so far:  Draw E/R from dataset description  Convert E/R to relations  Normalize badly designed relations Indices:  Why important  How BSTs work Constraints  Why important  Basic concepts SQL!