1. M.P. Johnson, DBMS, Stern/NYU, Spring 20081 C20.0046: Database Management Systems Lecture #7 M.P. Johnson Stern School of Business, NYU Spring, 2008

2. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 2 Agenda Basic SQL Joins Hw1 probably soon (email from Blackboard)…

3. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 3 Current topic: SQL Standard language for querying and manipulating data Structured Query Language Many standards: ANSI SQL, SQL92/SQL2, SQL3/SQL99 Originally: Structured English Query Language (SEQUEL) Vendors support various subsets/extensions We’ll do Oracle/MySQL/generic  “No one ever got fired for buying Oracle.” Basic form (many more bells and whistles in addition): SELECT attributes FROM relations (possibly multiple, joined) WHERE conditions (selections) SELECT attributes FROM relations (possibly multiple, joined) WHERE conditions (selections)

4. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 4 Next (parallel) topic: relational algebra Projection Selection Cartesian Product Joins: natural joins, theta joins Set operations: union, intersection, difference Combining operations to form queries Dependent and independent operations

5. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 5 What is relational algebra? An algebra for relations “High-school” algebra: an algebra for numbers Algebra = formalism for constructing expressions  Operations  Operands: Variables, Constants, expressions Expressions:  Vars & constants  Operators applied to expressions  They evaluate to values AlgebraVars/constsOperatorsEval to High-schoolNumbers+ * - / etc.Numbers RelationalRelations (=sets of tupes) union, intersection, join, etc. Relations

6. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 6 Why do we care about relational algebra? 1. The exprs are the form that questions about the data take  The relations these exprs cash out to are the answers to our questions 2. RA ~ more succinct rep. of many SQL queries 3. DBMS parse SQL into something like RA First proofs of concept for RDBMS/RA:  System R at IBM  Ingress at Berkeley “Modern” implementation of RA: SQL  Both state of the art, mid-70s

7. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 7 Relation operators Basic operators:  Selection:   Projection:   Cartesian Product:  Other set-theoretic ops:  Union:   Intersection:  Difference: - Additional operators:  Joins (natural, equijoin, theta join, semijoin)  Renaming:   Grouping…

8. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 8 Selection op Selects all tuples satisfying a condition Notation:  c (R) Examples   salary > 100000 (Employee)   name = “Smith” (Employee) The condition c can have  comparison ops:=,, , <>  boolean ops: and, or

9. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 9 Selection example Select the movies at Angelica:   Theater=“Sunshine” (Showings) Masc. Fem.VillageFilm Forum Village N’hood Bad Edu. Annie Hall Title Sunshine Theater Village N’hood Bad Edu. Annie Hall Title Sunshine Theater

10. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 10 Projection op Keep only certain columns Projection: op we used for decomposition  Eliminates other columns, then removes duplicates Notation:  A1,…,An (R)

11. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 11 Join op Corresponds to SQL query doing cross & equality test Specifically: R 1 R 2 =  every att once (  shared atts = (R 1  R 2 ))  I.e., first compute the cross product R 1 x R 2  Next, select the rows in which shared fields agree  Finally, project onto the union of R 1 and R 2 ’s fields (remove duplicates)

12. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 12 Rename op Changes the schema, not the instance Notation:  B1,…,Bn (R)  is spelled “rho”, pronounced “row” Example:  Employee(ssn,name)    social, name) (Employee)  Or just:   (Employee)

13. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 13 RA  SQL SQL SELECT  RA Projection  SQL WHERE  RA Selection  SQL FROM  RA Join/cross  Comma-separated list… SQL renaming  RA rho  More ops later Keep RA in the back of your mind…

14. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 14 Next: Joins in SQL Connect two or more tables: PNamePriceCategoryManufacturer Gizmo$19.99GadgetsGizmoWorks Powergizmo$29.99GadgetsGizmoWorks SingleTouch$149.99PhotographyCanon MultiTouch$203.99HouseholdHitachi Product Company CNameStockPriceCountry GizmoWorks25USA Canon65Japan Hitachi15Japan What is the connection between them?

15. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 15 Joins in SQL Product (pname, price, category, manufacturer) Company (cname, stockPrice, country) Find all products under $200 manufactured in Japan; return their names and prices. SELECT PName, Price FROM Product, Company WHERE Manufacturer=CName AND Country='Japan' AND Price <= 200 Join between Product and Company

16. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 16 Joins in SQL PNamePriceCategoryManufacturer Gizmo$19.99GadgetsGizmoWorks Powergizmo$29.99GadgetsGizmoWorks SingleTouch$149.99PhotographyCanon MultiTouch$203.99HouseholdHitachi Product Company CnameStockPriceCountry GizmoWorks25USA Canon65Japan Hitachi15Japan PNamePrice SingleTouch$149.99 SELECT PName, Price FROM Product, Company WHERE Manufacturer=CName AND Country='Japan' AND Price <= 200

17. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 17 Joins in SQL Product (pname, price, category, manufacturer) Company (cname, stockPrice, country) Find all countries that manufacture some product in the ‘Gadgets’ category. SELECT Country FROM Product, Company WHERE Manufacturer=CName AND Category='Gadgets'

18. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 18 Joins in SQL NamePriceCategoryManufacturer Gizmo$19.99GadgetsGizmoWorks Powergizmo$29.99GadgetsGizmoWorks SingleTouch$149.99PhotographyCanon MultiTouch$203.99HouseholdHitachi Product Company CnameStockPriceCountry GizmoWorks25USA Canon65Japan Hitachi15Japan Country ?? What is the problem? What’s the solution? SELECT Country FROM Product, Company WHERE Manufacturer=CName AND Category='Gadgets'

19. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 19 Joins Product (pname, price, category, manufacturer) Purchase (buyer, seller, store, product) Person(name, phone, city) Find names of Seattleites who bought Gadgets, and the names of the stores they bought such product from. SELECT DISTINCT name, store FROM Person, Purchase, Product WHERE persname=buyer AND product = pname AND city='Seattle' AND category='Gadgets'

20. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 20 SQL Query Semantics Parallel assignment – all tuples Doesn’t impose any order Answer = {} for all assignments x1 in R1, …, xn in Rn do if Conditions then Answer = Answer  {(a1,…,ak)} return Answer Answer = {} for all assignments x1 in R1, …, xn in Rn do if Conditions then Answer = Answer  {(a1,…,ak)} return Answer SELECT a1, a2, …, ak FROM R1 AS x1, R2 AS x2, …, Rn AS xn WHERE Conditions SELECT a1, a2, …, ak FROM R1 AS x1, R2 AS x2, …, Rn AS xn WHERE Conditions

21. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 21 SQL Query Semantics Nested loops: Answer = {} for x1 in R1 do for x2 in R2 do ….. for xn in Rn do if Conditions then Answer = Answer  {(a1,…,ak)} return Answer Answer = {} for x1 in R1 do for x2 in R2 do ….. for xn in Rn do if Conditions then Answer = Answer  {(a1,…,ak)} return Answer SELECT a1, a2, …, ak FROM R1 AS x1, R2 AS x2, …, Rn AS xn WHERE Conditions SELECT a1, a2, …, ak FROM R1 AS x1, R2 AS x2, …, Rn AS xn WHERE Conditions

22. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 22 Multiple join syntaxes Old-style syntax simply lists tables separated by commas New-style makes the join explicit: Functionally equivalent to old-style, but perhaps more elegant Introduced in Oracle 8i, MySQL 3.x/4.x Older versions / other DBMSs may not support this SELECT * FROM A,B WHERE …; SELECT * FROM A,B WHERE …; SELECT * FROM A JOIN B ON … WHERE …; SELECT * FROM A JOIN B ON … WHERE …;

23. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 23 New-style join types Cross joins (simplest):  FROM A CROSS JOIN B Inner joins (regular joins):  FROM A [INNER] JOIN B ON … Natural join:  FROM A NATURAL JOIN B;  Joins on common fields and merges Outer joins (later)  No dangling rows

24. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 24 CROSS JOIN e.g. NameAddressGenderBirthdate Hanks123 Palm RdM01/01/60 Taylor456 Maple AvF02/02/40 Lucas789 Oak StM03/03/55 NameAddressNetworth Spielberg246 Palm Rd10M Taylor456 Maple Av20M Lucas789 Oak St30M MovieStar MovieExec

25. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 25 CROSS JOIN e.g. MovieS tar.nam e MovieStar.add ress MovieSta r. Gender MovieStar.Birthdate MovieEx ec. Name MovieExec.Ad dress MovieEx ec. Networth Hanks123 Palm RdM01/01/60Spielberg246 Palm Rd10M Hanks123 Palm RdM01/01/60Taylor456 Maple Av20M Hanks123 Palm RdM01/01/60Lucas789 Oak St30M Taylor456 Maple AvF02/02/40Spielberg246 Palm Rd10M Taylor456 Maple AvF02/02/40Taylor456 Maple Av20M Taylor456 Maple AvF02/02/40Lucas789 Oak St30M Lucas789 Oak StM03/03/55Spielberg246 Palm Rd10M Lucas789 Oak StM03/03/55Taylor456 Maple Av20M Lucas789 Oak StM03/03/55Lucas789 Oak St30M Row 1 2 3 4 5 6 7 8 9 SELECT * FROM MovieStar CROSS JOIN MovieExec SELECT * FROM MovieStar CROSS JOIN MovieExec

26. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 26 JOIN … ON e.g MovieSt ar.name MovieStar.addr ess MovieS tar. Gender MovieStar. Birthdate MovieExec. Name MovieExec.Add ress MovieEx ec. Networth Hanks123 Palm RdM01/01/60Spielberg246 Palm Rd10M Hanks123 Palm RdM01/01/60Taylor456 Maple Av20M Hanks123 Palm RdM01/01/60Lucas789 Oak St30M Taylor456 Maple AvF02/02/40Spielberg246 Palm Rd10M Taylor456 Maple AvF02/02/40Taylor456 Maple Av20M Taylor456 Maple AvF02/02/40Lucas789 Oak St30M Lucas789 Oak StM03/03/55Spielberg246 Palm Rd10M Lucas789 Oak StM03/03/55Taylor456 Maple Av20M Lucas789 Oak StM03/03/55Lucas789 Oak St30M Row 1 2 3 4 5 6 7 8 9 SELECT * FROM MovieStar JOIN MovieExec ON MovieStar.Name <> MovieExec.Name SELECT * FROM MovieStar JOIN MovieExec ON MovieStar.Name <> MovieExec.Name

27. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 27 NATURAL JOIN MovieStar(name, address, gender, birthdate) MovieExec(name, address, networth) Natural Join syntax:  FROM MovieStar NATURAL JOIN MovieExec Results: list of movie stars who are also execs:  (Name, address, gender, birthdate, networth)

28. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 28 NATURAL JOIN e.g. NameAddressGenderBirthdate Hanks123 Palm RdM01/01/60 Taylor456 Maple AvF02/02/40 Lucas789 Oak StM03/03/55 NameAddressNetworth Spielberg246 Palm Rd10M Taylor456 Maple Av20M Lucas789 Oak St30M MovieStar MovieExec NameAddressGenderBirthdateNetworth Taylor456 Maple AvF02/02/4020M Lucas789 Oak StM03/03/5530M SELECT * FROM MovieStar NATURAL JOIN MovieExec

29. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 29 Disambiguating Attributes Sometimes two relations have the same attr: Person(pname, address, worksfor) Company(cname, address) SELECT DISTINCT pname, address FROM Person, Company WHERE worksfor = cname SELECT DISTINCT Person.pname, Company.address FROM Person, Company WHERE Person.worksfor = Company.cname Which address?

30. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 30 Tuple Var e.g. SELECT DISTINCT x.store FROM Purchase AS x, Purchase AS y WHERE x.product = y.product AND y.store = 'BestBuy' SELECT DISTINCT x.store FROM Purchase AS x, Purchase AS y WHERE x.product = y.product AND y.store = 'BestBuy' Find all stores that sold at least one product that the store BestBuy also sold: Result: (store) Product (pname, price, category, manufacturer) Purchase (buyer, seller, store, product) Person(persname, phoneNumber, city)

31. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 31 Details: Disambiguation in SQL Every selected field must be unambiguous For R(A,B),  Select A from R, R   Select R1.A from R R1, R R2 Consider: Prob:* is shorthand for all fields  each must be unambiguous Soln: Select * from R R1, R R2 SQL> Select * from R, R; Select * from R, R * ERROR at line 1: ORA-00918: column ambiguously defined SQL> Select * from R, R; Select * from R, R * ERROR at line 1: ORA-00918: column ambiguously defined

32. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 32 Details: Disambiguation in Oracle SQL Depending on DBMS, can rename fields by:  Select name as n …  Select name n …  Select name=n…(not in Oracle) Can rename relations only by  … from tab t1, tab t2

33. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 33 SQL e.g. with tuple vars Reps(ssn, name, etc.) Clients(ssn, name, rssn) Q: Who are George’s clients, in SQL? Conceptually:   Clients.name (  Reps.name=“George” and Reps.ssn=rssn (Reps x Clients))

34. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 34 Ordering the Results Ordering is ascending, unless you specify the DESC keyword per attribute: SELECT pname, price, manufacturer FROM Product WHERE category='gizmo' AND price > 50 ORDER BY price, pname SELECT pname, price, manufacturer FROM Product WHERE category='gizmo' AND price > 50 ORDER BY price, pname SELECT pname, price, manufacturer FROM Product WHERE category='gizmo' AND price > 50 ORDER BY price DESC, pname ASC SELECT pname, price, manufacturer FROM Product WHERE category='gizmo' AND price > 50 ORDER BY price DESC, pname ASC

35. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 35 Ordering the Results SELECT Category FROM Product ORDER BY PName SELECT Category FROM Product ORDER BY PName PNamePriceCategoryManufacturer Gizmo$19.99GadgetsGizmoWorks Powergizmo$29.99GadgetsGizmoWorks SingleTouch$149.99PhotographyCanon MultiTouch$203.99HouseholdHitachi ?

36. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 36 Details: Case-sensitivity In Oracle, compares are case-sensitive by default If want case-insensitive, some options: 1. Create a function index  Maybe later… 2. Manually convert vals to upper/lower case  SQL> select * from emp where upper(ename) = upper(‘Blake'); 3. Modify the nls_sort setting:  SQL> alter session set nls_sort=binary_ci;  SQL> alter session set nls_comp=ansi;  The other values: binary, binary_ai

37. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 37 The LIKE operator s LIKE p: pattern matching on strings p may contain two special symbols:  _ = any single character  % = zero or more chars Product(Name, Price, Category, Manufacturer) Find all products whose name contains 'gizmo': SELECT * FROM Products WHERE Name LIKE '%gizmo%'

38. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 38 The LIKE operator Q: How to search the actual '%' char?  The usual meta-char issue PName LIKE '%%' won’t work Instead, must use escape chars  In C/C++/J, prepend \  In SQL, prepend an arbitrary escape char: PName LIKE '%x%' ESCAPE 'x'

39. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 39 Details: more on escape chars SQL: no official default escape char In Oracle’s SQL*Plus: default escape char = '\'  Can set with SQL> set escape x  Other tools, DBMSs: your mileage may vary SQL string literals put in ' ':  'mystring' Single-quote literals escaped with single-quotes:  'George''s string' No distinction between strings and single chars

40. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 40 Details: more on escape chars Q: Can an escape char be an escape string? A: No. SQL> select * from newtable where a like '%\%' escape '\'; A B ---------- h%i there SQL> select * from newtable where a like '%\%' escape '\\'; select * from newtable where a like '%\%' escape '\\' * ERROR at line 1: ORA-01425: escape character must be character string of length 1 SQL> select * from newtable where a like '%\%' escape '\'; A B ---------- h%i there SQL> select * from newtable where a like '%\%' escape '\\'; select * from newtable where a like '%\%' escape '\\' * ERROR at line 1: ORA-01425: escape character must be character string of length 1

41. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 41 Details: more on single-quotes Dates with DATE:  DATE '1948-05-14' Timestamps with TIMESTAMP:  TIMESTAMP '1948-05-14 12:00:00' Details may vary by DBMS…

42. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 42 Details: more on quotes Q: What about double quotes? A: Can’t be used in place of single quotes  But are used… But can be used when Oracle would otherwise misparse your command, e.g.: 1. Names with spaces:  create table bad table name (a int, b int); 2. Reserved words as names:  create table badfieldname(from int, b int);

43. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 43 Another complex example People(ssn, name, street, city, state, state) Q: Who lives on George’s street? A: First, generate pairs of (renamed) people:   p1 (People) x  p2 (People) Then pick out pairs with George:   p1.name='George' (  p1 (People) x  p2 (People)) And refine to rows with George and someone else:   p1.name='George‘ AND p1.name<>p2.name (  p1 (People) x  p2 (People)) Finally, project out the names:   p2.name (  p1.name='George‘ AND p1.name<>p2.name (  p1 (People) x  p2 (People))

44. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 44 Live examples Q: produce a list of employees and their bosses  What if no boss? Or no subordinate? Joins on emp, emp man:  Comma-based  Inner  Natural  Cross  Outer – left, right, full

45. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 45 More live examples Inner joins require an ON clause  Like a where clause  Arbitrary boolean expression  If always true (1=1), reduces to cross join New compar op: BETWEEN  a between 5 and 10  a >= 5 and a <= 10 Q: produce a list of employees with their salary grades  emp, salgrade

46. M.P. Johnson, DBMS, Stern/NYU, Spring 2008 46 Review Examples from sqlzoo.netsqlzoo.net SELECT L FROM R 1, …, R n WHERE C SELECT L FROM R 1, …, R n WHERE C  L (  C (R 1 x … R n )