1. 1 Introduction to SQL Select-From-Where Statements Subqueries Grouping and Aggregation Source: slides by Jeffrey Ullman

2. 2 Why SQL? uSQL is a very-high-level language. wSay “what to do” rather than “how to do it.” wAvoid a lot of data-manipulation details needed in procedural languages like C++ or Java. uDatabase management system figures out “best” way to execute query. wCalled “query optimization.”

3. 3 Select-From-Where Statements SELECT desired attributes FROM one or more tables WHERE condition about tuples of the tables

4. 4 Our Running Example uAll our SQL queries will be based on the following database schema. wUnderline indicates key attributes. Candies(name, manf) Stores(name, addr, license) Consumers(name, addr, phone) Likes(consumer, candy) Sells(store, candy, price) Frequents(consumer, store)

5. 5 Example uUsing Candies(name, manf), what candies are made by Hershey? SELECT name FROM Candies WHERE manf = ’Hershey’; Notice SQL uses single-quotes for strings. SQL is case-insensitive, except inside strings.

6. 6 Result of Query name Twizzler Kitkat AlmondJoy... The answer is a relation with a single attribute, name, and tuples with the name of each candy by Hershey, such as Twizzler.

7. 7 Meaning of Single-Relation Query uBegin with the relation in the FROM clause. uApply the selection indicated by the WHERE clause. uApply the extended projection indicated by the SELECT clause.

8. 8 Operational Semantics uTo implement this algorithm think of a tuple variable (tv) ranging over each tuple of the relation mentioned in FROM. uCheck if the “current” tuple satisfies the WHERE clause. uIf so, compute the attributes or expressions of the SELECT clause using the components of this tuple.

9. 9 Operational Semantics Check if Hershey namemanf Twizzler Hershey tv Include tv.name in the result

10. 10 * In SELECT clauses uWhen there is one relation in the FROM clause, * in the SELECT clause stands for “all attributes of this relation.” uExample using Candies(name, manf): SELECT * FROM Candies WHERE manf = ’Hershey’;

11. 11 Result of Query: namemanf TwizzlerHershey KitkatHershey AlmondJoyHershey... Now, the result has each of the attributes of Candies.

12. 12 Renaming Attributes uIf you want the result to have different attribute names, use “AS ” to rename an attribute. uExample based on Candies(name, manf): SELECT name AS candy, manf FROM Candies WHERE manf = ’Hershey’

13. 13 Result of Query: candymanf TwizzlerHershey KitkatHershey AlmondJoyHershey...

14. 14 Expressions in SELECT Clauses uAny expression that makes sense can appear as an element of a SELECT clause. uExample: from Sells(store, candy, price): SELECT store, candy, price * 114 AS priceInYen FROM Sells;

15. 15 Result of Query storecandypriceInYen 7-11Twizzler285 KrogerSnickers342 … … …

16. 16 Another Example: Constant Expressions uFrom Likes(consumer, candy) : SELECT consumer,’likes Kitkats’ AS whoLikesKitkats FROM Likes WHERE candy = ’Kitkat’;

17. 17 Result of Query consumerwhoLikesKitkats Sallylikes Kitkats Fredlikes Kitkats … …

18. 18 Complex Conditions in WHERE Clause uFrom Sells(store, candy, price), find the price that 7-11 charges for Twizzlers: SELECT price FROM Sells WHERE store = ’7-11’ AND candy = ’Twizzler’;

19. 19 Patterns uWHERE clauses can have conditions in which a string is compared with a pattern, to see if it matches. uGeneral form: LIKE or NOT LIKE uPattern is a quoted string with % = “any string”; _ = “any character.”

20. 20 Example uFrom Consumers(name, addr, phone) find the consumers with exchange 555: SELECT name FROM Consumers WHERE phone LIKE ’%555-_ _ _ _’;

21. 21 NULL Values uTuples in SQL relations can have NULL as a value for one or more components. uMeaning depends on context. Two common cases: wMissing value : e.g., we know 7-11 has some address, but we don’t know what it is. wInapplicable : e.g., the value of attribute spouse for an unmarried person.

22. 22 Comparing NULL’s to Values uThe logic of conditions in SQL is really 3- valued logic: TRUE, FALSE, UNKNOWN. uWhen any value is compared with NULL, the truth value is UNKNOWN. uBut a query only produces a tuple in the answer if its truth value for the WHERE clause is TRUE (not FALSE or UNKNOWN).

23. 23 Three-Valued Logic uTo understand how AND, OR, and NOT work in 3-valued logic, think of TRUE = 1, FALSE = 0, and UNKNOWN = ½. uAND = MIN; OR = MAX, NOT(x) = 1-x. uExample: TRUE AND (FALSE OR NOT(UNKNOWN)) = MIN(1, MAX(0, (1 - ½ ))) = MIN(1, MAX(0, ½ ) = MIN(1, ½ ) = ½ = UNKNOWN.

24. 24 Surprising Example uFrom the following Sells relation: storecandyprice 7-11TwizzlerNULL SELECT store FROM Sells WHERE price = 2.00; UNKNOWN

25. 25 Reason: 2-Valued Laws != 3-Valued Laws uSome common laws, like commutativity of AND, hold in 3-valued logic. uBut not others, e.g., the “law of the excluded middle”: p OR NOT p = TRUE. wWhen p = UNKNOWN, the left side is MAX( ½, (1 – ½ )) = ½ != 1.

26. 26 Multirelation Queries uInteresting queries often combine data from more than one relation. uWe can address several relations in one query by listing them all in the FROM clause. uDistinguish attributes of the same name by “. ”

27. 27 Example uUsing relations Likes(consumer, candy) and Frequents(consumer, store), find the candies liked by at least one person who frequents 7-11. SELECT candy FROM Likes, Frequents WHERE store = ’7-11’ AND Frequents.consumer = Likes.consumer;

28. 28 Formal Semantics uAlmost the same as for single-relation queries: wStart with the product of all the relations in the FROM clause. wApply the selection condition from the WHERE clause. wProject onto the list of attributes and expressions in the SELECT clause.

29. 29 Operational Semantics uImagine one tuple-variable for each relation in the FROM clause. wThese tuple-variables visit each combination of tuples, one from each relation. uIf the tuple-variables are pointing to tuples that satisfy the WHERE clause, send these tuples to the SELECT clause.

30. 30 Example consumer store consumer candy tv1tv2 Sally Twizzler Sally 7-11 Likes Frequents to output check these are equal check for 7-11

31. 31 Explicit Tuple-Variables uSometimes, a query needs to use two copies of the same relation. uDistinguish copies by following the relation name by the name of a tuple- variable, in the FROM clause. uIt’s always an option to rename relations this way, even when not essential.

32. 32 Example uFrom Candies(name, manf), find all pairs of candies by the same manufacturer. wDo not produce pairs like (Twizzler, Twizzler). wProduce pairs in alphabetic order, e.g. (Kitkat, Twizzler), not (Twizzler, Kitkat). SELECT c1.name, c2.name FROM Candies c1, Candies c2 WHERE c1.manf = c2.manf AND c1.name < c2.name; tuple variables

33. 33 Subqueries uA parenthesized SELECT-FROM-WHERE statement (subquery ) can be used as a value in a number of places, including FROM and WHERE clauses. uExample: in place of a relation in the FROM clause, we can place another query, and then query its result. wCan use a tuple-variable to name tuples of the result.

34. 34 Subqueries That Return One Tuple uIf a subquery is guaranteed to produce one tuple, then the subquery can be used as a value. wUsually, the tuple has one component. wA run-time error occurs if there is no tuple or more than one tuple.

35. 35 Example uFrom Sells(store, candy, price), find the stores that sell Kitkats for the same price 7-11 charges for Twizzlers. uTwo queries would surely work: wFind the price 7-11 charges for Twizzlers. wFind the stores that sell Kitkats at that price.

36. 36 Query + Subquery Solution SELECT store FROM Sells WHERE candy = ’Kitkat’ AND price = (SELECT price FROM Sells WHERE store= ’7-11’ AND candy = ’Twizzler’); The price at which 7-11 sells Twizzlers

37. 37 The IN Operator u IN is true if and only if the tuple is a member of the relation. w NOT IN means the opposite. uIN-expressions can appear in WHERE clauses. uThe is often a subquery.

38. 38 Example uFrom Candies(name, manf) and Likes(consumer, candy), find the name and manufacturer of each candy that Fred likes. SELECT * FROM Candies WHERE name IN (SELECT candy FROM Likes WHERE consumer = ’Fred’); The set of candies Fred likes

39. 39 The Exists Operator uEXISTS( ) is true if and only if the is not empty. uExample: From Candies(name, manf), find those candies that are the unique candy by their manufacturer.

40. 40 Example Query with EXISTS SELECT name FROM Candies c1 WHERE NOT EXISTS( SELECT * FROM Candies WHERE manf = c1.manf AND name <> c1.name); Set of candies with the same manf as c1, but not the same candy Notice scope rule: manf refers to closest nested FROM with a relation having that attribute. Notice the SQL “not equals” operator

41. 41 The Operator ANY ux = ANY( ) is a boolean condition true if x equals at least one tuple in the relation. uSimilarly, = can be replaced by any of the comparison operators. uExample: x > ANY( ) means x is not the smallest tuple in the relation. wNote tuples must have one component only.

42. 42 The Operator ALL uSimilarly, x <> ALL( ) is true if and only if for every tuple t in the relation, x is not equal to t. wThat is, x is not a member of the relation. uThe <> can be replaced by any comparison operator. uExample: x >= ALL( ) means there is no tuple larger than x in the relation.

43. 43 Example uFrom Sells(store, candy, price), find the candies sold for the highest price. SELECT candy FROM Sells WHERE price >= ALL( SELECT price FROM Sells); price from the outer Sells must not be less than any price.

44. 44 Union, Intersection, and Difference uUnion, intersection, and difference of relations are expressed by the following forms, each involving subqueries: w( subquery ) UNION ( subquery ) w( subquery ) INTERSECT ( subquery ) w( subquery ) EXCEPT ( subquery )

45. 45 Example uFrom relations Likes(consumer, candy), Sells(store, candy, price), and Frequents(consumer, store), find the consumers and candies such that: wThe consumer likes the candy, and wThe consumer frequents at least one store that sells the candy.

46. 46 Solution (SELECT * FROM Likes) INTERSECT (SELECT consumer, candy FROM Sells, Frequents WHERE Frequents.store = Sells.store ); The consumer frequents a store that sells the candy.

47. 47 Bag Semantics uAlthough the SELECT-FROM-WHERE statement uses bag semantics, the default for union, intersection, and difference is set semantics. wThat is, duplicates are eliminated as the operation is applied.

48. 48 Motivation: Efficiency uWhen doing projection, it is easier to avoid eliminating duplicates. wJust work tuple-at-a-time. uFor intersection or difference, it is most efficient to sort the relations first. wAt that point you may as well eliminate the duplicates anyway.

49. 49 Controlling Duplicate Elimination uForce the result to be a set by SELECT DISTINCT... uForce the result to be a bag (i.e., don’t eliminate duplicates) by ALL, as in... UNION ALL...

50. 50 Example: DISTINCT uFrom Sells(store, candy, price), find all the different prices charged for candies: SELECT DISTINCT price FROM Sells; uNotice that without DISTINCT, each price would be listed as many times as there were store/candy pairs at that price.

51. 51 Example: ALL uUsing relations Frequents(consumer, store) and Likes(consumer, candy): (SELECT consumer FROM Frequents) EXCEPT ALL (SELECT consumer FROM Likes); uLists consumers who frequent more stores than they like candies, and does so as many times as the difference of those counts.

52. 52 Join Expressions uSQL provides several versions of (bag) joins. uThese expressions can be stand-alone queries or used in place of relations in a FROM clause.

53. 53 Products and Natural Joins uNatural join: R NATURAL JOIN S; uProduct: R CROSS JOIN S; uExample: Likes NATURAL JOIN Sells; uRelations can be parenthesized subqueries, as well.

54. 54 Theta Join uR JOIN S ON uExample: using Consumers(name, addr) and Frequents(consumer, store): Consumers JOIN Frequents ON name = consumer; gives us all (c, a, c, s) quadruples such that consumer c lives at address a and frequents store s.

55. 55 Outerjoins uR OUTER JOIN S is the core of an outerjoin expression. It is modified by: 1.Optional NATURAL in front of OUTER. 2.Optional ON after JOIN. 3.Optional LEFT, RIGHT, or FULL before OUTER. uLEFT = pad dangling tuples of R only. uRIGHT = pad dangling tuples of S only. uFULL = pad both; this choice is the default.

56. 56 Aggregations uSUM, AVG, COUNT, MIN, and MAX can be applied to a column in a SELECT clause to produce that aggregation on the column. uAlso, COUNT(*) counts the number of tuples.

57. 57 Example: Aggregation uFrom Sells(store, candy, price), find the average price of Twizzlers: SELECT AVG(price) FROM Sells WHERE candy = ’Twizzler’;

58. 58 Eliminating Duplicates in an Aggregation uUse DISTINCT inside an aggregation. uExample: find the number of different prices charged for Twizzlers: SELECT COUNT(DISTINCT price) FROM Sells WHERE candy = ’Twizzler’;

59. 59 NULL’s Ignored in Aggregation uNULL never contributes to a sum, average, or count, and can never be the minimum or maximum of a column. uBut if there are no non-NULL values in a column, then the result of the aggregation is NULL.

60. 60 Example: Effect of NULL’s SELECT count(*) FROM Sells WHERE candy = ’Twizzler’; SELECT count(price) FROM Sells WHERE candy = ’Twizzler’; The number of stores that sell Twizzlers. The number of stores that sell Twizzlers at a known price.

61. 61 Grouping uWe may follow a SELECT-FROM- WHERE expression by GROUP BY and a list of attributes. uThe relation that results from the SELECT-FROM-WHERE is grouped according to the values of all those attributes, and any aggregation is applied only within each group.

62. 62 Example: Grouping uFrom Sells(store, candy, price), find the average price for each candy: SELECT candy, AVG(price) FROM Sells GROUP BY candy;

63. 63 Example: Grouping uFrom Sells(store, candy, price) and Frequents(consumer, store), find for each consumer the average price of Twizzlers at the stores they frequent: SELECT consumer, AVG(price) FROM Frequents, Sells WHERE candy = ’Twizzler’ AND Frequents.store = Sells.store GROUP BY consumer; Compute consumer- store-price for Twiz. tuples first, then group by consumer.

64. 64 Restriction on SELECT Lists With Aggregation uIf any aggregation is used, then each element of the SELECT list must be either: 1.Aggregated, or 2.An attribute on the GROUP BY list.

65. 65 Illegal Query Example uYou might think you could find the store that sells Twizzlers the cheapest by: SELECT store, MIN(price) FROM Sells WHERE candy = ’Twizzler’; uBut this query is illegal in SQL.

66. 66 HAVING Clauses uHAVING may follow a GROUP BY clause. uIf so, the condition applies to each group, and groups not satisfying the condition are eliminated.

67. 67 Example: HAVING uFrom Sells(store, candy, price) and Candies(name, manf), find the average price of those candies that are either sold in at least three stores or are manufactured by Nestle.

68. 68 Solution SELECT candy, AVG(price) FROM Sells GROUP BY candy HAVING COUNT(store) >= 3 OR candy IN (SELECT name FROM Candies WHERE manf = ’Nestle’); Candies manufactured by Nestle. Candy groups with at least 3 non-NULL stores and also candy groups where the manufacturer is Nestle.

69. 69 Requirements on HAVING Conditions uThese conditions may refer to any relation or tuple-variable in the FROM clause. uThey may refer to attributes of those relations, as long as the attribute makes sense within a group; i.e., it is either: wA grouping attribute, or wAggregated.