The Structured Query Language Zachary G. Ives University of Pennsylvania CIS 550 – Database & Information Systems September 28, 2004 Some slide content courtesy of Susan Davidson & Raghu Ramakrishnan

Administrivia Homework 2 handed out today Please sign up for Oracle accounts ASAP http://www.seas.upenn.edu/ora/ Will be useful in testing your HW2’s Google talk at 3:00 this afternoon – Craig Neville-Manning, “Finding needles in a 20 TB haystack, 200 million times per day” Wu and Chen auditorium, Levine 101

Recall Basic SQL SELECT [DISTINCT] {T1.attrib, …, T2.attrib} FROM {relation} T1, {relation} T2, … WHERE {predicates} SELECT * All STUDENTs AS As a “range variable” (tuple variable): optional As an attribute rename operator select-list from-list qualification

Expressions in SQL Can do computation over scalars (int, real or string) in the select-list or the qualification Show all student IDs decremented by 1 Strings: Fixed (CHAR(x)) or variable length (VARCHAR(x)) Use single quotes: ’A string’ Special comparison operator: LIKE Not equal: <> Typecasting: CAST(S.sid AS VARCHAR(255))

Set Operations Set operations default to set semantics, not bag semantics: (SELECT … FROM … WHERE …) {op} (SELECT … FROM … WHERE …) Where op is one of: UNION INTERSECT, MINUS/EXCEPT (many DBs don’t support these last ones!) Bag semantics: ALL

Exercise Find all students who have taken DB but not AI Hint: use EXCEPT

Example Data Instance STUDENT Takes COURSE PROFESSOR Teaches sid name 1 Jill 2 Qun 3 Nitin 4 Marty sid exp-grade cid 1 A 550-0103 700-1003 3 C 500-0103 4 cid subj sem 550-0103 DB F03 700-1003 AI S03 501-0103 Arch PROFESSOR Teaches fid name 1 Ives 2 Saul 8 Roth fid cid 1 550-0103 2 700-1003 8 501-0103

Nested Queries in SQL Simplest: IN/NOT IN Example: Students who have taken subjects that have (at any point) been taught by Roth

Correlated Subqueries Most common: EXISTS/NOT EXISTS Find all students who have taken DB but not AI

Universal and Existential Quantification Generally used with subqueries: {op} ANY, {op} ALL Find the students with the best expected grades

Table Expressions Can substitute a subquery for any relation in the FROM clause: SELECT S.sid FROM (SELECT sid FROM STUDENT WHERE sid = 5) S WHERE S.sid = 4 Notice that we can actually simplify this query! What is this equivalent to?

Aggregation GROUP BY SELECT {group-attribs}, {aggregate-operator}(attrib) FROM {relation} T1, {relation} T2, … WHERE {predicates} GROUP BY {group-list} Aggregate operators AVG, COUNT, SUM, MAX, MIN DISTINCT keyword for AVG, COUNT, SUM

Some Examples Number of students in each course offering Number of different grades expected for each course offering Number of (distinct) students taking AI courses

Example Data Instance STUDENT Takes COURSE PROFESSOR Teaches sid name 1 Jill 2 Qun 3 Nitin 4 Marty sid exp-grade cid 1 A 550-0103 700-1003 3 C 501-0103 4 cid subj sem 550-0103 DB F03 700-1003 AI S03 501-0103 Arch PROFESSOR Teaches fid name 1 Ives 2 Saul 8 Roth fid cid 1 550-0103 2 700-1003 8 501-0103

What If You Want to Only Show Some Groups? The HAVING clause lets you do a selection based on an aggregate (there must be 1 value per group): SELECT C.subj, COUNT(S.sid) FROM STUDENT S, Takes T, COURSE C WHERE S.sid = T.sid AND T.cid = C.cid GROUP BY subj HAVING COUNT(S.sid) > 5 Exercise: For each subject taught by at least two professors, list the minimum expected grade

Aggregation and Table Expressions (aka Derived Relations) Sometimes need to compute results over the results of a previous aggregation: SELECT subj, AVG(size) FROM ( SELECT C.cid AS id, C.subj AS subj, COUNT(S.sid) AS size FROM STUDENT S, Takes T, COURSE C WHERE S.sid = T.sid AND T.cid = C.cid GROUP BY cid, subj) GROUP BY subj

Thought Exercise… Tables are great, but… Not everyone is uniform – I may have a cell phone but not a fax We may simply be missing certain information We may be unsure about values How do we handle these things?

One Answer: Null Values We designate a special “null” value to represent “unknown” or “N/A” But a question: what does: do? Name Home Fax Sam 123-4567 NULL Li 234-8972 234-8766 Maria 789-2312 789-2121 SELECT * FROM CONTACT WHERE Fax < “789-1111”

Three-State Logic Need ways to evaluate boolean expressions and have the result be “unknown” (or T/F) Need ways of composing these three-state expressions using AND, OR, NOT: Can also test for null-ness: attr IS NULL, attr IS NOT NULL Finally: need rules for arithmetic, aggregation T AND U = U F AND U = F U AND U = U T OR U = T F OR U = U U OR U = U NOT U = U

Nulls and Joins Sometimes need special variations of joins: I want to see all courses and their students … But what if there’s a course with no students? Outer join: Most common is left outer join: SELECT C.subj, C.cid, T.sid FROM COURSE C LEFT OUTER JOIN Takes T ON C.cid = T.cid WHERE …

Revised Example Data Instance STUDENT Takes COURSE sid name 1 Jill 2 Qun 3 Nitin 4 Marty sid exp-grade cid 1 A 550-0103 700-1003 3 C 501-0103 4 cid subj sem 550-0103 DB F03 700-1003 AI S03 501-0103 Arch 555-0103 Food PROFESSOR Teaches fid name 1 Ives 2 Saul 8 Roth fid cid 1 550-0103 2 700-1003 8 501-0103

Warning on Outer Join Oracle doesn’t support standard SQL syntax here: SELECT C.subj, C.cid, T.sid FROM COURSE C , Takes T WHERE C.cid =(+) T.cid

Beyond Null Can have much more complex ideas of incomplete or approximate information Probabilistic models (tuple 80% likely to be an answer) Naïve tables (can have variables instead of NULLs) Conditional tables (tuple IF some condition holds) … And what if you want “0 or more”? In relational databases, create a new table and foreign key But can have semistructured data (like XML)

Modifying the Database: Inserting Data Inserting a new literal tuple is easy, if wordy: INSERT INTO PROFESSOR(fid, name) VALUES (4, ‘Simpson’) But we can also insert the results of a query! INSERT INTO PROFESSOR(fid, name) SELECT sid AS fid, name FROM STUDENT WHERE sid < 20

Deleting Tuples Deletion is a fairly simple operation: DELETE FROM STUDENT S WHERE S.sid < 25

Updating Tuples What kinds of updates might you want to do? UPDATE STUDENT S SET S.sid = 1 + S.sid, S.name = ‘Janet’ WHERE S.name = ‘Jane’

Now, How Do I Talk to the DB? Generally, apps are in a different (“host”) language with embedded SQL statements Static: SQLJ, embedded SQL in C Runtime: ODBC, JDBC, ADO, OLE DB, … Typically, predefined mappings between host language types and SQL types (e.g., VARCHAR  string or char[])

Embedded SQL in C EXEC SQL BEGIN DECLARE SECTION int sid; char name[20]; EXEC SQL END DECLARE SECTION … EXEC SQL INSERT INTO STUDENT VALUES (:sid, :name); EXEC SQL SELECT name, age INTO :sid, :name FROM STUDENT WHERE sid < 20

The Impedance Mismatch and Cursors SQL is set-oriented – it returns relations There’s no relation type in most languages! Solution: cursor that’s opened, read DECLARE sinfo CURSOR FOR SELECT sid, name FROM STUDENT … OPEN sinfo; while (…) { FETCH sinfo INTO :sid, :name … } CLOSE sinfo;

JDBC: Dynamic SQL Roughly speaking, a Java version of ODBC See Chapter 6 of the text for more info import java.sql.*; Connection conn = DriverManager.getConnection(…); PreparedStatement stmt = conn.prepareStatement(“SELECT * FROM STUDENT”); … ResultSet rs = stmt.executeQuery (); while (rs.next()) { sid = rs.getInteger(1); … }

Database-Backed Web Sites We all know traditional static HTML web sites: Web-Browser Web-Server HTTP-Request GET ... HTML-File File-System Load File All pages are static Need to generate web pages on the fly depending on user input

Common Gateway Interface (CGI) Can have the web server invoke code (with parameters) to generate HTML Web Server Web Server File-System HTTP-Request Load File HTML HTML? HTML-File Execute Program Program? Output File I/O, Network, DB

CGI: Discussion Advantages: Disadvantages: Standardized: works for every web-server, browser Flexible: Any language (C++, Perl, Java, …) can be used Disadvantages: Statelessness: query-by-query approach Inefficient: new process forked for every request Security: CGI programmer is responsible for security Updates: To update layout, one has to be a programmer

DB Access in Java Browser JVM Java Applet Java-Server-Process TCP/UDP IP Java-Server-Process JDBC Driver manager JDBC-Driver JDBC-Driver JDBC-Driver Sybase Oracle ...

Java Applets: Discussion Advantages: Can take advantage of client processing Platform independent – assuming standard java Disadvantages: Requires JVM on client; self-contained Inefficient: loading can take a long time ... Resource intensive: Client needs to be state of the art Restrictive: can only connect to server where applet was loaded from (for security … can be configured)

*SP Server Pages and Servlets (IIS, Tomcat, …) Web Server Web Server HTTP Request File-System Load File HTML HTML? HTML File File Script? Output Server Extension I/O, Network, DB

One Step Beyond: DB-Driven Web Sites (Strudel, Cocoon, …) Web Server DB-Driven Web Server HTTP Request Styles Cache HTML File HTML Script? Local Database Dynamic HTML Generation Data Other data sources

Wrapping Up We’ve seen how to query in SQL (DML) Basic foundation is TRC-based Subqueries and aggregation add extra power Nulls and outer joins add flexibility of representation We can update tables We’ve seen that SQL doesn’t precisely match standard host language semantics Embedded SQL Dynamic SQL Data-driven web sites