Advanced SQL Zachary G. Ives University of Pennsylvania CIS 550 – Database & Information Systems September 29, 2005 Some slide content courtesy of Susan Davidson & Raghu Ramakrishnan

2 Administrivia  Homework 2 due Tuesday  Further project details will be given out next week

3 Aggregation  GROUP BY: SELECT {group-attribs}, {aggregate-operator}(attrib) FROM {relation} T 1, {relation} T 2, … WHERE {predicates} GROUP BY {group-list}  Aggregate operators  AVG, COUNT, SUM, MAX, MIN  DISTINCT keyword for AVG, COUNT, SUM  Can do COUNT(*)  After the GROUP BY, only {group-list} or aggregated values are accessible!

4 Some Examples  Number of students in each course offering  Number of different grades expected for each course offering  Number of (distinct) students taking AI courses

5 Revised Example Data Instance sidname 1Jill 2Qun 3Nitin 4Marty fidname 1Ives 2Saul 8Martin sidexp-gradecid 1A A A 3C C cidsubjsem DBF AIS ArchF SysS06 fidcid STUDENT Takes COURSE PROFESSOR Teaches

6 What If You Want to Only Show Some Groups?  The HAVING clause lets you do a selection based on an aggregate (there must be 1 value per group): SELECT C.subj, MIN(T.exp-grade) FROM STUDENT S, Takes T, COURSE C WHERE S.sid = T.sid AND T.cid = C.cid GROUP BY subj HAVING COUNT(DISTINCT S.sid) > 5  Note that you can ONLY use aggregate functions or {group-list} attributes in the HAVING clause  Exercise: For each subject taught by at least two professors, list the minimum expected grade

7 Aggregation and Table Expressions (aka Derived Relations)  Sometimes need to compute results over the results of a previous aggregation: SELECT subj, AVG(size) FROM ( SELECT C.cid AS id, C.subj AS subj, COUNT(S.sid) AS size FROM STUDENT S, Takes T, COURSE C WHERE S.sid = T.sid AND T.cid = C.cid GROUP BY cid, subj) GROUP BY subj

8 Thought Exercise…  Tables are great, but…  Not everyone is uniform – I may have a cell phone but not a fax  We may simply be missing certain information  We may be unsure about values  How do we handle these things?

9 One Answer: Null Values  We designate a special “null” value to represent “unknown” or “N/A”  But a question: what does: do? NameHomeFax Sam NULL Li Maria SELECT * FROM CONTACT WHERE Fax < “ ”

10 Three-State Logic  Need ways to evaluate boolean expressions and have the result be “unknown” (or T/F)  Need ways of composing these three-state expressions using AND, OR, NOT:  Can also test for null-ness: attr IS NULL, attr IS NOT NULL  Finally: need rules for arithmetic, aggregation T AND U = U F AND U = F U AND U = U T OR U = T F OR U = U U OR U = U NOT U = U

11 Nulls and Joins  Sometimes need special variations of joins:  I want to see all courses and their students  … But what if there’s a course with no students?  Outer join:  Most common is left outer join: SELECT C.subj, C.cid, T.sid FROM COURSE C LEFT OUTER JOIN Takes T ON C.cid = T.cid WHERE …

12 Revised Example Data Instance sidname 1Jill 2Qun 3Nitin 4Marty fidname 1Ives 2Saul 8Martin sidexp-gradecid 1A A A 3C C cidsubjsem DBF AIS ArchF SysS06 fidcid STUDENT Takes COURSE PROFESSOR Teaches

13 Warning on Outer Join  Oracle doesn’t support standard SQL syntax here: SELECT C.subj, C.cid, T.sid FROM COURSE C, Takes T WHERE C.cid =(+) T.cid

14 Beyond Null  Can have much more complex ideas of incomplete or approximate information  Probabilistic models (tuple 80% likely to be an answer)  Naïve tables (can have variables instead of NULLs)  Conditional tables (tuple IF some condition holds)  … And what if you want “0 or more”?  In relational databases, create a new table and foreign key  But can have semistructured data (like XML)

15 Modifying the Database: Inserting Data  Inserting a new literal tuple is easy, if wordy: INSERT INTO PROFESSOR(fid, name) VALUES (4, ‘Simpson’)  But we can also insert the results of a query! INSERT INTO PROFESSOR(fid, name) SELECT sid AS fid, name FROM STUDENT WHERE sid < 20

16 Deleting Tuples  Deletion is a fairly simple operation: DELETE FROM STUDENT S WHERE S.sid < 25

17 Updating Tuples  What kinds of updates might you want to do? UPDATE STUDENT S SET S.sid = 1 + S.sid, S.name = ‘Janet’ WHERE S.name = ‘Jane’

18 Now, How Do I Talk to the DB?  Generally, apps are in a different (“host”) language with embedded SQL statements  Static: SQLJ, embedded SQL in C  Runtime: ODBC, JDBC, ADO, OLE DB, …  Typically, predefined mappings between host language types and SQL types (e.g., VARCHAR  String or char[])

19 Embedded SQL in C EXEC SQL BEGIN DECLARE SECTION int sid; char name[20]; EXEC SQL END DECLARE SECTION … EXEC SQL INSERT INTO STUDENT VALUES (:sid, :name); EXEC SQL SELECT name, age INTO :sid, :name FROM STUDENT WHERE sid < 20

20 The Impedance Mismatch and Cursors  SQL is set-oriented – it returns relations  There’s no relation type in most languages!  Solution: cursor that’s opened, read DECLARE sinfo CURSOR FOR SELECT sid, name FROM STUDENT … OPEN sinfo; while (…) { FETCH sinfo INTO :sid, :name … } CLOSE sinfo;

21 JDBC: Dynamic SQL for Java  Roughly speaking, a Java version of ODBC  You’ll likely use this in the course project  See Chapter 6 of the text for more info import java.sql.*; Connection conn = DriverManager.getConnection(…); PreparedStatement stmt = conn.prepareStatement(“SELECT * FROM STUDENT”); … ResultSet rs = stmt.executeQuery(); while (rs.next()) { sid = rs.getInteger(1); … }

22 Database-Backed Web Sites  We all know traditional static HTML web sites: Web-Browser HTTP-Request GET... Web-Server File-System Load File HTML-File

23 DB-Driven Web Server DB-Generated Web Sites (Strudel, Cocoon, …) Local Database HTTP Request HTML File Web Server Cache Data HTML Other data sources Script? Dynamic HTML Generation Styles

24 Procedural Methods for D-HTML: Common Gateway Interface (CGI) Can have the web server invoke code (with parameters) to generate HTML Web Server HTTP-Request HTML-File Web Server File-System Load File File HTML? HTML Execute Program Program?Output I/O, Network, DB

25 HTML Forms

26 CGI Pros and Cons  Advantages:  Standardized: works for every web-server, browser  Flexible: Any language (C++, Perl, Java, …) can be used  Disadvantages:  Statelessness: query-by-query approach  Inefficient: new process forked for every request  Security: CGI programmer is responsible for security  Updates: To update layout, one has to be a programmer  In general, CGI isn’t used very much today

27 Java-Server-Process DB Access with Java Applets and Server Processes Sybase Java Applet TCP/UDP IP Oracle... JDBC- Driver App. Server EJB Layer Browser JVM

28 Java Applets: Discussion  Advantages:  Can take advantage of client processing  Platform independent – assuming standard java  Disadvantages:  Requires JVM on client; self-contained  Inefficient: loading can take a long time...  Resource intensive: Client needs to be state of the art  Restrictive: can only connect to server where applet was loaded from (for security … can be configured)

29 *SP Server Pages, PHP, and Servlets (IIS, Tomcat, WebSphere, WebLogic, …) File-System Web Server HTTP Request HTML File Web Server Load File File HTML? HTML I/O, Network, DB Script? Output Server Extension

30 ASP/JSP/PHP Versus Servlets  The goal: combine direct HTML (or XML) output with program code that’s executed at the server The code is responsible for generating more HTML, e.g., to output the results of a database table as HTML table elements  How might I do this?  HTML with embedded code  Code that prints out HTML

31 ASP/JSP/PHP “Escapes” Sample Sample

32 Servlets class MyClass extends HttpServlet { public void doGet(HttpRequest req, HttpResponse res) … { … res.println(“ Test ”); }

33 Wrapping Up  We’ve seen how to query in SQL (DML)  Basic foundation is TRC-based  Subqueries and aggregation add extra power  Nulls and outer joins add flexibility of representation  We can update tables  We’ve seen that SQL doesn’t precisely match standard host language semantics  Embedded SQL  Dynamic SQL  Data-driven web sites