1. 1 Real SQL Programming Embedded SQL Java Database Connectivity Stored Procedures

2. 2 Three-Tier Architecture uA common environment for using a database has three tiers of processors: 1.Web servers --- talk to the user. 2.Application servers --- execute the business logic. 3.Database servers --- get what the app servers need from the database.

3. 3 Example: Amazon uDatabase holds the information about products, customers, etc. uBusiness logic includes things like “what do I do after someone clicks ‘checkout’?” wAnswer: Show the “how will you pay for this?” screen.

4. 4 Environments, Connections, Queries uThe database is, in many DB-access languages, an environment. uDatabase servers maintain some number of connections, so app servers can ask queries or perform modifications. uThe app server issues statements : queries and modifications, usually.

5. 5 Diagram to Remember Environment Connection Statement

6. 6 Real SQL Programming uEmbedded SQL (write SQL with “markup” and mixed with host language) uCall Level Interface (host level programming via SQL API) wJava, C++, Python, Ruby, etc uStored Procedures wUser defined procedures/function that become part of the schema (server level)

7. 7 Embedded SQL uKey idea: A preprocessor turns SQL statements into procedure calls that fit with the surrounding host-language code. uAll embedded SQL statements begin with EXEC SQL, so the preprocessor can find them easily.

8. 8 Shared Variables uTo connect SQL and the host-language program, the two parts must share some variables. uDeclarations of shared variables are bracketed by: EXEC SQL BEGIN DECLARE SECTION; EXEC SQL END DECLARE SECTION; Always needed

9. 9 Use of Shared Variables uIn SQL, the shared variables must be preceded by a colon. wThey may be used as constants provided by the host-language program. wThey may get values from SQL statements and pass those values to the host- language program. uIn the host language, shared variables behave like any other variable.

10. 10 Example: Looking Up Prices uWe’ll use C with embedded SQL to sketch the important parts of a function that obtains a beer and a bar, and looks up the price of that beer at that bar. uAssumes database has our usual Sells(bar, beer, price) relation.

11. 11 Example: C Plus SQL EXEC SQL BEGIN DECLARE SECTION; char theBar[21], theBeer[21]; float thePrice; EXEC SQL END DECLARE SECTION; /* obtain values for theBar and theBeer */ EXEC SQL SELECT price INTO :thePrice FROM Sells WHERE bar = :theBar AND beer = :theBeer; /* do something with thePrice */ Note 21-char arrays needed for 20 chars + endmarker SELECT-INTO as in PSM

12. 12 Embedded Queries uEmbedded SQL has the some limitations regarding queries: wSELECT-INTO for a query should produce a single tuple. wOtherwise, you have to use a cursor.

13. 13 Cursor Statements uDeclare a cursor c with: EXEC SQL DECLARE c CURSOR FOR ; uOpen and close cursor c with: EXEC SQL OPEN CURSOR c; EXEC SQL CLOSE CURSOR c; uFetch from c by: EXEC SQL FETCH c INTO ; wMacro NOT FOUND is true if and only if the FETCH fails to find a tuple.

14. 14 Example: Print Joe’s Menu uLet’s write C + SQL to print Joe’s menu – the list of beer-price pairs that we find in Sells(bar, beer, price) with bar = Joe’s Bar. uA cursor will visit each Sells tuple that has bar = Joe’s Bar.

15. 15 Example: Declarations EXEC SQL BEGIN DECLARE SECTION; char theBeer[21]; float thePrice; EXEC SQL END DECLARE SECTION; EXEC SQL DECLARE c CURSOR FOR SELECT beer, price FROM Sells WHERE bar = ’Joe’’s Bar’; The cursor declaration goes outside the declare-section

16. 16 Example: Executable Part EXEC SQL OPEN CURSOR c; while(1) { EXEC SQL FETCH c INTO :theBeer, :thePrice; if (NOT FOUND) break; /* format and print theBeer and thePrice */ } EXEC SQL CLOSE CURSOR c; The C style of breaking loops

17. 17 Need for Dynamic SQL uMost applications use specific queries and modification statements to interact with the database. wThe DBMS compiles EXEC SQL … statements into specific procedure calls and produces an ordinary host-language program that uses a library. uWhat about dynamic sql queries

18. 18 Dynamic SQL uPreparing a query: EXEC SQL PREPARE FROM ; uExecuting a query: EXEC SQL EXECUTE ; u“Prepare” = optimize query. uPrepare once, execute many times.

19. 19 Example: A Generic Interface EXEC SQL BEGIN DECLARE SECTION; char query[MAX_LENGTH]; EXEC SQL END DECLARE SECTION; while(1) { /* issue SQL> prompt */ /* read user’s query into array query */ EXEC SQL PREPARE q FROM :query; EXEC SQL EXECUTE q; } q is an SQL variable representing the optimized form of whatever statement is typed into :query

20. 20 Execute-Immediate uIf we are only going to execute the query once, we can combine the PREPARE and EXECUTE steps into one. uUse: EXEC SQL EXECUTE IMMEDIATE ;

21. 21 Example: Generic Interface Again EXEC SQL BEGIN DECLARE SECTION; char query[MAX_LENGTH]; EXEC SQL END DECLARE SECTION; while(1) { /* issue SQL> prompt */ /* read user’s query into array query */ EXEC SQL EXECUTE IMMEDIATE :query; }

22. 22 Host/SQL Interfaces Via Libraries uAnother approach to connecting databases to conventional languages is to use library calls. 1.C + CLI 2.Java + JDBC 3.PHP + PEAR/DB 4.Python+PyGreSQL

23. 23 JDBC uJava Database Connectivity (JDBC) is a library with Java as the host language.

24. 24 Making a Connection import java.sql.*; Class.forName(org.postgresql.Driver); Connection myCon = DriverManager.getConnection(…); The JDBC classes The driver for Postgres; others exist URL of the database your name, and password go here. Loaded by forName

25. 25 Statements uJDBC provides two classes: 1.Statement = an object that can accept a string that is a SQL statement and can execute such a string. 2.PreparedStatement = an object that has an associated SQL statement ready to execute.

26. 26 Creating Statements uThe Connection class has methods to create Statements and PreparedStatements. Statement stat1 = myCon.createStatement(); PreparedStatement stat2 = myCon.createStatement( ”SELECT beer, price FROM Sells ” + ”WHERE bar = ’Joe’ ’s Bar’ ” ); createStatement with no argument returns a Statement; with one argument it returns a PreparedStatement.

27. 27 Executing SQL Statements uJDBC distinguishes queries from modifications, which it calls “updates.” uStatement and PreparedStatement each have methods executeQuery and executeUpdate. wFor Statements: one argument: the query or modification to be executed. wFor PreparedStatements: no argument.

28. 28 Example: Update ustat1 is a Statement. uWe can use it to insert a tuple as: stat1.executeUpdate( ”INSERT INTO Sells ” + ”VALUES(’Brass Rail’,’Bud’,3.00)” );

29. 29 Example: Query ustat2 is a PreparedStatement holding the query ”SELECT beer, price FROM Sells WHERE bar = ’Joe’’s Bar’ ”. uexecuteQuery returns an object of class ResultSet – we’ll examine it later. uThe query: ResultSet menu = stat2.executeQuery();

30. 30 Accessing the ResultSet uAn object of type ResultSet is something like a cursor. uMethod next() advances the “cursor” to the next tuple. wThe first time next() is applied, it gets the first tuple. wIf there are no more tuples, next() returns the value false.

31. 31 Accessing Components of Tuples uWhen a ResultSet is referring to a tuple, we can get the components of that tuple by applying certain methods to the ResultSet. uMethod getX (i ), where X is some type, and i is the component number, returns the value of that component. wThe value must have type X.

32. 32 Example: Accessing Components uMenu = ResultSet for query “SELECT beer, price FROM Sells WHERE bar = ’Joe’ ’s Bar’ ”. uAccess beer and price from each tuple by: while ( menu.next() ) { theBeer = Menu.getString(1); thePrice = Menu.getFloat(2); /*something with theBeer and thePrice*/ }

33. 33 Stored Procedures uPSM, or “persistent stored modules,” allows us to store procedures as database schema elements. uPSM = a mixture of conventional statements (if, while, etc.) and SQL. uLets us do things we cannot do in SQL alone.

34. 34 Basic PSM Form: PL/PgSQL CREATE FUNCTION ( ) RETURNS AS $$ [ DECLARE declarations ] BEGIN statements END; $$ language plpgsql;

35. 35 Example: Stored Procedure uLet’s write a procedure that takes two arguments b and p, and adds a tuple to Sells(bar, beer, price) that has bar = ’Joe’’s Bar’, beer = b, and price = p. wUsed by Joe to add to his menu more easily.

36. 36 Kinds of PSM statements – (1) uRETURN sets the return value of a function. wUnlike C, etc., RETURN does not terminate function execution. uDECLARE used to declare local variables. uBEGIN... END for groups of statements. wSeparate statements by semicolons.

37. 37 Kinds of PSM Statements – (2) uAssignment statements: := ;  Example: b := ’Bud’; uStatement labels: give a statement a label by prefixing a name and a colon.

38. 38 Example: IF uLet’s rate bars by how many customers they have, based on Frequents(drinker,bar). w<100 customers: ‘unpopular’. w100-199 customers: ‘average’. w>= 200 customers: ‘popular’. uFunction Rate(b) rates bar b.

39. 39 Example: IF (continued) CREATE FUNCTION Rate (b CHAR(20) ) RETURNS CHAR(10) AS $$ DECLARE cust INTEGER; BEGIN SELECT COUNT(*) INTO cust FROM Frequents WHERE bar = b); IF cust < 100 THEN RETURN ’unpopular’ ELSEIF cust < 200 THEN RETURN ’average’ ELSE RETURN ’popular’ END IF; END; $$ Number of customers of bar b Return occurs here, not at one of the RETURN statements Nested IF statement

40. 40 Loops uBasic form: : LOOP END LOOP; uExit from a loop by: LEAVE

41. 41 Example: Exiting a Loop loop1: LOOP... LEAVE loop1;... END LOOP; If this statement is executed... Control winds up here

42. 42 Breaking Cursor Loops – (4) uThe structure of a cursor loop is thus: cursorLoop: LOOP … FETCH c INTO … ; IF NotFound THEN LEAVE cursorLoop; END IF; … END LOOP;

43. 43 Example: Cursor uLet’s write a function that examines Sells(bar, beer, price), and raises by $1 the price of all beers at Joe’s Bar that are under $3. Returns a count of number of changes made.

44. 44 The Needed Declarations CREATE FUNCTION JoeGouge( ) RETURNS int AS $$ DECLARE theBeer CHAR(20); DECLARE thePrice REAL; DECLARE cnt INT; DECLARE c CURSOR FOR (SELECT beer, price FROM Sells WHERE bar = ’Joe’’s Bar’); Used to hold beer-price pairs when fetching through cursor c Returns Joe’s menu

45. 45 The Function Body BEGIN OPEN c; cnt := 0; > LOOP FETCH c INTO theBeer, thePrice; IF NOT FOUND THEN LEAVE menuLoop END IF; IF thePrice < 3.00 THEN UPDATE Sells SET price = thePrice + 1.00 WHERE bar = ’Joe’’s Bar’ AND beer = theBeer; cnt := cnt+1; END IF; END LOOP; CLOSE c; RETURN cnt; END; Check if the recent FETCH failed to get a tuple If Joe charges less than $3 for the beer, raise its price at Joe’s Bar by $1.

46. Disks and Files Record Formats, Files & Indexing

47. 47 Disks, Memory, and Files Query Optimization and Execution Relational Operators Files and Access Methods Buffer Management Disk Space Management DB The BIG picture…

48. 48 Hierarchy of Storage uCache is faster and more expensive than... uRAM is faster and more expensive than... uDISK is faster and more expensive than... uDVD is faster and more expensive than... uTape is faster and more expensive than... uIf more information than will fit at one level, have to store some of it at the next level

49. Why Not Store Everything in Main Memory? uCosts too much. $100 will buy you either 4GB of RAM or 1TB of disk today. uMain memory is volatile. We want data to be saved between runs. (Obviously!) uTypical storage hierarchy: w Main memory (RAM) for currently used data. w Disk for the main database (secondary storage). w Tapes for archiving older versions of the data (tertiary storage).

50. Disks and Files uDBMS stores information on (“hard”) disks. uThis has major implications for DBMS design! w READ: transfer data from disk to main memory (RAM). w WRITE: transfer data from RAM to disk. w Both are high-cost operations, relative to in- memory operations, so must be planned carefully!

51. Disks uSecondary storage device of choice. uMain advantage over tapes: random access vs. sequential. uData is stored and retrieved in units called disk blocks or pages. uUnlike RAM, time to retrieve a disk page varies depending upon location on disk. w Therefore, relative placement of pages on disk has major impact on DBMS performance!

52. 52 Pages and Blocks uData files decomposed into pages wFixed size piece of contiguous information in the file wUnit of exchange between disk and main memory uDisk divided into page size blocks of storage wPage can be stored in any block uApplication’s request for data satisfied by wRead data page to DBMS buffer in main memory wTransfer requested data from buffer to application uApplication’s request to change data satisfied by wUpdate DBMS buffer w(Eventually) copy buffer page to page on disk

53. Components of a Disk Platters  The platters spin (7200rpm). Spindle  The arm assembly is moved in or out to position a head on a desired track. Tracks under heads make a cylinder (imaginary!). Disk head Arm movement Arm assembly  Only one head reads/writes at any one time. Tracks Sector  Block size is a multiple of sector size (which is fixed).

54. 54 I/O Time to Access a Page uSeek latency- time to position heads over cylinder containing page (~ 10 - 20 ms) uRotational latency - additional time for platters to rotate so that start of page is under head (~ 5 - 10 ms) uTransfer time - time for platter to rotate over page (depends on size of page) uLatency = seek latency + rotational latency uGoal - minimize average latency, reduce number of page transfers

55. Accessing a Disk Page uTime to access (read/write) a disk block: w seek time ( moving arms to position disk head on track ) w rotational delay ( waiting for block to rotate under head ) w transfer time ( actually moving data to/from disk surface ) uSeek time and rotational delay dominate. w Seek time varies from about 1 to 20msec w Rotational delay varies from 0 to 10msec w Transfer rate is about 1msec per 4KB page uLatency = seek latency + rotational latency uGoal - minimize average latency, reduce number of page transfers

56. 56 Reducing Latency uStore pages containing related information close together on disk wJustification: If application accesses x, it will next access data related to x with high probability uPage size tradeoff: wLarge page size - data related to x stored in same page; hence additional page transfer can be avoided wSmall page size - reduce transfer time, reduce buffer size in main memory wTypical page size - 4096 bytes

57. Arranging Pages on Disk u`Next’ block concept: w blocks on same track, followed by w blocks on same cylinder, followed by w blocks on adjacent cylinder uBlocks in a file should be arranged sequentially on disk (by `next’), to minimize seek and rotational delay. uFor a sequential scan, pre-fetching several pages at a time is a big win!

58. RAID uDisk Array: Arrangement of several disks that gives abstraction of a single, large disk. uGoals: Increase performance and reliability. uTwo main techniques: w Data striping: Data is partitioned; size of a partition is called the striping unit. Partitions are distributed over several disks. w Redundancy: More disks => more failures. Redundant information allows reconstruction of data if a disk fails.