1. Jerry Post Copyright © 2013 DATABASE Database Management Systems Chapter 7 Database Integrity and Transactions 1

2. Objectives  Why would you need to use procedural code when SQL is so powerful?  How are SQL commands integrated into more traditional programming structures?  What capabilities exist in procedural code?  How are business rules added to the database?  How does a DBMS handle multiple transaction events?  How do you prevent problems arising when two processes change the same data?  What are the primary rules to ensure integrity of transactions?  How are key values generated?  How can procedural code track row-by-row through a query?  What issues arise when maintaining totals in the database? 2

3. Programming Environment  Create code (1) Within the query system (2) In forms and reports (3) Hosted in external programs 3 DBMS Tables Forms & Reports Queries If (Click) Then MsgBox... End If If (.. ) Then SELECT... Else... UPDATE... End If C++ External Program if (...) { // embed SQL SELECT … } (2) (1) (3)

4. User-Defined Function 4 CREATE FUNCTION EstimateCosts (ListPrice Currency, ItemCategory VarChar) RETURNS Currency BEGIN IF (ItemCategory = ‘Clothing’) THEN RETURN ListPrice * 0.5 ELSE RETURN ListPrice * 0.75 END IF END

5. Function to Perform Conditional Update 5 CREATE FUNCTION IncreaseSalary (EmpID INTEGER, Amt CURRENCY) RETURNS CURRENCY BEGIN IF (Amt > 50000) THEN RETURN -1-- error flag END UPDATE Employee SET Salary = Salary + Amt WHERE EmployeeID = EmpID; RETURN Amt END

6. Looking Up Data 6 CREATE FUNCTION IncreaseSalary (EmpID INTEGER, Amt CURRENCY) RETURNS CURRENCY DECLARE CURRENCY MaxAmount; BEGIN SELECT MaxRaise INTO MaxAmount FROM CompanyLimits WHERE LimitName = ‘Raise’; IF (Amt > MaxAmount) THEN RETURN -1-- error flag END UPDATE Employee SET Salary = Salary + Amt WHERE EmployeeID = EmpID; RETURN Amt; END

7. Programming Tools 7 (1)Sequence (2)Variables (3)Conditions (4)Loops (5)Input and Output (6)Procedures and functions (subroutines)

8. (1) Sequence 8 INSERT INTO Customer (CustomerID, LastName, …) VALUES (1551, ‘Jones’, …); INSERT INTO Sales(SaleDate, CustomerID) VALUES (Now(), 1551); Control the order of execution for multiple tasks.

9. (2) Variables 9 Temporary locations in memory to hold data. FUNCTION ComputeTax(SalesTotal)… DECLARE Tax as currency; TaxRate as float; BEGIN SELECT TaxRate = SalesTax FROM TaxTable WHERE … Tax = SalesTotal*TaxRate; RETURN Tax; END; Scope: Within this function only.

10. (3) Conditions 10 Compare values or logical tests. FUNCTION sample(…) BEGIN IF (amount < 0) THEN action 1 ELSE action 2 END IF END;

11. (4) Loops 11 Repeat a set of operations. i=0; DO action 1 i = i + 1; WHILE (i < 10); C IS CURSOR FOR SELECT … DO UNTIL c.EOF examine one row of data FETCH(c) LOOP Useful for examining one row of data at a time from a SELECT statement.

12. (5) Input and Output 12 In a DBMS, interact with a form or a data table. DBMS, use SQL SELECT variables … INSERT INTO … variables FORM or other devices, depends on DBMS and location. Form code can access form fields, Trigger code cannot access forms.

13. (6) Procedures and Functions 13 Split code into meaningful pieces that are easier to debug. CREATE PROCEDURE AS … CREATE FUNCTION AS … To call a procedure or function: EXEC procedure … variable = function(…)

14. Data Trigger Events  Oracle additions:  TablesALTER, CREATE, DROP  UserLOGOFF, LOGON  DatabaseSERVERERROR, SHUTDOWN, STARTUP 14 INSERT DELETE UPDATE BEFOREAFTER

15. Example Trigger: Employee Salary  Goal: Keep a separate record of changes to salaries  Create a new audit-log table with columns for time, old salary, new salary, employee changed, and person making the change  Trigger: AFTER UPDATE of SALARY on EMPLOYEE  Basic code:  INSERT a row into the audit-log table with the tracking data 15

16. Statement v. Row Triggers 16 UPDATE Employee SET Salary = Salary + 10000 WHERE EmployeeID=442 OR EmployeeID=558 SQL time Before Update On table After Update On table Before Update Row 442 After Update Row 442 Update Row 442 … other rows Triggers for overall table Triggers for each row

17. Data Trigger Example 17 CREATE TRIGGER LogSalaryChanges AFTER UPDATE OF Salary ON Employee REFERENCINGOLD ROW as oldrow NEW ROW AS newrow FOR EACH ROW INSERT INTO SalaryChanges (EmpID, ChangeDate, User, OldValue, NewValue) VALUES (newrow.EmployeeID, CURRENT_TIMESTAMP, CURRENT_USER, oldrow.Salary, newrow.Salary);

18. Canceling Data Changes in Triggers 18 CREATE TRIGGER TestDeletePresident BEFORE DELETE ON Employee REFERENCING OLD ROW AS oldrow FOR EACH ROW WHEN (oldrow.Title = ‘President’) SIGNAL _CANNOT_DELETE_PRES;

19. Cascading Triggers 19 Sale(SaleID, SaleDate, …) OrderItem(OrderID, ItemID, Quantity, …) Order(OrderID, OrderDate, …) Inventory(ItemID, QOH, …) SaleItem(SaleID, ItemID, Quantity, …) AFTER INSERT UPDATE Inventory SET QOH = QOH – newrow.Quantity AFTER UPDATE WHEN newrow.QOH < newrow.Reorder INSERT {new order} INSERT {new OrderItem}

20. Trigger Loop 20 Employee(EID, Salary) BonusPaid(EID, BonusDate, Amount) StockOptions(EID, OptionDate, Amount, SalaryAdj) AFTER UPDATE IF newrow.Salary > 100000 THEN Add Bonus END AFTER UPDATE Or INSERT IF newrow.Bonus > 50000 THEN Reduce Bonus Add Options END AFTER UPDATE Or INSERT IF newrow.Amount > 100000 THEN Reduce Salary END

21. Transactions 21  Some transactions result in multiple changes.  These changes must all be completed successfully, or the group must fail.  Protection for hardware and communication failures.  example: bank customer transfers money from savings account to checking account. Decrease savings balance Increase checking balance Problem if one transaction and machine crashes.  Possibly: give users a chance to reverse/undo a transaction.  Performance gain by executing transactions as a block. Savings Accounts Inez:5340.92 4340.92 Checking Accounts Inez:1424.27 Transaction 1. Subtract $1000 from savings. (machine crashes) 2. Add $1000 to Checking. (money disappears) $1000

22. Transaction Steps StepsSavings BalanceChecking Balance 0. Start5,340.921,424.27 1. Subtract 1,0004,340.921,424.27 2. Add 1,0004,340.922,424.27 Problem arises if transaction is not completed 1. Subtract 1,0004,340.921,424.27 2. Machine crashes1,000 is gone 22

23. Transaction Logs 23 Update Table 1 … Update Table 2 … Table 1Table 2 Log File Transaction changes are written to the log file so they can be restarted if the computer crashes during a transaction. Low-end DBMSs (e.g., Access) do not have log files.

24. Defining Transactions 24  The computer needs to be told which changes must be grouped into a transaction.  Turn on transaction processing.  Signify a transaction start.  Signify the end. Success: save all changes Failure: cancel all changes  Must be set in module code  Commit  Rollback

25. SQL Transaction Code 25 CREATE FUNCTION TransferMoney(Amount Currency, AccountFrom Number, AccountTo Number) RETURNS NUMBER curBalance Currency; BEGIN DECLARE HANDLER FOR SQLEXCEPTION BEGIN ROLLBACK; Return -2;-- flag for completion error END; START TRANSACTION;-- optional SELECT CurrentBalance INTO curBalance FROM Accounts WHERE (AccountID = AccountFrom); IF (curBalance < Amount) THEN RETURN -1;-- flag for insufficient funds END IF UPDATE Accounts SET CurrentBalance = CurrentBalance – Amount WHERE AccountID = AccountFrom; UPDATE Accounts SET CurrentBalance = CurrentBalance + Amount WHERE AccountID = AccountTo; COMMIT; RETURN 0;-- flag for success END;

26. SAVEPOINT 26 START TRANSACTION; SELECT … UPDATE … SAVEPOINT StartOptional; UPDATE … If error THEN ROLLBACK TO SAVEPOINT StartOptional; END IF COMMIT; time start Required elements SAVEPOINT StartOptional Risky steps commit Partial rollback

27. Concurrent Access  Concurrent Access  Multiple users or processes changing the same data at the same time.  Final data will be wrong!  Force sequential  Locking  Delayed, batch updates  Two processes  Receive payment ($200)  Place new order ($150)  Initial balance $800  Result should be $800 -200 + 150 = $750  Interference result is either $600 or $950 27 IDBalance Jones$800 $600 $950 Customers 1) Read balance800 2) Subtract pmt-200 4) Save new bal.600 3) Read balance800 5) Add order150 6) Write balance950 Receive PaymentPlace New Order

28. Concurrent Access Steps Receive PaymentBalancePlace New Order 1. Read balance800 2. Subtract Pmt.-200 4. Save balance600 800 600 950 3. Read balance800 5. Add order150 6. Write balance950 28

29. Optimistic Locks  Assume that collisions are rare  Improved performance, fewer resources  Allow all code to read any data (no locks)  When code tries to write a new value  Check to see if the existing value is different from the one you were given earlier  If it is different, someone changed the database before you finished, so it is a collision--raise an error  Reread the value and try again 29

30. Optimistic Locks for Simple Update 30 (1) Read the balance (2) Add the new order value (3) Write the new balance (4) Check for errors (5) If there are errors, go back to step (1).

31. Optimistic Locks with SQL 31 CREATE FUNCTION ReceivePayment ( AccountID NUMBER, Amount Currency) RETURNS NUMBER oldAmount Currency; testEnd Boolean = FALSE; BEGIN DO UNTIL testEnd = TRUE BEGIN SELECT Amount INTO oldAmount WHERE AccountNumber = AccountID; … (slow code) UPDATE Accounts SET AccountBalance = AccountBalance - Amount WHERE AccountNumber = AccountID AND Amount = oldAmount; COMMIT; IF SQLCODE = 0 and nrows > 0 THEN testEnd = TRUE; RETURN 0; END IF -- keep a counter to avoid infinite loops END

32. Optimistic UPDATE: WHERE 32 var: oldAmount = SELECT … … UPDATE … WHERE ID=AccountID AND (Amount = oldAmount) To handle missing values, better to write: ((Amount = oldAmount) OR (Amount Is Null AND oldAmount Is Null) )

33. Minimize Concurrency Issues 33 Avoid concurrency issues by keeping code fast and avoiding delays. Bad: Read BalanceSELECT … Do slow stuff Add newBal + Bal + New Write new totalUPDATE … Better: UPDATE table SET Balance = Balance + NewValue WHERE … Delays can arise when you present screens of data to users. Particularly on Web forms. But, when possible, perform the computations within the DBMS not on the Web server forms. That means the data displayed to the user sometimes might be out of date, but concurrency within the DBMS is rare. Also: Avoid storing totals. Just write time-stamped change logs and compute the totals when they are needed.

34. Pessimistic Locks: Serialization  One answer to concurrent access is to prevent it.  When a transaction needs to alter data, it places a SERIALIZABLE lock on the data used, so no other transactions can even read the data until the first transaction is completed. 34 IDBalance Jones$800 $600 Customers 1) Read balance800 2) Subtract pmt-200 4) Save new bal.600 3) Read balance Receive error message that it is locked. Receive PaymentPlace New Order SET TRANSACTION SERIALIZABLE, READ WRITE

35. SQL Pessimistic Lock 35 CREATE FUNCTION ReceivePayment ( AccountID NUMBER, Amount Currency) RETURNS NUMBER BEGIN DECLARE HANDLER FOR SQLEXCEPTION BEGIN ROLLBACK; RETURN -2; END SET TRANSACTION SERIALIZABLE, READ WRITE; UPDATE Accounts SET AccountBalance = AccountBalance - Amount WHERE AccountNumber = AccountID; COMMIT; RETURN 0; END

36. Serialization Effects Receive PaymentBalancePlace New Order 1. Read balance800 2. Subtract Pmt.-200 4. Save balance600 800 600 750 3. Read balance Error: Blocked 3. Read balance600 4. Add order150 5. Write balance750 36

37. Transaction to Transfer Money 37 CREATE FUNCTION ReceivePayment ( AccountID NUMBER, Amount Currency) RETURNS NUMBER BEGIN DECLARE HANDLER FOR SQLEXCEPTION BEGIN ROLLBACK; RETURN -2; END SET TRANSACTION SERIALIZABLE, READ WRITE; UPDATE Accounts SET AccountBalance = AccountBalance - Amount WHERE AccountNumber = AccountID; COMMIT; RETURN 0; END

38. Deadlock  Deadlock  Two (or more) processes have placed locks on data and are waiting for the other’s data.  Many solutions  Random wait time  Global lock manager  Two-phase commit - messages 38 Data AData B 1) Lock Data A 3) Wait for Data B 2) Lock Data B 4) Wait for Data A

39. Deadlock Sequence Process 1Data AData BProcess2 1.Lock Data A 3. Wait for Data B Locked by 1 Locked by 2 2. Lock Data B 4. Wait for Data A 39

40. Lock Manager 40

41. ACID Transactions  Atomicity: all changes succeed or fail together.  Consistency: all data remain internally consistent (when committed) and can be validated by application checks.  Isolation: The system gives each transaction the perception that it is running in isolation. There are no concurrent access issues.  Durability: When a transaction is committed, all changes are permanently saved even if there is a hardware or system failure. 41

42. SQL 99/2003 Isolation Levels (Advanced)  READ UNCOMMITTED  Problem: might read dirty data that is rolled back  Restriction: not allowed to save any data  READ COMMITTED  Problem: Second transaction might change or delete data  Restriction: Need optimistic concurrency handling  REPEATABLE READ  Problem: Phantom rows  SERIALIZABLE  Provides same level of control as if all transactions were run sequentially.  But, still might encounter locks and deadlocks 42

43. Phantom Rows (Advanced) ItemIDQOHPrice 111515 11367 1171230 118412 119722 120817 121716 122314 43 SELECT SUM(QOH) FROM Inventory WHERE Price BETWEEN 10 and 20 Result: 5 + 4 + 8 = 17 INSERT INTO Inventory VALUES (121, 7, 16) INSERT INTO Inventory VALUES (122, 3, 14) Included in first query SELECT SUM(QOH) FROM Inventory WHERE Price BETWEEN 10 and 20 Result: 5 + 4 + 8 + 7 + 3 = 27 Additional rows will be included in the second query

44. Generated Keys 44 Create an order for a new customer: (1) Create new key for CustomerID (2) INSERT row into Customer (3) Create key for new OrderID (4) INSERT row into Order Customer Table CustomerID, Name, … Order Table OrderID, CustomerID, …

45. Methods to Generate Keys 45 1.The DBMS generates key values automatically whenever a row is inserted into a table. Drawback: it is tricky to get the generated value to use it in a second table. 2.A separate key generator is called by a programmer to create a new key for a specified table. Drawback: programmers have to write code to generate a key for every table and each row insertion. Overall drawbacks: neither method is likely to be transportable. If you change the DBMS, you will have to rewrite the procedures to generate keys.

46. Auto-Generated Keys 46 Create an order for a new customer: (1)INSERT row into Customer (2)Get the key value that was generated (3)Verify the key value is correct. How? (4)INSERT row into Order Major problem: Step 2 requires that the DBMS return the key value that was most recently generated. How do you know it is the right value? What happens if two transactions generate keys at almost the same time on the same table?

47. Key-Generation Routine 47 Create an order for a new customer: (1)Generate a key for CustomerID (2)INSERT row into Customer (3)Generate a key for OrderID (4)INSERT row into Order This method ensures that unique keys are generated, and that you can use the keys in multiple tables because you know the value. But, none of it is automatic. It always requires procedures and sometimes data triggers.

48. Database Cursors  Purpose  Track through table or query one row at a time.  Data cursor is a pointer to active row.  Why?  Performance.  SQL cannot do everything. Complex calculations. Compare multiple rows. 48 YearSales 1998104,321 1999145,998 2000276,004 2001362,736 1998104,321 1999145,998 2000276,004 2001362,736

49. Database Cursor Program Structure 49 DECLARE cursor1 CURSOR FOR SELECT AccountBalance FROM Customer; sumAccount, balance Currency; SQLSTATE Char(5); BEGIN sumAccount = 0; OPEN cursor1; WHILE (SQLSTATE = ‘00000’) BEGIN FETCH cursor1 INTO balance; IF (SQLSTATE = ‘00000’) THEN sumAccount = sumAccount + balance; END IF END CLOSE cursor1; -- display the sumAccount or do a calculation END

50. Cursor Positioning with FETCH 50 DECLARE cursor2 SCROLL CURSOR FOR SELECT … OPEN cursor2; FETCH LAST FROM cursor2 INTO … Loop… FETCH PRIOR FROM cursor2 INTO … End loop CLOSE cursor2; FETCH positioning options: FETCH NEXTnext row FETCH PRIORprior row FETCH FIRSTfirst row FETCH LASTlast row FETCH ABSOLUTE 5fifth row FETCH RELATIVE -3back 3 rows

51. Problems with Multiple Users 51 NameSales Alice444,321 Carl254,998 Donna652,004 Ed411,736 Original Data NameSales Alice444,321 Bob333,229 Carl254,998 Donna652,004 Ed411,736 Modified Data New row is added--while code is running. The SQL standard can prevent this problem with the INSENSITIVE option: DECLARE cursor3 INSENSITIVE CURSOR FOR … But, this is an expensive approach, because the DBMS usually makes a copy of the data. Instead, avoid moving backwards.

52. Changing Data with Cursors YearSalesGain 2000151,039 2001179,332 2002195,453 2003221,883 2004223,748 52 DECLARE cursor1 CURSOR FOR SELECT Year, Sales, Gain FROM SalesTotal ORDER BY Year FOR UPDATE OF Gain; priorSales, curYear, curSales, curGain BEGIN priorSales = 0; OPEN cursor1; Loop: FETCH cursor1 INTO curYear, curSales, curGain UPDATE SalesTotal SET Gain = Sales – priorSales WHERE CURRENT OF cursor1; priorSales = curSales; Until end of rows CLOSE cursor1; COMMIT; END

53. Dynamic Parameterized Cursor Queries 53 DECLARE cursor2 CURSOR FOR SELECT ItemID, Description, Price FROM Inventory WHERE Price < :maxPrice; maxPrice Currency; BEGIN maxPrice = …-- from user or other query OPEN cursor2;-- runs query with current value Loop: -- Do something with the rows retrieved Until end of rows CLOSE cursor2; END Parameters enable you to control the rows retrieved dynamically from within the procedure code. The value is applied when the cursor is opened.

54. Sally’s Pet Store Inventory  Inventory method 1: calculate the current quantity on hand by totaling all purchases and sales every time the total is needed.  Drawback: performance  Inventory method 2: keep a running balance in the inventory table and update it when an item is purchased or sold.  Drawback: tricky code  Also, you need an adjustment process for “inventory shrink” 54

55. Inventory QuantityOnHand 55 ItemID Description QuantityOnHand ListPrice Category Merchandise Add items purchased Subtract items sold Adjust for shrink SaleID ItemID Quantity SalePrice SaleItem

56. Inventory Events  For a new sale, a row is added to the SaleItem table.  A sale or an item could be removed because of a clerical error or the customer changes his or her mind. A SaleItem row will be deleted.  An item could be returned, or the quantity could be adjusted because of a counting error. The Quantity is updated in the SaleItem table.  An item is entered incorrectly. ItemID is updated in the SaleItem table. 56 SaleID ItemID Quantity SalePrice SaleItem (1)Add a row. (2)Delete a row. (3)Update Quantity. (4)Update ItemID.

57. New Sale: Insert SaleItem Row 57 CREATE TRIGGER NewSaleItem AFTER INSERT ON SaleItem REFERENCINGNEW ROW AS newrow FOR EACH ROW UPDATE Merchandise SET QuantityOnHand = QuantityOnHand – newrow.Quantity WHERE ItemID = newrow.ItemID;

58. Delete SaleItem Row 58 CREATE TRIGGER DeleteSaleItem AFTER DELETE ON SaleItem REFERENCINGOLD ROW AS oldrow FOR EACH ROW UPDATE Merchandise SET QuantityOnHand = QuantityOnHand + oldrow.Quantity WHERE ItemID = oldrow.ItemID;

59. Inventory Update Sequence SaleItemClerkEvent CodeMerchandise SaleID101 ItemID15 Quantity10 Quantity8 1.Enter new sale item, enter Quantity of 10. 3. Change Quantity to 8. 2. Subtract Quantity 10 from QOH. 4. Subtract Quantity 8 from QOH. ItemID15 QOH50 QOH40 QOH32 Solution that corrects for change SaleID101 ItemID15 Quantity10 Quantity8 1.Enter new sale item, enter Quantity of 10. 3. Change Quantity to 8. 2. Subtract Quantity 10 from QOH. 4. Add original Quantity 10 back and subtract Quantity 8 from QOH. ItemID15 QOH50 QOH40 QOH42 59

60. Quantity Changed Event 60 CREATE TRIGGER UpdateSaleItem AFTER UPDATE ON SaleItem REFERENCINGOLD ROW AS oldrow NEW ROW AS newrow FOR EACH ROW UPDATE Merchandise SET QuantityOnHand = QuantityOnHand + oldrow.Quantity – newrow.Quantity WHERE ItemID = oldrow.ItemID;

61. ItemID or Quantity Changed Event 61 CREATE TRIGGER UpdateSaleItem AFTER UPDATE ON SaleItem REFERENCINGOLD ROW AS oldrow NEW ROW AS newrow FOR EACH ROW BEGIN UPDATE Merchandise SET QuantityOnHand = QuantityOnHand + oldRow.Quantity WHERE ItemID = oldrow.ItemID; UPDATE Merchandise SET QuantityOnHand = QuantityOnHand – newRow.Quantity WHERE ItemID = newrow.ItemID; COMMIT; END