1. CS240A: Databases and Knowledge Bases Temporal Applications and SQL Carlo Zaniolo Department of Computer Science University of California, Los Angeles

2. Temporal Databases  The problem is harder than what you think  A time ontology—Temporal data type in SQL  Temporal Applications and SQL  Plenty of temporally-oriented DB applications  Not supported well by SQL  Many research approaches proposed to solve the problem  TSQL2  The physical level: efficient storage and indexing techniques.

3. Applications Abound: Examples  Academic: Transcripts record courses taken in previous and the current semester or term and grades for previous courses  Accounting: What bills were sent out and when, what payments were received and when?  Delinquent accounts, cash flow over time  Money­management software such as Quickencan show e.g., account balance over time.  Budgets: Previous and projected budgets, multi­ quarter or multi­year budgets

4. Temporal DB Applications (cont.)  Data Warehousing: Historical trend analysis for decision support  Financial: Stock market data  Audit: why were financial decisions made, and with what information available?  GIS: Geographic Information Systems ()  Land use over time: boundary of parcels changeover time, as parcels get partitioned and merged.  Title searches  Insurance: Which policy was in effect at each point in time, and what time periods did that policy cover?

5. Temporal DB Applications (cont.)  Medical records: Patient records, drug regimes, lab tests.Tracking course of disease  Payroll: Past employees, employee salary history, salaries for future months, records of withholdingrequested by employees  Capacity planning for roads and utilities. Configuring new routes, ensuring high utilization  Project scheduling: Milestones, task assignments  Reservation systems: airlines, hotels, trains.  Scientific: Timestamping satellite images. Dating archeological finds

6. Temporal DBs Applications: Conclusion  It is difficult to identify applications that do not involve the management of temporal data.  These applications would benefit from built­in temporal support in the DBMS. Main benefits:  More efficient application development  Potential increase in performance

7. A Case Study on using SQL for temporal Apps  University of Arizona's Office of Appointed Personnel has some information in a database. Employee(Name, Salary, Title)  The OAP wishes to add the date of birth Employee(Name, Salary, Title, DateofBirth DATE) SELECT Salary, DateofBirth FROM Employee WHERE Name = 'Bob‘  Finding an employee's DoB is as easy as finding his/her salary.  Managing Dates and instants in time are not a problem--- the hard problems come with periods (a.k.a. intervals)

8. Converting to a Temporal Database  Now the OAP wishes to computerize the employment history.  Adding validity periods to tuples: Employee (Name, Salary, Title, DateofBirth, Start DATE, Stop DATE)

9. A Temporal Database using Periods Employee (Name, Salary, Title, DateofBirth,Start DATE, Stop DATE) NameSalaryTitleDateofBirthStartStop Bob 60000 AssistantProvost 1945­04­19 1993­01­01 1993­06­01 Bob 70000 AssistantProvost 1945­04­19 1993­06­01 1993­10­01 Bob 70000 Provost 1945­04­19 1993­10­01 1994­02­01 Bob 70000 Professor 1945­04­19 1994­02­01 1995­01­01 Here we use closed intervals—intervals open to the right are also used often: if the new period begin at 1993-10-01 the old one must end at 1993-09-30.

10. Temporal Representations  Temporal Intervals (i.e., periods) are the most popular representations for time. For a short overview of the topic see: www.ise.bgu.ac.il/courses/trp/TR-in-CS-and-AI.revised.ppt www.ise.bgu.ac.il/courses/trp/TR-in-CS-and-AI.revised.ppt  Operators and predicates associated with periods include:  Overlap, Contains, Meets, Precedes, and Follows  Several other representations have been used including:  Sets of Periods,  Point-Based Representations  TSQL2: the model is set of periods where each period is a set of cronons--but there is no direct access to those: there manipulation follows implicitly from special constructs.

11. Allen’s 13 Temporal Predicates on Periods A B A B A B A B A B A B A B A FINISHES B B is FINISHED by A A is BEFORE B B is AFTER A A MEETS B B is MET by A A OVERLAPS B B is OVERLAPPED by A A STARTS B B is STARTED by A A is EQUAL to B B is EQUAL to A A DURING B B CONTAINS A

12. Period-based temporal representations are simple in SQL  But temporal queries are quite complex..  E.g., Find the employee's salary at a given time: e.g. the current one: SELECT Salary FROM Employee WHERE Name = 'Bob‘ AND Start <= CURRENT_TIMESTAMP AND CURRENT_TIMESTAMP <= Stop Instead of CURRENT_TIMESTAMP we could have given any timestamp or date

13. Distributing the Salary History  OAP wants to distribute to all employees their salary history. For Bob 3 consecutive periods at 70000.  In general: employee could have arbitrarily many title changes between salary changes  Canonical representation: maximal intervals at each salary Name Salary Start Stop Bob 60000 1993­01­01 1993­06­01 Bob 70000 1993­06­01 1995­01­01  A complex operation called coalescing is needed to compute the maximal intervals—this must be computed after each projection.

14. Coalescing Using Embedded SQL The coalescing operations that are needed after each projection operation are difficult to express in pure SQL.  Coalescing Using Embedded SQL: Use SQL only to open a cursor on the table and perform the actual coalescing in a programming language.  Coalescing Using a 4GL: solution given in the textbook (1995) is obsolete: use recursive queries in SQL1999/2003

15. Coaleshing using a 4GL CREATE TABLE Temp(Salary, Start, Stop) AS SELECT Salary, Start, Stop FROM Employee WHERE Name = 'Bob'; repeat UPDATE Temp AS T1 SET (T1.Stop) = (SELECT MAX(T2.Stop) FROM Temp AS T2 WHERE T1.Salary = T2.Salary AND T1.Start = T2.Start AND T1.Stop = T2.Start AND T1.Stop < T2.Stop) until no tuples updated;

16. Salary History (cont.)  Intervals that are not maximal must be deleted DELETE FROM Temp T1 WHERE EXISTS (SELECT * FROM Temp AS T2 WHERE T1.Salary = T2.Salary AND ( (T1.Start > T2.Start AND T1.Stop = T2.Start AND T1.Stop < T2.Stop) ) The loop is executed lgN times in the worst case, where N is the number of tuples in a chain of overlapping or adjacent, value­equivalent tuples. Then delete extraneous, non­ maximal intervals.

17. A Better Alternative Recursion  Use recursion.  It two periods overlap merge them into a new period. Overlap: Not(E1 =S2 and E2>=S1.

18. Surprise: Non-Recursive SQL CREATE TABLE Temp(Salary, Start, Stop) AS SELECT Salary, Start, Stop FROM Employee WHERE Name = 'Bob'; SELECT DISTINCT F.Salary, F.Start, L.Stop FROM Temp AS F, Temp AS L WHERE F.Start < L.Stop AND F.Salary = L.Salary AND NOT EXISTS (SELECT * FROM Temp AS M WHERE M.Salary = F.Salary AND F.Start < M.Start AND M.Start < L.Stop AND NOT EXISTS (SELECT * FROM Temp AS T1 WHERE T1.Salary = F.Salary AND T1.Start < M.Start AND M.Start <= T1.Stop)) AND NOT EXISTS (SELECT * FROM Temp AS T2 WHERE T2.Salary = F.Salary AND ( (T2.Start < F.Start AND F.Start <= T2.Stop) OR (T2.Start < L.Stop AND L.Stop < T2.Stop)))

19. The Curse of Coalescing  Maximal periods is the standard state-based representation  Projection is very common in queries and was no-op in SQL: but now requires coalescing  Expressing coalescing in SQL is possible but complex  A better solution could be to introduce a special operator—which is basically an aggregate, and aggregates in SQL require a particular structure.  Question: Can we find representations that eliminate or minimize the need for coalescing?

20. Minimize the need for coalescing by Reorganizing the schema  Separate Salary, Title, and DateofBirth information: Employee1 (Name, Salary, Start DATE, Stop DATE) Employee2 (Name, Title, Start DATE, S top DATE)  Getting the salary information is now easy: SELECT Salary, Start, Stop FROM Employee1 WHERE Name = 'Bob‘  But what if we want a table with both salary and title?

21. Temporal Joins NameSalaryStartStop Bob600001993­01­011993­06­01 Bob700001993­06­011995­01­01 NameTitleStartStop BobAssistantProvost1993­01­011993­10­01 BobProvost1993­10­011994­02­01 BobFullProfessor1994­02­011995­01­01 NameSalaryTitleStartStop Bob60000AssistantProvost1993­01­011993­06­01 Bob70000AssistantProvost1993­06­011993­10­01 Bob70000Provost1993­10­011994­02­01 Bob70000FullProfessor1994­02­011995­01­01 Their Temporal Join: Employee 1 : Employee 2 :

22. Temporal Join in SQL SELECT E1.Name, Salary, Title, E1.Start, E1.Stop FROM Employee1 AS E1, Employee2 AS E2 WHERE E1.Name=E2.Name AND E2.Start <= E1.Start AND E1.Stop <= E2.Stop UNION ALL SELECT E1.Name, Salary, Title, E1.Start, E2.Stop FROM Employee1 AS E1, Employee2 AS E2 WHERE E1.Name = E2.Name AND E1.Start > E2.Start AND E2.Stop< E1.Stop AND E1.Start < E2.Stop UNION ALL SELECT E1.Name, Salary, Title E2.Start, E1.Stop FROM Employee1 AS E1, Employee2 AS E2 WHERE E1.Name = E2.Name AND E2.Start > E1.Start AND E1.Stop <= E2.Stop AND E2.Start < E1.Stop UNION ALL SELECT E1.Name, Salary, Title E2.Start, E2.Stop FROM Employee1 AS E1, Employee2 AS E2 WHERE E1.Name = E2 Name AND E2.Start => E1.Start AND E2.Stop <= E1.Stop AND NOT (E1.Start = E2.Start AND E1.Stop = E2.Stop)

23. Simpler Temporal Join in SQL Overlap: E1>=S2 and E2>=S1 What is the actual intersection? larger(X, Y, X) =Y. larger(X, Y, Y) <- X < Y. Symmetrically for smaller. You can also use if… then else in Deals, or CASE in SQL Now: intersection(B, E)<- period(S1, E1), period(S2, E2), larger(B1, B2, B), smaller(E1, E2, E), B<E.

24. Summary  Coalescing and temporal joins are very difficult to express in SQL.  Solutions proposed …  Time stamp attributes rather than tuples—but then many temporal joins must be used  Point-Based Representation  Others, including combinations of above (more than 40 counted that were using SQL)  We will discuss TSQL2 and XML later.

25. Reviewing the Situation  The importance of temporal applications has motivated much research on temporal DBs: but no satisfactory solution has been found yet:  SQL does not support temporal queries well  Temporal DBs remain an open research problem.  The problem is much more difficult than it appears at first: we have become so familiar with the time domain that we tend to overlook its intrinsic complexity.  Other issues that we have not discussed yet include:  Support for bitemporal models  temporal clustering, indexing and implementation-oriented issues

26. This was proposed by temporal DB people Point-Based Model Employee1 (Name, Sal, Day ) Bob 60000 1993­01­01 … Bob 60000 1993­05­31 Bob 70000 1993­05­31 … Bob 70000 1994­12­31 To project out Sal you only need to eliminate that column. Internally we need to use more concise representations—e.g., the period- based representations: NameSalaryStartStop Bob600001993­01­011993­06­01 Bob700001993­06­011995­01­01

27. Queries in Point-Based  No coalescing needed in the query: e.g., project out salary: SELECT E1.Name, E1.Day FROM Employee1 AS E1  Temporal Joins are simple: SELECT E1.Name, Sal, Title FROM Employee1 AS E1, Employee2 AS E2 WHERE E1.Name = E2.Name AND E1.Day=E2.Day The point –based representation can only be used as a logical view. Since a very different internal representation must be used, support for queries can be a challenge. No serious takers so far.