Temporal Databases Microsoft SQL Server 2016 Amit Arora| Microsoft SQL Server Consultant

http://difinity.co.nz/

Don’t Forget Silence your cell phones Online Evaluations www.sqlsaturday.com/571/sessions/sessionevaluation.aspx www.sqlsaturday.com/571/eventeval.aspx Submit for raffles by 3:30PM

About Me Presently working with WARDY IT Solutions as a Microsoft SQL Server consultant. Worked as a Technical Advisor for 3 years in Microsoft India GTSC. Have a total of 11 years of industry experience primarily in Database & Infrastructure Solutions. LinkedIn (https://in.linkedin.com/in/amit-arora-21633699) E mail: amit.arora@wardyit.com

Agenda What is a Temporal Table? Why Temporal? And how it adds value to business? How Temporal works? How to query Temporal Tables? Temporal Table Partitioning Considerations and Limitations of using Temporal In-Memory Temporal Tables and its performance considerations

What is a Temporal Table? A system-versioned table designed to keep history of data changes. Two explicitly defined columns of DateTime2 data type. Reference to a history table with mirrored schema. History table gets updated on DML operations (Updates, Deletes). Works on row-versioning concept. New DateTime2 columns can be HIDDEN for select * from <tablename> and Insert scenarios. Can turn system versioning ON for an existing table.

Why Temporal (Use-Cases)? Auditing all data changes and performing data forensics when necessary Reconstructing state of the data as of any time in the past Calculating trends over time Maintaining a slowly changing dimension for decision support applications Recovering from accidental data changes and application errors

Demo Create temporal table

How Temporal Works? Inserts SysStartTime column value is set to Transaction Begin time in UTC time based on the system clock. SysEndTime column is populated with 9999-12-31. Marks the record as OPEN. Only get populated in the parent table. Updates Previous value of the record is stored in History Table. SysEndTime column value is set to Transaction Begin time in UTC time. Marks the record as closed SysStartTime and SysEndTime in history table signify the period for which the record was active Record is updated with the new value in current table.

How Temporal Works? (Contd.) Deletes Previous value of the record is stored in History Table. SysEndTime column value is set to Transaction Begin time in UTC time. Marks record as closed In the current table, record is removed. Queries for current table will NOT return this record. Merge Merge operation will behave as Insert, Update, or delete depending on the order of operation and condition of each record.

How Temporal Works (Contd.)

Demo Insert Data Update Delete Merge

How do I query Temporal Tables? New clause FOR SYSTEM_TIME in Select statements. AS OF: This sub-clause returns a record for each row that was valid at the specified point-in-time. SysStartTime <= A POINT IN TIME and SysEndTime > A POINT IN TIME. FROM .. TO: Returns all the row versions where StartTime is before the input <EndDateTime> and EndTime after the input parameter <StartDateTime>). SysStartTime < EndDateTime and SysEndTime > <StartDateTime> BETWEEN…AND: This clause is same as the previous FROM <StartDateTime> TO <EndDateTime> clause except that it also includes the records which became active on <EndDateTime> (i.e. StartTime = <EndDateTime>). CONTAINED IN: This clause returns only the row versions which became active and ended in the specified date range.  ALL: This clause returns all the row versions from both the Temporal and History table

Partitioning with Temporal Tables Enterprise edition feature Current table SWITCH IN and OUT NOT allowed when SYSTEM_VERSIONING=ON History Table SWITCH IN NOT allowed while SYSTEM_VERSIONING=ON SWITCH OUT allowed while SYSTEM_VERSIONING=ON to purge portions of historical data.

Considerations and Limitations History table is PAGE compressed by default. Partitioning configuration is not replicated by default between current and history tables. Current and history tables CANNOT be FILETABLES. (n)varchar(max), varbinary(max), (n)text, and image will have storage and performance costs involved. History table must be in same database and Temporal querying over linked server is not supported. Direct modification of history table data is NOT supported. InsteadOF triggers not supported on both tables. AFTER triggers supported only on current table

Considerations and Limitations (Contd.) AlwaysON: Fully supported. CDC and CT: Supported only on current table. Snapshot and Transactional Replication: One Publisher/subscriber pair supported. Publisher CANNOT support temporal, whereas ONE subscriber CAN. Pub for OLTP and Sub for reporting. Multiple subscribers NOT supported. Possible, however, may not be supported by Microsoft. As both subscribers will depend on their local clock and may cause inconsistencies. Merge Replication: Not supported. Design Indexing strategies carefully. A history table cannot be configured as a parent history table in a chain of history tables. Following are not replicated between parent and history table: Period definition Identity definition Indexes Statistics Check constraints Triggers Partitioning configuration Permissions

In-Memory Temporal Tables Supported only for DURABILITY = SCHEMA_AND_DATA History tables must be disk based. A staging memory optimized history table created automatically. Synchronous data transfer to In-Memory staging history table. Asynchronous flush to disk based history tables. SNAPSHOT ISOLATION level used when querying historical data. Returns UNION of In-Memory temporal and disk based history table. sys.dm_db_xtp_memory_consumers  (FOR MEMORY CONSUMPTION OF IN MEMORY STAGING TABLE) AND sys.sp_xtp_flush_temporal_history (FOR FLUSHING DATA MANUALLY)

Architecture of In-Memory Temporal operation

Performance considerations for In-Memory Temporal Load Overhead for Update and Delete operations due to synchronous flush to In-Memory staging history table. Perform Update and Delete operations in Batches. Data Flush is not configurable. Trigger it manually in case of load. Consider clustered column store indexes on disk based history table for analytics.

QUESTIONS

Thank You This FREE SQL Saturday is brought to you courtesy of these sponsors, speakers and volunteers who staff this event