1. juliandyke.com 1 © 2013 - Julian Dyke Web Version Oracle 12c New Features

2. juliandyke.com 2 © 2013 - Julian Dyke Agenda Introduction Pluggable Database Partial Indexes Online Data File Move Online Partition Move Index Columns Invisible Columns Identity Clause Session Sequences Global Temporary Table Undo Temporal Validity Extended Columns Row Limiting Clause Histograms Application Continuity

3. juliandyke.com 3 © 2013 - Julian Dyke Introduction This presentation investigates a selection of Oracle 12c new features that I believe will be interesting to DBAs The presentation was originally delivered at the UKOUG Conference 2013 in Manchester, England I have added section headers containing comments and feedback from delegates

4. juliandyke.com 4 © 2013 - Julian Dyke What History Tells Us.... Oracle 9i, 10g and 11g R1 releases have been available for 18-24 months R2 releases have been available for several years R2 releases include terminal release Support has often been extended for terminal release CPU and PSU support is limited for R1 releases Longer and more comprehensive for R2 releases It is occasionally necessary to upgrade to a terminal release in order to migrate to new functionality In past releases there have been compatibility issues between new features Occasionally bugs.... Sometimes new features are documented but not released

5. juliandyke.com 5 © 2013 - Julian Dyke New Features Oracle Marketing concentrates on a limited subset of new features Particularly new licensing options Product Managers and Pre Sales are usually a better source of information New features are often overlooked by everyone: Particularly additional features in Standard/Enterprise Editions Too many in each release to investigate them all Documentation and support is often limited at initial release

6. juliandyke.com 6 © 2013 - Julian Dyke Pluggable Database Other presenters will have discussed pluggable databases in more detail The concept was announced in September 2012 and I now believe it is time to consider how and where it is appropriate to deploy pluggable databases My example of a possible deployment was a container database with a large number of pluggable databases replacing SYBASE. I know that SYBASE replacement has been a goal at a few of the larger banks for many years. Something I missed is that pluggable databases can be cloned allowing test databases to be created rapidly from production databases. I have not investigated this feature yet, so have limited my comments to technical questions I would still like to answer

7. juliandyke.com 7 © 2013 - Julian Dyke Pluggable Database Definitely the most attractive marketing feature Easy to explain Impresses managers and technical staff Obvious benefits May be more efficient than virtualization for large numbers of similar databases For example migrations from SYBASE Potential reduction in resource consumption including: CPU memory background processes management costs

8. juliandyke.com 8 © 2013 - Julian Dyke Pluggable Database Reduction in CPU might reduce processor license requirements Separately licensed as Oracle Multi Tenant option US list price is $17,500 per processor (EE is $47,500) All options will need to be licenced for all pluggable databases Possibly irrespective of usage e.g. Partitioning, Advanced Compression, Advanced Security

9. juliandyke.com 9 © 2013 - Julian Dyke Pluggable Database Only one redo thread per container instance Online redo logs may be a bottleneck Data Guard Single configuration for container database Pluggable databases share redo thread May become difficult to manage if standby databases need to be rebuilt Single large SGA may increase size of kernel page tables area for each process (foreground / background or both) Will offset some of the savings in background process memory New In-Memory database may have same problem Pluggable databases may contend for resources e.g. RAC background processes

10. juliandyke.com 10 © 2013 - Julian Dyke Partial Indexes I believe this is one of the best features in Oracle 12c for sites using the Partitioning Option Most sites partition their tables based on time e.g. year, month, week, day etc. Most activity centres around the latest (hot) partitions where indexes are often required to optimize access paths. However, the cost of creating an index for the entire table often prevents creation of appropriate indexes as, in current versions, the index needs to be created for all partitions, requiring additional storage and increasing backup and restore times. Partial indexes will not reduce redo generation, but could significantly reduce overall database sizes as they will often affect the largest tables I think the current implementation is limited, but I still think this is a great feature

11. juliandyke.com 11 © 2013 - Julian Dyke Partial Indexes One of the most important new features in Oracle 12.1 Allows additional indexes to be created for performance tuning Potentially reduces amount of storage required for indexes May reduce backup and restore times Will probably not reduce redo / archive generation Functionality is limited For a specific table, only one set of table partitions can be enabled for index partitions

12. juliandyke.com 12 © 2013 - Julian Dyke Partial Indexes Useful for range-based partitioned tables Create partial indexes on most recent (hot) partitions Alternatively create partial indexes on older (archived) partitions However cannot create partial indexes on both Partial indexing must specified on table partitions INDEXING ON – partial indexes enabled INDEXING OFF – partial indexes disabled If a table partition has INDEXING ON then all rows in that partition will be indexed in each partial index

13. juliandyke.com 13 © 2013 - Julian Dyke Partial Indexes Partial Local and Global Indexes CREATE TABLE pcar ( season_keyNUMBER, race_keyNUMBER, driver_keyVARCHAR2(4), team_keyVARCHAR2(3), positionNUMBER, laps_completedNUMBER, race_pointsNUMBER ) PARTITION BY RANGE (season_key) ( PARTITION p2008 VALUES LESS THAN (2009) INDEXING OFF, PARTITION p2009 VALUES LESS THAN (2010) INDEXING OFF, PARTITION p2010 VALUES LESS THAN (2011) INDEXING OFF, PARTITION p2011 VALUES LESS THAN (2012) INDEXING ON, PARTITION p2012 VALUES LESS THAN (2013) INDEXING ON );

14. juliandyke.com 14 © 2013 - Julian Dyke Partial Indexes Partial Local and Global Indexes SELECT season_key, COUNT(*) FROM pcar GROUP BY season_key ORDER BY season_key; SEASON_KEYCOUNT(*) 2008368 2009338 2010456 2011456 2012480

15. juliandyke.com 15 © 2013 - Julian Dyke Partial Indexes Example - Partial Local Index CREATE INDEX pcar1 ON pcar (driver_key) LOCAL INDEXING PARTIAL; dbms_stats.gather_index_stats ( ownname => 'GP', indname => 'PCAR1', estimate_percent => NULL ); SELECT partition_name,num_rows FROM dba_ind_partitions WHERE index_name = 'PCAR1'; PARTITION_NAMENUM_ROWS P20080 P20090 P20100 P2011456 P2012480

16. juliandyke.com 16 © 2013 - Julian Dyke Partial Indexes Example - Partial Global Index CREATE INDEX pcar2 ON pcar (team_key) GLOBAL INDEXING PARTIAL; dbms_stats.gather_index_stats ( ownname => 'GP', indname => 'PCAR2', estimate_percent => NULL ); SELECT num_rows FROM dba_indexes WHERE index_name = 'PCAR2'; NUM_ROWS 936

17. juliandyke.com 17 © 2013 - Julian Dyke Partial Indexes Execution Plans - Partial Local Index CREATE INDEX pcar3 ON pcar (season_key,race_key,position) LOCAL INDEXING PARTIAL; SELECT COUNT(*) FROM pcar WHERE season_key = '2010';-- Unindexed 0SELECT STATEMENT 1SORT AGGREGATE 2PARTITION RANGE SINGLE 3TABLE ACCESS FULL (PCAR) Predicate Information (identified by operation id): 3 - filter("SEASON_KEY"=2010) Cost = 14

18. juliandyke.com 18 © 2013 - Julian Dyke Partial Indexes Execution Plans - Partial Local Index CREATE INDEX pcar3 ON pcar (season_key,race_key,position) LOCAL INDEXING PARTIAL; SELECT COUNT(*) FROM pcar WHERE season_key = '2011';-- Indexed 0SELECT STATEMENT 1SORT AGGREGATE 2PARTITION RANGE SINGLE 3INDEX FAST FULL SCAN (PCAR3) Predicate Information (identified by operation id): 3 - filter("SEASON_KEY"=2011) Cost = 2

19. juliandyke.com 19 © 2013 - Julian Dyke Partial Indexes Execution Plans - Partial Local Index SELECT COUNT(*) FROM pcar WHERE season_key IN ('2010','2011');-- Combined 0SELECT STATEMENT 1SORT AGGREGATE 2PARTITION RANGE INLIST 3TABLE ACCESS FULL (PCAR) Predicate Information (identified by operation id): 3 - filter("SEASON_KEY"=2010 OR "SEASON_KEY"=2011) Cost = 27 CREATE INDEX pcar3 ON pcar (season_key,race_key,position) LOCAL INDEXING PARTIAL;

20. juliandyke.com 20 © 2013 - Julian Dyke Online Data File Move This is a great new feature which I have already been using to resolve space issues in my own virtual machines I have successfully used this to move the data file containing the SYSAUX tablespace – not sure I would want to risk it with the SYS tablespace

21. juliandyke.com 21 © 2013 - Julian Dyke Online Data File Move In Oracle 12.1 and above any data file can be moved online For example: ALTER DATABASE MOVE DATAFILE '/u01/app/oradata/PROD/users01.dbf TO '/u02/app/oradata/PROD/users01.dbf'; The database can be open and accessing the data file while the move is in progress Data files can be moved online: from file system to file system from file system to ASM from ASM to file system from ASM to ASM

22. juliandyke.com 22 © 2013 - Julian Dyke Online Partition Move This feature could be very useful for sites with partitioned tables on tiered storage. Most likely usage is migrating partitions from fast expensive storage (SSD) to slower cheaper storage (SAS or SATA) The Oracle documentation hints that there are a lot of places where this partition move can fail, and the DBMS_PART package contains some subroutines that allow recovery from failures.

23. juliandyke.com 23 © 2013 - Julian Dyke Online Partition Move In Oracle 12c partitions can be moved online Useful for tiered storage Move from SSD to SAS to SATA May be useful with OLTP compression Also works for sub-partitions Not supported in the following cases: For tables owned by SYS For IOTs For heap tables containing object types For heap tables containing bitmap join indexes or domain indexes If database-supplemental logging is enabled When parallel DML or direct path INSERTs are executing on the table

24. juliandyke.com 24 © 2013 - Julian Dyke Online Partition Move Consider the following example CREATE TABLE pcar ( season_key NUMBER, race_key NUMBER, driver_key VARCHAR2(4), team_key VARCHAR2(3), position NUMBER, laps_completed NUMBER, race_points NUMBER ) PARTITION BY RANGE (season_key) ( PARTITION p2010 VALUES LESS THAN (2011) TABLESPACE sas, PARTITION p2011 VALUES LESS THAN (2012) TABLESPACE sas, PARTITION p2012 VALUES LESS THAN (2013) TABLESPACE ssd, PARTITION p2013 VALUES LESS THAN (2014) TABLESPACE ssd ); ALTER TABLE pcar MOVE PARTITION P2012 TABLESPACE sas;

25. juliandyke.com 25 © 2013 - Julian Dyke Online Partition Move If online partition move operation fails, it can be cleaned up manually using: DBMS_PART.CLEANUP_ONLINE_OP DBMS_PART.CLEANUP_ONLINE_OP (,, ); Clean up failed operations on DBMS_PART.CLEANUP_ONLINE_OP (, ); Clean up failed operations on DBMS_PART.CLEANUP_ONLINE_OP ( ); Clean up all failed operations in database DBMS_PART.CLEANUP_ONLINE_OP;

26. juliandyke.com 26 © 2013 - Julian Dyke Index Columns This is a useful new feature that allows multiple indexes to be created with the same column list For any given column list, only one index can be visible at a time. However, this enhancement will allow new indexes to be created invisibly and then made visible at an appropriate time.

27. juliandyke.com 27 © 2013 - Julian Dyke Index Columns Multiple indexes can be created on the same set of columns The following conditions must be met: The indexes must have different properties e.g. type, partitioning, uniqueness Only one of the indexes can be VISIBLE at any given time Recommendation: Check existing databases for indexes that have been made invisible and then forgotten.

28. juliandyke.com 28 © 2013 - Julian Dyke Index Columns Consider the following table and global index CREATE TABLE pcar ( season_key NUMBER, race_key NUMBER, driver_key VARCHAR2(4), team_key VARCHAR2(3), position NUMBER, laps_completed NUMBER, race_points NUMBER ) PARTITION BY RANGE (season_key) ( PARTITION p2010 VALUES LESS THAN (2011), PARTITION p2011 VALUES LESS THAN (2012), PARTITION p2012 VALUES LESS THAN (2013), PARTITION p2013 VALUES LESS THAN (2014) ); CREATE INDEX pcar_global ON pcar (season_key,race_key,position);

29. juliandyke.com 29 © 2013 - Julian Dyke Index Columns We realise the index should be local so we can drop partitions efficiently The following statement fails with ORA-01408 CREATE INDEX pcar_local ON pcar (season_key,race_key,position) LOCAL; * ERROR at line 1: ORA-01408: such column list already indexed Create the new index INVISIBLE CREATE INDEX pcar_local ON pcar (season_key,race_key,position) LOCAL INVISIBLE; Index created* ALTER INDEX pcar_global INVISIBLE; ALTER INDEX pcar_local VISIBLE; Switch the indexes The new index (PCAR_LOCAL) is now visible The old index (PCAR_GLOBAL) can be dropped

30. juliandyke.com 30 © 2013 - Julian Dyke Invisible Columns I strongly believe this is a very dangerous feature. Whilst it achieves its objectives, it is open to both accidental and malicious misuse as shown in the example. Misuse of this feature could introduce data corruptions that may go unnoticed for months or years and prove to be extremely difficult to resolve

31. juliandyke.com 31 © 2013 - Julian Dyke Invisible Columns Consider the following table: CREATE TABLE icar ( season_keyNUMBER, race_keyNUMBER, driver_keyVARCHAR2(4), team_keyVARCHAR2(3), positionNUMBER, laps_completedNUMBER, race_pointsNUMBER ); DESCRIBE icar NameNull?Type SEASON_KEYNUMBER RACE_KEYNUMBER DRIVER_KEYVARCHAR2(4) TEAM_KEYVARCHAR2(3) POSITIONNUMBER LAPS_COMPLETEDNUMBER RACE_POINTSNUMBER

32. juliandyke.com 32 © 2013 - Julian Dyke Invisible Columns In the data dictionary COL$ contains the following rows for the ICAR table SELECT c.name,c.type#,c.col#,c.intcol#,c.segcol#, TO_CHAR (c.property,'XXXXXXXXXXXX') AS property FROM sys.col$ c, sys.obj$ o, sys.user$ u WHERE c.obj# = o.obj# AND o.owner# = u.user# AND u.name = 'GP AND o.name = 'ICAR'; NAMETYPE#COL#INTCOL#SEGCOL#PROPERTY SEASON_KEY21110 RACE_KEY22220 DRIVER_KEY13330 TEAM_KEY14440 POSITION 25550 LAPS_COMPLETED26660 RACE_POINTS 27770

33. juliandyke.com 33 © 2013 - Julian Dyke Invisible Columns Make the LAPS_COMPLETED column invisible: ALTER TABLE icar MODIFY laps_completed INVISIBLE; DESCRIBE icar NameNull?Type SEASON_KEYNUMBER RACE_KEYNUMBER DRIVER_KEYVARCHAR2(4) TEAM_KEYVARCHAR2(3) POSITIONNUMBER RACE_POINTSNUMBER Describe the table again The LAPS_COMPLETED column is now invisible

34. juliandyke.com 34 © 2013 - Julian Dyke Invisible Columns In the data dictionary COL$ now contains the following rows for ICAR: SELECT c.name,c.type#,c.col#,c.intcol#,c.segcol#, TO_CHAR (c.property,'XXXXXXXXXXXX') AS property FROM sys.col$ c, sys.obj$ o, sys.user$ u WHERE c.obj# = o.obj# AND o.owner# = u.user# AND u.name = 'GP AND o.name = 'ICAR'; NAMETYPE#COL#INTCOL#SEGCOL#PROPERTY SEASON_KEY21110 RACE_KEY22220 DRIVER_KEY13330 TEAM_KEY14440 POSITION 25550 LAPS_COMPLETED2066400000020 RACE_POINTS 26770 0x400000000 = Invisible Column? 0x20 = Hidden Column

35. juliandyke.com 35 © 2013 - Julian Dyke Invisible Columns Make the LAPS_COMPLETED column visible again: ALTER TABLE icar MODIFY laps_completed VISIBLE; DESCRIBE icar NameNull?Type SEASON_KEYNUMBER RACE_KEYNUMBER DRIVER_KEYVARCHAR2(4) TEAM_KEYVARCHAR2(3) POSITIONNUMBER RACE_POINTSNUMBER LAPS_COMPLETEDNUMBER The LAPS_COMPLETED column now appears at end of table Describe the table again:

36. juliandyke.com 36 © 2013 - Julian Dyke Invisible Columns In the data dictionary COL$ now contains the following rows for ICAR: SELECT c.name,c.type#,c.col#,c.intcol#,c.segcol#, TO_CHAR (c.property,'XXXXXXXXXXXX') AS property FROM sys.col$ c, sys.obj$ o, sys.user$ u WHERE c.obj# = o.obj# AND o.owner# = u.user# AND u.name = 'GP AND o.name = 'ICAR'; NAMETYPE#COL#INTCOL#SEGCOL#PROPERTY SEASON_KEY21110 RACE_KEY22220 DRIVER_KEY13330 TEAM_KEY14440 POSITION 25550 LAPS_COMPLETED27660 RACE_POINTS 26770 LAPS_COMPLETED is now COL# 7

37. juliandyke.com 37 © 2013 - Julian Dyke Invisible Columns Why is this dangerous? Consider the following: INSERT INTO icar VALUES (2013,1,'KRAI','LOT',1,58,25); SEASON_KEYRACE_KEYDRIVER_KEYTEAM_KEYPOSITIONLAPS_COMPLETEDRACE_POINTS 2013 1KRAILOT158 25 SELECT * FROM icar; ALTER TABLE icar MODIFY laps_completed INVISIBLE; ALTER TABLE icar MODIFY laps_completed VISIBLE; SEASON_KEYRACE_KEYDRIVER_KEYTEAM_KEYPOSITIONRACE_POINTSLAPS_COMPLETED 2013 1KRAILOT125 58 20131FALOFER25818 INSERT INTO icar VALUES (2013,1,'FALO','FER',2,58,18); SELECT * FROM icar;

38. juliandyke.com 38 © 2013 - Julian Dyke Invisible Columns Continued... ALTER TABLE icar MODIFY race_points INVISIBLE; ALTER TABLE icar MODIFY race_points VISIBLE; SEASON_KEYRACE_KEYDRIVER_KEYTEAM_KEYPOSITIONLAPS_COMPLETEDRACE_POINTS 2013 1KRAILOT158 25 20131FALOFER21858 20131SVETRBR35815 INSERT INTO icar VALUES (2013,1,SVET',RBR',3,58,15); SELECT * FROM icar; Column order is restored, but Fernando Alonso now has 58 points

39. juliandyke.com 39 © 2013 - Julian Dyke Identity Clause This new feature simplifies management of sequences used as primary keys for tables. The identity clause allows an implicit index to be created for the specified column. If the table is truncated, the sequence is unaffected If the table is dropped and recreated the sequence will dropped and recreated and will restart at the minimum value for the next insertion

40. juliandyke.com 40 © 2013 - Julian Dyke Identity Clause In Oracle 12.1 and above an identity clause can be used to specify a sequence column in CREATE TABLE and ALTER TABLE statements GENERATED [ ALWAYS | BY DEFAULT [ ON NULL ] ] AS IDENTITY [ ( identity_options ) ] Syntax is: where are: { START WITH ( integer | LIMIT VALUE ) | INCREMENT BY integer | ( MAXVALUE integer | NOMAXVALUE ) | ( MINVALUE integer | NOMINVALUE ) | ( CYCLE | NOCYCLE ) | ( CACHE integer | NOCACHE ) | ( ORDER | NOORDER ) }...

41. juliandyke.com 41 © 2013 - Julian Dyke Identity Clause Example: CREATE TABLE driver2 ( driver_key NUMBER GENERATED AS IDENTITY, driver_name VARCHAR2(30), driver_dob DATE, country_key VARCHAR2(3) ); INSERT INTO driver2 (driver_name,driver_dob,country_key) VALUES ('Sebastian Vettel','03-JUL-1987','GER'); INSERT INTO driver2 (driver_name,driver_dob,country_key) VALUES ('Fernando Alonso',29-JUL-1981','SPA'); INSERT INTO driver2 (driver_name,driver_dob,country_key) VALUES ('Kimi Raikkonen','17-OCT-1979','FIN'); SELECT * FROM driver2; DRIVER_KEYDRIVER_NAMEDRIVER_DOBCOUNTRY_KEY 1Sebastian Vettel03-JUL-1987GER 2Fernando Alonso29-JUL-1981SPA 3Kimi Raikkonen17-OCT-1979FIN

42. juliandyke.com 42 © 2013 - Julian Dyke Identity Clause DESCRIBE includes identity column DESCRIBE driver2 NameNull?Type DRIVER_KEYNOT NULLNUMBER DRIVER_NAME VARCHAR2(30) DRIVER_DOB DATE COUNTRY_KEY VARCHAR2(3) No additional indexes are created

43. juliandyke.com 43 © 2013 - Julian Dyke Identity Clause Columns are stored in the data dictionary as follows: SELECT c.name,c.type#,c.col#,c.intcol#,c.segcol#, TO_CHAR (c.property,'XXXXXXXXXX') AS property FROM sys.col$ c, sys.obj$ o, sys.user$ u WHERE c.obj# = o.obj# AND o.owner# = u.user# AND u.name = 'GP AND o.name = 'DRIVER2 ORDER BY intcol#; NAMETYPE#COL#INTCOL#SEGCOL#PROPERTY DRIVER_KEY21112800000000 DRIVER_NAME12220 DRIVER_DOB123330 COUNTRY_KEY14440 0x800000000 = Default as Sequence 0x2000000000 = Generated ALWAYS identity column

44. juliandyke.com 44 © 2013 - Julian Dyke Identity Clause Default value for DRIVER_KEY column can be found in DBA_TAB_COLUMNS: SELECT data_default FROM dba_tab_columns WHERE owner = 'GP AND table_name = 'DRIVER2 AND column_name = 'DRIVER_KEY'; "GP"."ISEQ$$_92584".nextval SELECT sequence_owner AS owner,min_value,max_value,increment_by, cycle_flag,order_flag,cache_size FROM dba_sequences WHERE sequence_name = 'ISEQ$$_92584'; OWNERMIN_VALUEMAX_VALUEINCREMENT_BYCOCACHE_SIZE GP11.0000E+281NN20 In this example 92584 is the object ID of the GP.DRIVER2 table

45. juliandyke.com 45 © 2013 - Julian Dyke Session Sequences This new feature allows sequences to be created that exist for the lifetime of the current session only. Intended for use with global temporary tables, but possibly useful in other places and more flexible than the ROWNUM pseudo column

46. juliandyke.com 46 © 2013 - Julian Dyke Session Sequences In Oracle 12.1 and above sequences can have session visibility Current value only visible to session For example: CREATE SEQUENCE seq1 SESSION; SQL> CONNECT gp/gp SQL> SELECT seq1.NEXTVAL FROM dual; NEXTVAL 1 SQL> SELECT seq1.NEXTVAL FROM dual; NEXTVAL 2 SQL> CONNECT gp/gp SQL> SELECT seq1.NEXTVAL FROM dual; NEXTVAL 1

47. juliandyke.com 47 © 2013 - Julian Dyke Global Temporary Table Undo This new feature allows undo for global temporary tables to be written to the temporary table space It will not have much impact for insertions, but could have a significant impact on redo generation caused by GTT undo during updates I envisage this becoming the default in future versions

48. juliandyke.com 48 © 2013 - Julian Dyke Global Temporary Table Undo By default DML on Global Temporary Tables Does not generate redo directly Does generate undo and indirect redo Undo is required to rollback transactions Redo will be archived, backed up, propagated to standby etc In Oracle 12c Global Temporary Table undo can be stored in a temporary tablespace Set TEMP_UNDO_ENABLED = TRUE Will not have much impact for INSERT statements May have significant impact for UPDATE and DELETE statements Review whether DELETE statements are necessary

49. juliandyke.com 49 © 2013 - Julian Dyke Temporal Validity Temporal validity allows tables to be created where rows are valid for a specific period of time A major defect is that is not possible to create primary keys with temporal validity. This functionality may be added in a future release, until which time this feature may be of limited use.

50. juliandyke.com 50 © 2013 - Julian Dyke Temporal Validity Flashback Data Archive was introduced in Oracle 11.1 Originally known as Total Recall Allows historic data to be inspected at any point in time Was a separately licensed option Consequently not very popular Now available free in Enterprise Edition (at least) Including Oracle 11.2 In Oracle 12.1 and above Temporal Validity builds on these concepts

51. juliandyke.com 51 © 2013 - Julian Dyke Temporal Validity For example: CREATE TABLE driver ( driver_keyVARCHAR2(4), team_key VARCHAR2(3), joining_date DATE, leaving_date DATE, PERIOD FOR team_member_valid_time (joining_date,leaving_date) ); INSERT INTO driver VALUES ('FALO','FER','01-JAN-2010',NULL); INSERT INTO driver VALUES ('FMAS','FER','01-JAN-2006','31-DEC-2013'); INSERT INTO driver VALUES ('KRAI','FER','01-JAN-2007','31-DEC-2009'); INSERT INTO driver VALUES ('RBAR','FER','01-JAN-2000','31-DEC-2007'); INSERT INTO driver VALUES ('MSCH','FER','01-JAN-1996','31-DEC-2006'); Insert some data Note: the above data is inaccurate

52. juliandyke.com 52 © 2013 - Julian Dyke Temporal Validity For example: SELECT * FROM driver; DRIVER_KEY TEAM_KEYJOINING_DATE LEAVING_DATE FALOFER 01-JAN-2010 FMASFER01-JAN-200631-DEC-2013 KRAIFER 01-JAN-200731-DEC-2009 RBARFER 01-JAN-200031-DEC-2007 MSCHFER01-JAN-199631-DEC-2006 SELECT * FROM driver AS OF PERIOD FOR team_member_valid_time TO_DATE (20-JUN-2009); DRIVER_KEY TEAM_KEYJOINING_DATE LEAVING_DATE FMASFER01-JAN-200631-DEC-2013 KRAIFER 01-JAN-200731-DEC-2009 Who was in the team for the 2009 British Grand Prix qualifying?

53. juliandyke.com 53 © 2013 - Julian Dyke Temporal Validity Describe the driver table DESCRIBE driver Name Null? Type DRIVER_KEY VARCHAR2(4) TEAM_KEY VARCHAR2(3) JOINING_DATE DATE LEAVING_DATE DATE

54. juliandyke.com 54 © 2013 - Julian Dyke Temporal Validity List the columns in COL$ SELECT c.name,c.col#,c.intcol#,c.segcol#,c.type#,TO_CHAR (c.property,'XXXXX') FROM sys.col$ c, sys.obj$ o WHERE c.obj# = o.obj# AND o.name = 'DRIVER3 ORDER BY c.intcol# NAMECOL#INTCOL#SEGCOL#TYPE#PROPERTY TEAM_MEMBER_VALID_TIME010210028 DRIVER_KEY12110 TEAM_KEY23210 JOINING_DATE343120 LEAVING_DATE454120 0x10000 = Virtual Column 0x20 = Hidden Column 0x8 = Virtual Column

55. juliandyke.com 55 © 2013 - Julian Dyke Extended Columns This feature allows the size of VARCHAR2, NVARCHAR2 and RAW columns stored in the database to be increased to 32767 bytes. If the value is longer than 4000 bytes it is stored as an out of line LOB Built-in functions appear to work correctly with the longer column sizes This feature needs to be enabled by setting MAX_STRING_SIZE to EXTENDED. This parameter is not set by default. You may want to set this parameter before creating a database, otherwise you will need an outage as the parameter must be set when the database is in UPGRADE mode

56. juliandyke.com 56 © 2013 - Julian Dyke Extended Columns In Oracle 12c and above maximum column length has increased Data TypeOracle 11.2 and below Oracle 12.1 and above VARCHAR2400032767 NVARCHAR2200016383 RAW200032767 Note that NVARCHAR2 limits assume two bytes per character Maximum length of CHAR and NCHAR remains at 2000 and 1000 respectively Extended columns are stored as SECUREFILE LOBs Stored in line if <= 4K Stored out of line if > 4K COMPATIBLE parameter must be 12.0.0.0.0 or above

57. juliandyke.com 57 © 2013 - Julian Dyke Extended Columns By default attempts to create an extended column will fail: ALTER SYSTEM SET max_string_size = 'EXTENDED' * ERROR at line 1: ORA-02097: parameter cannot be modified because specified value is invalid ORA-14694: database must in UPGRADE mode to begin MAX_STRING_SIZE migration MAX_STRING_SIZE parameter must be set to EXTENDED Default is value is STANDARD ALTER TABLE car MODIFY notes VARCHAR2(32767); * ERROR at line 1: ORA-00910: specified length too long for its datatype MAX_STRING_SIZE parameter cannot be updated when database is open:

58. juliandyke.com 58 © 2013 - Julian Dyke Extended Columns To change the MAX_STRING_SIZE parameter, restart the database in UPGRADE mode SQL> SHUTDOWN IMMEDIATE SQL> STARTUP MIGRATE Set the parameter value to EXTENDED: SQL> ALTER SYSTEM SET max_string_size = EXTENDED; Run the utl32k.sql script SQL> @$ORACLE_HOME/rdbms/admin/utl32k.sql; Restart the database SQL> SHUTDOWN IMMEDIATE SQL> STARTUP It is not possible to convert the MAX_STRING_SIZE parameter back from EXTENDED to STANDARD

59. juliandyke.com 59 © 2013 - Julian Dyke Extended Columns When MAX_STRING_SIZE is set to EXTENDED then tables can be created with extended columns: CREATE TABLE ecar ( season_keyNUMBER, race_keyNUMBER, driver_keyVARCHAR2(4), team_keyVARCHAR2(3), positionNUMBER, laps_completedNUMBER, notesVARCHAR2(32767), race_pointsNUMBER ); Alternatively maximum size of columns in existing tables can be increased: ALTER TABLE car MODIFY notes VARCHAR2(32767);

60. juliandyke.com 60 © 2013 - Julian Dyke Extended Columns Extended columns are implemented as SECUREFILE LOBs SELECT column_name,segment_name,securefile FROM dba_lobs WHERE owner = 'GP AND table_name = 'ECAR'; COLUMN_NAMESEGMENT_NAMESECUREFILE NOTESSYS_LOB0000092626C00007$$YES SECUREFILE LOBs have a system-created index SELECT column_name,index_name FROM dba_lobs WHERE owner = 'GP AND table_name = 'ECAR'; COLUMN_NAMEINDEX_NAME NOTESSYS_IL0000092626C00007$$

61. juliandyke.com 61 © 2013 - Julian Dyke Row Limiting Clause This feature provides a more comprehensive syntax for Top-N queries The new syntax uses analytic query operations as opposed to regular sort options It is probably worth doing comparative performance tests before adopting the new syntax Beware with the OFFSET clause – each invocation will require a full sort of the data before returning any rows

62. juliandyke.com 62 © 2013 - Julian Dyke Row Limiting Clause In Oracle 12c and above SELECT statements can include the FETCH FIRST clause Limits rows returned by query Optional replacement syntax for TOP-N queries [ OFFSET offset { ROW | ROWS } ] [ FETCH { FIRST | NEXT } [ { rowcount | percent PERCENT } ] { ROW | ROWS } { ONLY | WITH TIES } ] Syntax is: OFFSET specifies number of rows to skip before row limiting begins FETCH specifies number of rows or percentage of rows to return ONLY return exactly the number of rows specified WITH TIES return additional rows with same sort key as last row fetched

63. juliandyke.com 63 © 2013 - Julian Dyke Row Limiting Clause An ORDER BY clause is normally required to ensure that sort order is deterministic Restrictions Cannot be specified in SELECT FOR UPDATE statements Cannot be used with CURRVAL or NEXTVAL pseudo-columns Cannot be used with materialized view incremental refresh

64. juliandyke.com 64 © 2013 - Julian Dyke Row Limiting Clause Example – top ten drivers in 2012 Driver NamePoints Sebastian Vettel281 Fernando Alonso278 Kimi Raikkonen207 Lewis Hamilton190 Jenson Button188 Mark Webber179 Felipe Massa122 Romain Grosjean96 Nico Rosberg93 Sergio Perez66

65. juliandyke.com 65 © 2013 - Julian Dyke Row Limiting Clause Example – Top N query SELECT * FROM ( SELECT d.driver_name,t.team_name,SUM(c.driver_points) FROM car c,driver d,team t WHERE c.season_key = 2012 AND c.driver_key = d.driver_key AND c.team_key = t.team_key GROUP BY d.driver_name,t.team_name ORDER BY SUM(c.driver_points) DESC ) WHERE ROWNUM <= 5; ----------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| ----------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 5 | 335 | 41 (0)| |* 1 | COUNT STOPKEY | | | | | | 2 | VIEW | | 480 | 32160 | 41 (0)| |* 3 | SORT ORDER BY STOPKEY| | 480 | 23520 | 41 (0)| | 4 | HASH GROUP BY | | 480 | 23520 | 41 (0)| |* 5 | HASH JOIN | | 480 | 23520 | 41 (0)| | 6 | TABLE ACCESS FULL | TEAM | 104 | 1248 | 2 (0)| |* 7 | HASH JOIN | | 480 | 17760 | 39 (0)| |* 8 | TABLE ACCESS FULL| CAR | 480 | 8160 | 36 (0)| | 9 | TABLE ACCESS FULL| DRIVER | 493 | 9860 | 3 (0)| ----------------------------------------------------------------------- Driver NamePoints Sebastian Vettel281 Fernando Alonso278 Kimi Raikkonen207 Lewis Hamilton190 Jenson Button188

66. juliandyke.com 66 © 2013 - Julian Dyke Row Limiting Clause Example – Fetch Only Clause SELECT d.driver_name,t.team_name,SUM(c.driver_points) FROM car c,driver d,team t WHERE c.season_key = 2012 AND c.driver_key = d.driver_key AND c.team_key = t.team_key GROUP BY d.driver_name,t.team_name ORDER BY SUM(c.driver_points) DESC FETCH FIRST 5 ROWS ONLY; ------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| ------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 480 | 44640 | 41 (0)| | 1 | SORT ORDER BY | | 480 | 44640 | 41 (0)| |* 2 | VIEW | | 480 | 44640 | 41 (0)| |* 3 | WINDOW SORT PUSHED RANK| | 480 | 23520 | 41 (0)| | 4 | HASH GROUP BY | | 480 | 23520 | 41 (0)| |* 5 | HASH JOIN | | 480 | 23520 | 41 (0)| | 6 | TABLE ACCESS FULL | TEAM | 104 | 1248 | 2 (0)| |* 7 | HASH JOIN | | 480 | 17760 | 39 (0)| |* 8 | TABLE ACCESS FULL | CAR | 480 | 8160 | 36 (0)| | 9 | TABLE ACCESS FULL | DRIVER | 493 | 9860 | 3 (0)| ------------------------------------------------------------------------- Driver NamePoints Sebastian Vettel281 Fernando Alonso278 Kimi Raikkonen207 Lewis Hamilton190 Jenson Button188

67. juliandyke.com 67 © 2013 - Julian Dyke Row Limiting Clause Example – Fetch Percent With Ties Clause SELECT d.driver_name,t.team_name,SUM(c.driver_points) FROM car c,driver d,team t WHERE c.season_key = 2012 AND c.driver_key = d.driver_key AND c.team_key = t.team_key GROUP BY d.driver_name,t.team_name ORDER BY SUM(c.driver_points) DESC FETCH FIRST 20 PERCENT ROWS WITH TIES; ----------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| ----------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 480 | 50880 | 41 (0)| | 1 | SORT ORDER BY | | 480 | 50880 | 41 (0)| |* 2 | VIEW | | 480 | 50880 | 41 (0)| | 3 | WINDOW SORT | | 480 | 23520 | 41 (0)| | 4 | HASH GROUP BY | | 480 | 23520 | 41 (0)| |* 5 | HASH JOIN | | 480 | 23520 | 41 (0)| | 6 | TABLE ACCESS FULL | TEAM | 104 | 1248 | 2 (0)| |* 7 | HASH JOIN | | 480 | 17760 | 39 (0)| |* 8 | TABLE ACCESS FULL| CAR | 480 | 8160 | 36 (0)| | 9 | TABLE ACCESS FULL| DRIVER | 493 | 9860 | 3 (0)| ----------------------------------------------------------------------- Driver NamePoints Sebastian Vettel281 Fernando Alonso278 Kimi Raikkonen207 Lewis Hamilton190 Jenson Button188

68. juliandyke.com 68 © 2013 - Julian Dyke Row Limiting Clause Example – Fetch with Offset Clause SELECT d.driver_name,t.team_name,SUM(c.driver_points) FROM car c,driver d,team t WHERE c.season_key = 2012 AND c.driver_key = d.driver_key AND c.team_key = t.team_key GROUP BY d.driver_name,t.team_name ORDER BY SUM(c.driver_points) DESC OFFSET 5 ROWS FETCH FIRST 5 ROWS ONLY; ------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| ------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 480 | 44640 | 41 (0)| | 1 | SORT ORDER BY | | 480 | 44640 | 41 (0)| |* 2 | VIEW | | 480 | 44640 | 41 (0)| |* 3 | WINDOW SORT PUSHED RANK| | 480 | 23520 | 41 (0)| | 4 | HASH GROUP BY | | 480 | 23520 | 41 (0)| |* 5 | HASH JOIN | | 480 | 23520 | 41 (0)| | 6 | TABLE ACCESS FULL | TEAM | 104 | 1248 | 2 (0)| |* 7 | HASH JOIN | | 480 | 17760 | 39 (0)| |* 8 | TABLE ACCESS FULL | CAR | 480 | 8160 | 36 (0)| | 9 | TABLE ACCESS FULL | DRIVER | 493 | 9860 | 3 (0)| ------------------------------------------------------------------------- Driver NamePoints Mark Webber179 Felipe Massa122 Romain Grosjean96 Nico Rosberg93 Sergio Perez66

69. juliandyke.com 69 © 2013 - Julian Dyke Histograms There are several enhancements to histograms in Oracle 12c. This section concentrates on the increase in maximum number of buckets from 254 to 2048. Increasing the number of buckets allows better cardinalities to be estimated by the optimization, potentially generating more efficient execution plans The increased bucket sizes work for both single column and multi column statistics This is particular useful with my Formula 1 database which (for the period 1961 to 2012) contains 492 drivers and 1289 driver/team combinations.

70. juliandyke.com 70 © 2013 - Julian Dyke Histograms Maximum bucket size increased to 2048 Default bucket size is still 256 For example, an inefficient execution plan has been generated for a query We determine that the root cause is poor cardinality estimates for the DRIVER_KEY column in the CAR table The DRIVER_KEY column has 492 distinct values SELECT COUNT (DISTINCT (driver_key)) AS driver_key FROM car; DRIVER_KEY 492

71. juliandyke.com 71 © 2013 - Julian Dyke Histograms Default statistics collection only gathers minimum and maximum values: dbms_stats.gather_table_stats ( ownname => 'GP', tabname => 'CAR', estimate_percent => NULL ); SELECT COUNT (*) FROM dba_histograms WHERE owner = GP AND table_name = CAR AND column_name = DRIVER_KEY; COUNT (*) 2

72. juliandyke.com 72 © 2013 - Julian Dyke Histograms Collect histograms on the DRIVER_KEY column dbms_stats.gather_table_stats ( ownname => 'GP', tabname => 'CAR', estimate_percent => NULL method_opt => 'FOR COLUMNS driver_key' ); SELECT COUNT (*) FROM dba_histograms WHERE owner = GP AND table_name = CAR AND column_name = DRIVER_KEY; COUNT (*) 75 Default behaviour is to create a maximum of 256 buckets

73. juliandyke.com 73 © 2013 - Julian Dyke Histograms If more than 256 buckets are required, this must be specified explicitly: dbms_stats.gather_table_stats ( ownname => 'GP', tabname => 'CAR', estimate_percent => NULL method_opt => 'FOR COLUMNS driver_key SIZE 2048' ); SELECT COUNT (*) FROM dba_histograms WHERE owner = GP AND table_name = CAR AND column_name = DRIVER_KEY; COUNT (*) 492

74. juliandyke.com 74 © 2013 - Julian Dyke Histograms Multi-Column Statistics DECLARE l_extension_name VARCHAR2(30); BEGIN l_extension_name := dbms_stats.create_extended_stats ( ownname => 'GP', tabname => 'CAR6', extension => '(driver_key,team_key) ); END; BEGIN dbms_stats.gather_table_stats ( ownname => 'GP', tabname => 'CAR6', estimate_percent => NULL, method_opt => 'FOR COLUMNS (DRIVER_KEY,TEAM_KEY) SIZE 2048 ); END;

75. juliandyke.com 75 © 2013 - Julian Dyke Histograms Multi-Column Statistics DECLARE l_extension_name VARCHAR2(30); BEGIN l_extension_name := dbms_stats.create_extended_stats ( ownname => 'GP', tabname => 'CAR6', extension => '(driver_key,team_key) ); END; BEGIN dbms_stats.gather_table_stats ( ownname => 'GP', tabname => 'CAR6', estimate_percent => NULL, method_opt => 'FOR COLUMNS (DRIVER_KEY,TEAM_KEY) SIZE 2048 ); END;

76. juliandyke.com 76 © 2013 - Julian Dyke Histograms Multi-Column Statistics 76 IdOperationNameRowsBytesCost (%CPU)Time 0SELECT STATEMENT1939(0)00:00:01 1SORT AGGREGATE19 2TABLE ACCESS FULLCAR181162939 (0)00:00:01 SELECT COUNT(*) FROM gp.car WHERE driver_key = MSCH' AND team_key = 'FER'; COUNT(*) 181 Correct Cardinality

77. juliandyke.com 77 © 2013 - Julian Dyke Histograms Multi-Column Statistics 77 IdOperationNameRowsBytesCost (%CPU)Time 0SELECT STATEMENT1939(0)00:00:01 1SORT AGGREGATE19 2TABLE ACCESS FULLCAR1162939 (0)00:00:01 SELECT COUNT(*) FROM gp.car WHERE driver_key = MSCH' AND team_key = JOR'; COUNT(*) 1 Correct Cardinality

78. juliandyke.com 78 © 2013 - Julian Dyke Application Continuity This is potentially a very important new feature which allows uncommitted transactions to be replayed in another instance following a RAC or Data Guard failover or session relocation I anticipate many sites will wish to take advantage of this new functionality. Initially I have attempted to create a simple test example of this functionality using a JDBC thin client application, but have so far been unsuccessful. I know that Trivadis have successful created a demonstration of Application Continuity using the Universal Connection Pool (UCP) so it does work. Further investigation is required for the JDBC Thin example. In the meantime this session contains the configuration that I have completed so far.

79. juliandyke.com 79 © 2013 - Julian Dyke Application Continuity Failed transactions are replayed on another instance / database Similar goals to TAF and FCF Better implementation Can be configured for: RAC Data Guard Single Instance Must use one of: Weblogic pool Universal Connection Pool (UCP) JDBC Thin OCI not currently supported Limitations may drive future development decisions e.g. connections pools

80. juliandyke.com 80 © 2013 - Julian Dyke Application Continuity JDBC calls should handle events for the current session such as: Service shutdown Instance failure Network failure Node failure Session will attempt to reconnect again (same or different instance) Failed transactions will be rolled back and re-executed Similar (but not the same) as Database Replay Calls replayed with bind variables etc. Fewer synchronization issues – replay only includes last uncommitted transaction

81. juliandyke.com 81 © 2013 - Julian Dyke Application Continuity Must connect to a user-defined service Not the database service E.g. for single instance database DECLARE l_arr DBMS_SERVICE.SVC_PARAMETER_ARRAY; BEGIN l_arr ('FAILOVER_TYPE') := 'TRANSACTION'; l_arr ('REPLAY_INITIATION_TIMEOUT') := 600; l_arr ('FAILOVER_DELAY') := 3; l_arr ('FAILOVER_RETRIES') := 20; l_arr ('SESSION_STATE_CONSISTENCY') := 'DYNAMIC'; l_arr ('COMMIT_OUTCOME') := 'TRUE'; l_arr ('AQ_HA_NOTIFICATIONS') := 'TRUE'; DBMS_SERVICE.CREATE_SERVICE ( service_name => 'SERVICE1', network_name => 'SERVICE1, parameter_array => l_arr ); END;

82. juliandyke.com 82 © 2013 - Julian Dyke Application Continuity E.g. for a RAC database srvctl add service -db TEST \ -service SERVICE1 \ -preferred TEST1 \ -available TEST2 \ -failovertype TRANSACTION \ -notification TRUE \ -commit_outcome TRUE \ -replay_init_time 600 \ -failoverretry 30 \ -failoverdelay 10 srvctl start service –d TEST –s SERVICE1 –i TEST1 Remember to start the service...

83. juliandyke.com 83 © 2013 - Julian Dyke Application Continuity Client connection string should include values for: TRANSPORT_CONNECT_TIMEOUT CONNECT_TIMEOUT RETRY_COUNT For example: jdbc:oracle:thin:gp/gp@(DESCRIPTION=(TRANSPORT_CONNECT_TIMEOUT=3) (CONNECT_TIMEOUT=60)(RETRY_COUNT=10)(FAILOVER=ON) (ADDRESS=(PROTOCOL=tcp)(PORT=1521)(HOST=vmcluster1-scan.juliandyke.com)) (CONNECT_DATA=(SERVICE_NAME=SERVICE1))) REMOTE_LISTENER database parameter must include SCAN name if clients specify SCAN names Node names if clients specify address list

84. juliandyke.com 84 © 2013 - Julian Dyke Application Continuity Configure the Oracle JDBC 12c Replay Data Source in the property file or in the thin JDBC application e.g. import oracle.jdbc.replay.OracleDataSourceImpl; import oracle.jdbc.replay.ReplayableConnection; OracleDataSourceImpl ods = new OracleDataSourceImpl(); ods.setURL(url); connection = ods.getConnection(); connection.setAutoCommit (false);... ((ReplayableConnection)connection).beginRequest(); # Application processing ((ReplayableConnection)connection).endRequest(); Requires $ORACLE_HOME/jdbc/lib/ojdbc6.jar on CLASSPATH

85. juliandyke.com 85 © 2013 - Julian Dyke Application Continuity Debugging replayable connections Add $ORACLE_HOME/jdbc/lib/ojdbc6_g.jar to the CLASSPATH java -Djava.util.logging.config.file=/home/oracle/appcon/properties J7 Execute using: Writes trace to replay_0.trc.0 Add the following to the properties file oracle.jdbc.internal.replay.level = FINEST handlers = java.util.logging.FileHandler java.util.logging.FileHandler.pattern = /home/oracle/12c/appcon2/replay_%U.trc java.util.logging.FileHandler.limit = 500000000 java.util.logging.FileHandler.count = 1000 java.util.logging.FileHandler.formatter = java.util.logging.XMLFormatter

86. juliandyke.com 86 © 2013 - Julian Dyke Application Continuity Potentially a very powerful feature Easier to implement, test and support than TAF Builds on FCF Applications need to be designed specifically for application continuity Very difficult to retrofit existing applications Special attention required for pseudo columns such as SYSDATE Sequences should use the new KEEP clause If you plan to use this feature in the future, I recommend DBAs become familiar with it in Oracle 12.1 so they can support developments New applications follow the development guidelines for this feature Expect to deploy the new applications in Oracle 12.2

87. juliandyke.com 87 © 2013 - Julian Dyke Thank You For Your Interest info@juliandyke.com