1. Real Application Clusters Trevor Crljenko Emilija Knezevic

2. Objectives Background Information A Working Database Database Creation and Maintenance Summary Q & A

3. Real Application Clusters Emilija and Trevor working on the Linux implementation of the RAC

4. Architectural Characteristics Each instance has its own: –SGA and background processes –Redo Logs –Undo Tablespaces Data files and control files are shared Cache Fusion guarantees cache coherency among cluster nodes

5. Basic RAC Layout Instance A Comm Layer Shared Disk Driver Node 2 Cluster Interconnect Shared Disk Subsystem Oracle Cluster Manager Node 1 Shared Disk Driver Comm Layer Oracle Cluster Manager Instance B

6. Oracle Cluster Manager Operating System dependant Provides a cluster-wide view of node membership –discovers new nodes –tracks the state of existing nodes –coordinates session tracking when a node fails

7. Cache Fusion Enables sharing of database caches among multiple instances to avoid I/O calls Synchronizes access to the database Implemented through GCS (Global Cache Services)

8. The Cluster Interconnect Supports the cluster and Cache Fusion Can use: –TCP/IP over Ethernet –User Datagram Protocol (UDP) over Ethernet –Virtual Interface Architecture over a Gigabit network

9. Limitations? The use of Transparent Application Failover (also called TAF) has a few limitations: –Uncommitted transactions must be rolled back –Global temporary tables are lost –PL/SQL package states are lost –The effect of ALTER SESSION is lost

10. A working RAC on the Linux OS A demonstration of a Real Application Cluster Database

11. The Environment Node1:TPOLinstance: RAC1 Node2:SCOTTYinstance: RAC2 Database name: RAC1 Clients: –Referee - runs a heavy query –Scott Both clients on machine TORRES The query takes approx. 4 min. to execute by one uninterrupted instance

12. Scripts Used – gvinstance SELECT inst_id, instance_number inst#, instance_name name, host_name host, startup_time startup, status, parallel, thread#, logins, database_status dbstat, instance_role, active_state state FROM gv$instance

13. Scripts Used – gvsession SELECT inst_id, sid, username, user#, server, machine, program, logon_time FROM gv$session WHERE username IS NOT NULL

14. Scripts Used – gvtaf SELECT inst_id, sid, username, machine, failover_type, failover_method, failed_over FROM gv$session WHERE username IS NOT NULL AND username != 'SYS'

15. Scripts Used – gvbeen_here SELECT bh.inst_id, o.owner, o.object_name, o.object_type, count(distinct bh.block#) "Num. Buffers" FROM dba_objects o, gv$bh bh WHERE o.object_id = bh.objd AND owner != 'SYS' AND owner != 'SYSTEM' GROUP BY o.owner, o.object_name, o.object_type, bh.inst_id ORDER BY 2, 4, 5 DESC

16. The Demo Our RAC

17. TEST CASE Start Up Two Instances Connect Client Start The Query Shut Down A Instance Start Up The Aborted Inst. Connect Another Client Demonstrate The Load Balancing Stage 1 Setup Stage 2: Availability Stage 3: Load Balance

18. Server SCOTTY Server TPOL User REFEREE Real Application Cluster TESTING SETUP

19. Step 1: Start Up Both INSTANCES - TPOL & SCOTTY SYS@TPOL> @ $RAC_SCRIPTS/gvsessiont.sql SYSDATE ------------------- 2004-01-14:12:24:21 INST SYS HOSTUSER USER# SERVER PROGRAM ID ID NAMENAME ------- ------ --------- --------- ------------------------------- ------------------ 122 tpolSYS 0 DEDICATED sqlplus@tpol (TNS V1-V3) 1 18 tpolSYS 0 DEDICATED oracle@tpol (P000) 218 scottySYS 0 DEDICATED sqlplus@scotty (TNS V1-V3) 223 scottySYS 0 DEDICATED oracle@scotty (P000) SYS@SCOTTY> @ $RAC_SCRIPTS/gvinstance.sql INST INST INST HOST STARTUP STATUS INSTANCE_ROLE STATE ID # NAME NAME ------- ------ --------- --------- --------------------------- --------- ------------------------- --------- 2 2 RAC2 scotty 2004-01-14:10:01:17 OPEN PRIMARY_INSTANCE NORMAL 1 1 RAC1 tpol 2004-01-14:11:21:57 OPEN PRIMARY_INSTANCE NORMAL 223 scottyREFEREE 27 DEDICATED oracle@torres (TNS V1-V3) Step 2: Connect User REFEREE to INSTANCE SCOTTY

20. REFEREE@TORRES> select * 2 from tab 3 / TNAME TABTYPE CLUSTERID ------------------------------------------------------------------------------------------ --------------- CERTIFICATIONS TABLE PERSONS TABLE PERSON_CERTIFICATIONS TABLE REF_ASSIGNMENT_REQUEST_QUEUE TABLE ROLE_APPLICANTS TABLE RUN_TIMES TABLE STATS_CERTS_1 TABLE STATS_CERTS_2 TABLE 8 rows selected REFEREE@TORRES> @ $RAC_SCRIPTS/sysdate SYSDATE ------------------- 2004-01-14:12:37:44 Step 3: Start the Query by User REFEREE

21. Step 4: Shut Down INSTANCE SCOTTY Transparent Application Failover User REFEREE Server SCOTTY Server TPOL

22. BEFORE Shut Down INSTANCE SCOTTY SYS@SCOTTY> @ $RAC_SCRIPTS/gvinstance.sql INST INST INST HOST STARTUP STATUS INSTANCE_ROLE STATE ID # NAME NAME ------- ------ --------- --------- --------------------------- --------- ------------------------- --------- 1 1 RAC1 tpol 2004-01-14:11:21:57 OPEN PRIMARY_INSTANCE NORMAL SYS@SCOTTY> @ $RAC_SCRIPTS/inst_no INSTANCE_NUMBER --------------- 2 SYS@SCOTTY> @ $RAC_SCRIPTS/sysdate SYSDATE ------------------- 2004-01-14:12:20:07 2 2 RAC2 scotty 2004-01-14:10:01:17 OPEN PRIMARY_INSTANCE NORMAL AFTER Shut Down INSTANCE SCOTTY

23. Server SCOTTY Server TPOL Transparent Application Failover SYS@SCOTTY> @ $RAC_SCRIPTS/gvinstance.sql INST INST INST HOST STARTUP STATUS INSTANCE_ROLE STATE ID # NAME NAME ------- ------ --------- --------- --------------------------- --------- ------------------------- --------- 11 RAC1 tpol 2004-01-14:11:21:57 OPEN PRIMARY_INSTANCE NORMAL SYS@TPOL> @ $RAC_SCRIPTS/gvsessiont.sql SYSDATE ------------------- 2004-01-14:12:24:21 INST SYS HOSTUSER USER# SERVER PROGRAM ID ID NAMENAME ------- ------ --------- --------- ------------------------------- ------------------ 118 tpolREFEREE 27 DEDICATED sqlplus@torres (TNS V1-V3) 1 22 tpolSYS 0 DEDICATED sqlplus@tpol (TNS V1-V3) Referee

24. SYS@SCOTTY> @ $RAC_SCRIPTS/gvinstance.sql INST INST INST HOST STARTUP STATUS INSTANCE_ROLE STATE ID # NAME NAME ------- ------ --------- --------- --------------------------- --------- ------------------------- --------- 2 2 RAC2 scotty 2004-01-14:12:30:41 OPEN PRIMARY_INSTANCE NORMAL SYS@SCOTTY> @ $RAC_SCRIPTS/gvsessiont.sql INST SYS HOST USER USER# SERVER PROGRAM ID ID NAME NAME ------- ------ ---------- --------- --------- -------------- ------------------------------------- 2 18 scottySYS 0 DEDICATED sqlplus@scotty (TNS V1-V3) STEP 6: Start Up the Aborted INSTANCE SCOTTY

25. Step 7: Connect User SCOTT Load Balancing Server SCOTTY Server TPOL User REFEREE User SCOTT

26. SYS@TPOL> @ $RAC_SCRIPTS/gvsessiont.sql SYSDATE ------------------- 2004-01-14:12:24:21 INST SYS HOSTUSER USER# SERVER PROGRAM ID ID NAMENAME ------- ------ --------- --------- ------------------------------- ------------------ 118 tpolREFEREE 27 DEDICATED sqlplus@torres (TNS V1-V3) 122 tpolSYS 0 DEDICATED sqlplus@tpol (TNS V1-V3) 1 25 tpolSYS 0 DEDICATED oracle@tpol (P000) 218 scottySYS 0 DEDICATED sqlplus@scotty (TNS V1-V3) 219 scottySYS 0 DEDICATED oracle@scotty (P000) 2 24 scotty SCOTT 26 DEDICATED sqlplus@torres (TNS V1-V3) Step 7: Connect User SCOTT Load Balancing

27. RAC Database Creation Verify that the shared disks have been configured correctly Configure the Oracle network services Start the listeners and test the configuration Create the database Add and set up additional instances

28. Database Creation and Maintenance An example of a Real Application Cluster installed on 3 PCs using SuSE Linux 8.1

29. RAC Specific Parameters In the INIT.ORA file *.db_name=RAC1 *.cluster_database=true *.cluster_database_instances=2 rac1.instance_name='RAC1' rac1.instance_number=1 rac1.thread=1 rac1.undo_tablespace='UNDOTBS1' rac2.instance_name='RAC2' rac2.instance_number=2 rac2.thread=2 rac2.undo_tablespace='UNDOTBS2' rac1.local_listener='(ADDRESS=(PROTOCOL = TCP)(HOST = tpol)(PORT = 1521))' rac1.remote_listener='(ADDRESS=(PROTOCOL = TCP)(HOST = scotty)(PORT = 1521))' rac2.local_listener='(ADDRESS=(PROTOCOL = TCP)(HOST = scotty)(PORT = 1521))' rac2.remote_listener='(ADDRESS=(PROTOCOL = TCP)(HOST = tpol)(PORT = 1521))

30. RAC Specific Issues Preconditions for running the CREATE DATABASE script: –GSD should be running on both nodes –Listeners should be running on both nodes tnsping was used to verify the network configuration An UNDO tablespace should be created for each node in the cluster Cluster redo logs: –Each instance has its own thread of redo log files –There are an equal number of redo log groups for each node –All log files must be placed on the shared storage Run the catalog scripts for the RAC database: @ $ORACLE_HOME/rdbms/admin/catclust.sql

31. Add Another Instance The database should be started using the PFILE Create a password file, and a PFILE for the next instance Start the instance RAC2 from the PFILE Create an SPFILE from the PFILE of RAC1 In both PFILEs, remove everything but the pointer to the SPFILE Each instance has its own password file, but they MUST be synchronized across all instances

32. Configuring Transparent Application Failover (TAF) TAF instructs Oracle Net to transfer a failed connection to a different listener A user can continue to work using the new connection TAF configuration: –tnsnames.ora - on both server and client side –set up the 'Local Listener' and 'Remote Listener' in the init file Configuration that works: (CONNECT_DATA = (SERVICE_NAME=RAC1.eis.ca) (FAILOVER_MODE = (TYPE = SELECT) (METHOD = BASIC) (RETRIES = 64) (DELAY = 4))))

33. REDO Logs and Threads SYS@TPOL> @ $RAC_SCRIPTS/gvlog.sql INST_ID GROUP# THREAD# SEQ# BYTES MEMBERS ARC STATUS FIRST_CHAN FIRST_TIME ------- ------ ------- ------ ---------- ---------- --- -------- ---------- --------------- 2 1 1 61 20971520 2 YES INACTIVE 571934296 04-01-14:12:22 2 2 1 62 20971520 2 NO CURRENT 571975370 04-01-14:12:35 2 3 1 60 20971520 2 YES INACTIVE 571810001 04-01-13:06:33 2 4 2 49 20971520 2 YES INACTIVE 571934293 04-01-14:12:22 2 5 2 50 20971520 2 NO CURRENT 571954935 04-01-14:12:31 2 6 2 48 20971520 2 YES INACTIVE 571905460 04-01-14:10:01 1 1 1 61 20971520 2 YES INACTIVE 571934296 04-01-14:12:22 1 2 1 62 20971520 2 NO CURRENT 571975370 04-01-14:12:35 1 3 1 60 20971520 2 YES INACTIVE 571810001 04-01-13:06:33 1 4 2 49 20971520 2 YES INACTIVE 571934293 04-01-14:12:22 1 5 2 50 20971520 2 NO CURRENT 571954935 04-01-14:12:31 1 6 2 48 20971520 2 YES INACTIVE 571905460 04-01-14:10:01 12 rows selected.

34. Switching from NOARCHIVELOG to ARCHIVELOG mode There should be only one instance running, and the instance should be mounted in EXCLUSIVE mode The only way to mount the instance in EXCLUSIVE mode is to set the cluster_database parameter to FALSE)

35. Archivelog List SYS@TPOL> archive log list Database log mode Archive Mode Automatic archival Enabled Archive destination /var/opt/oracle/RAC1/archive Oldest online log sequence 58 Next log sequence to archive 60 Current log sequence 60 SYS@SCOTTY> archive log list Database log mode Archive Mode Automatic archival Enabled Archive destination /var/opt/oracle/RAC1/archive Oldest online log sequence 45 Next log sequence to archive 47 Current log sequence 47

36. Backups with RMAN connected to target database: RAC1 (DBID=410737804) Starting backup at 2004-01-13:14:25:44 channel t1: starting full datafile backupset channel t1: specifying datafile(s) in backupset including current controlfile in backupset input datafile fno=00001 name=/var/opt/oracle/RAC1/system/system01.dbf input datafile fno=00002 name=/var/opt/oracle/RAC1/undo/undotbs1.dbf input datafile fno=00005 name=/var/opt/oracle/RAC1/undo/undotbs2.dbf input datafile fno=00004 name=/var/opt/oracle/RAC1/data/users01.dbf channel t1: starting piece 1 at 2004-01-13:14:26:20 channel t1: finished piece 1 at 2004-01-13:15:01:48 piece handle=/var/opt/oracle/RAC1/backup/rman/RAC1_28_1_515341546.bak comment=NONE channel t1: backup set complete, elapsed time: 00:36:02 channel t1: starting full datafile backupset channel t1: specifying datafile(s) in backupset input datafile fno=00003 name=/var/opt/oracle/RAC1/index/index01.dbf channel t1: starting piece 1 at 2004-01-13:15:01:49 channel t1: finished piece 1 at 2004-01-13:15:02:04 piece handle=/var/opt/oracle/RAC1/backup/rman/RAC1_29_1_515343709.bak comment=NONE channel t1: backup set complete, elapsed time: 00:00:15 Finished backup at 2004-01-13:15:02:04

37. Summary Architecture components How does it work in practice? Creating and Maintaining a Real Application Cluster Database

38. Additional Sources of Information Oracle Online Documentation: Real Application Clusters Concepts Oracle Metalink document # 184821.1: Step-By-Step Installation of 9.2.0.4 RAC on Linux Linux SuSE 8.1 documentation (http://sdb.suse.de/en/sdb/html/)http://sdb.suse.de/en/sdb/html/ Bill Garner, Wally Pereira, Gary Dodge: "Oracle9i Real Application Clusters (RAC) with Red Hat Linux Advanced Server* 2.1 Powered by Intel Itanium 2 Processors", Oracle World, San Francisco, 2003.

39. Our Thanks Enterprise Information Systems OOUG Contact Information –Emilija Knezevic emilija_knezevic@yahoo.com –Trevor Crljenko trevor_crljenko@eis.ca

40. Questions & Answers