Real Application Clusters Trevor Crljenko Emilija Knezevic

Objectives Background Information A Working Database Database Creation and Maintenance Summary Q & A

Real Application Clusters Emilija and Trevor working on the Linux implementation of the RAC

Architectural Characteristics Each instance has its own: –SGA and background processes –Redo Logs –Undo Tablespaces Data files and control files are shared Cache Fusion guarantees cache coherency among cluster nodes

Basic RAC Layout Instance A Comm Layer Shared Disk Driver Node 2 Cluster Interconnect Shared Disk Subsystem Oracle Cluster Manager Node 1 Shared Disk Driver Comm Layer Oracle Cluster Manager Instance B

Oracle Cluster Manager Operating System dependant Provides a cluster-wide view of node membership –discovers new nodes –tracks the state of existing nodes –coordinates session tracking when a node fails

Cache Fusion Enables sharing of database caches among multiple instances to avoid I/O calls Synchronizes access to the database Implemented through GCS (Global Cache Services)

The Cluster Interconnect Supports the cluster and Cache Fusion Can use: –TCP/IP over Ethernet –User Datagram Protocol (UDP) over Ethernet –Virtual Interface Architecture over a Gigabit network

Limitations? The use of Transparent Application Failover (also called TAF) has a few limitations: –Uncommitted transactions must be rolled back –Global temporary tables are lost –PL/SQL package states are lost –The effect of ALTER SESSION is lost

A working RAC on the Linux OS A demonstration of a Real Application Cluster Database

The Environment Node1:TPOLinstance: RAC1 Node2:SCOTTYinstance: RAC2 Database name: RAC1 Clients: –Referee - runs a heavy query –Scott Both clients on machine TORRES The query takes approx. 4 min. to execute by one uninterrupted instance

Scripts Used – gvinstance SELECT inst_id, instance_number inst#, instance_name name, host_name host, startup_time startup, status, parallel, thread#, logins, database_status dbstat, instance_role, active_state state FROM gv$instance

Scripts Used – gvsession SELECT inst_id, sid, username, user#, server, machine, program, logon_time FROM gv$session WHERE username IS NOT NULL

Scripts Used – gvtaf SELECT inst_id, sid, username, machine, failover_type, failover_method, failed_over FROM gv$session WHERE username IS NOT NULL AND username != 'SYS'

Scripts Used – gvbeen_here SELECT bh.inst_id, o.owner, o.object_name, o.object_type, count(distinct bh.block#) "Num. Buffers" FROM dba_objects o, gv$bh bh WHERE o.object_id = bh.objd AND owner != 'SYS' AND owner != 'SYSTEM' GROUP BY o.owner, o.object_name, o.object_type, bh.inst_id ORDER BY 2, 4, 5 DESC

The Demo Our RAC

TEST CASE Start Up Two Instances Connect Client Start The Query Shut Down A Instance Start Up The Aborted Inst. Connect Another Client Demonstrate The Load Balancing Stage 1 Setup Stage 2: Availability Stage 3: Load Balance

Server SCOTTY Server TPOL User REFEREE Real Application Cluster TESTING SETUP

Step 1: Start Up Both INSTANCES - TPOL & $RAC_SCRIPTS/gvsessiont.sql SYSDATE :12:24:21 INST SYS HOSTUSER USER# SERVER PROGRAM ID ID NAMENAME tpolSYS 0 DEDICATED (TNS V1-V3) 1 18 tpolSYS 0 DEDICATED (P000) 218 scottySYS 0 DEDICATED (TNS V1-V3) 223 scottySYS 0 DEDICATED $RAC_SCRIPTS/gvinstance.sql INST INST INST HOST STARTUP STATUS INSTANCE_ROLE STATE ID # NAME NAME RAC2 scotty :10:01:17 OPEN PRIMARY_INSTANCE NORMAL 1 1 RAC1 tpol :11:21:57 OPEN PRIMARY_INSTANCE NORMAL 223 scottyREFEREE 27 DEDICATED (TNS V1-V3) Step 2: Connect User REFEREE to INSTANCE SCOTTY

select * 2 from tab 3 / TNAME TABTYPE CLUSTERID CERTIFICATIONS TABLE PERSONS TABLE PERSON_CERTIFICATIONS TABLE REF_ASSIGNMENT_REQUEST_QUEUE TABLE ROLE_APPLICANTS TABLE RUN_TIMES TABLE STATS_CERTS_1 TABLE STATS_CERTS_2 TABLE 8 rows $RAC_SCRIPTS/sysdate SYSDATE :12:37:44 Step 3: Start the Query by User REFEREE

Step 4: Shut Down INSTANCE SCOTTY Transparent Application Failover User REFEREE Server SCOTTY Server TPOL

BEFORE Shut Down INSTANCE $RAC_SCRIPTS/gvinstance.sql INST INST INST HOST STARTUP STATUS INSTANCE_ROLE STATE ID # NAME NAME RAC1 tpol :11:21:57 OPEN PRIMARY_INSTANCE $RAC_SCRIPTS/inst_no INSTANCE_NUMBER $RAC_SCRIPTS/sysdate SYSDATE :12:20: RAC2 scotty :10:01:17 OPEN PRIMARY_INSTANCE NORMAL AFTER Shut Down INSTANCE SCOTTY

Server SCOTTY Server TPOL Transparent Application $RAC_SCRIPTS/gvinstance.sql INST INST INST HOST STARTUP STATUS INSTANCE_ROLE STATE ID # NAME NAME RAC1 tpol :11:21:57 OPEN PRIMARY_INSTANCE $RAC_SCRIPTS/gvsessiont.sql SYSDATE :12:24:21 INST SYS HOSTUSER USER# SERVER PROGRAM ID ID NAMENAME tpolREFEREE 27 DEDICATED (TNS V1-V3) 1 22 tpolSYS 0 DEDICATED (TNS V1-V3) Referee

@ $RAC_SCRIPTS/gvinstance.sql INST INST INST HOST STARTUP STATUS INSTANCE_ROLE STATE ID # NAME NAME RAC2 scotty :12:30:41 OPEN PRIMARY_INSTANCE $RAC_SCRIPTS/gvsessiont.sql INST SYS HOST USER USER# SERVER PROGRAM ID ID NAME NAME scottySYS 0 DEDICATED (TNS V1-V3) STEP 6: Start Up the Aborted INSTANCE SCOTTY

Step 7: Connect User SCOTT Load Balancing Server SCOTTY Server TPOL User REFEREE User SCOTT

@ $RAC_SCRIPTS/gvsessiont.sql SYSDATE :12:24:21 INST SYS HOSTUSER USER# SERVER PROGRAM ID ID NAMENAME tpolREFEREE 27 DEDICATED (TNS V1-V3) 122 tpolSYS 0 DEDICATED (TNS V1-V3) 1 25 tpolSYS 0 DEDICATED (P000) 218 scottySYS 0 DEDICATED (TNS V1-V3) 219 scottySYS 0 DEDICATED (P000) 2 24 scotty SCOTT 26 DEDICATED (TNS V1-V3) Step 7: Connect User SCOTT Load Balancing

RAC Database Creation Verify that the shared disks have been configured correctly Configure the Oracle network services Start the listeners and test the configuration Create the database Add and set up additional instances

Database Creation and Maintenance An example of a Real Application Cluster installed on 3 PCs using SuSE Linux 8.1

RAC Specific Parameters In the INIT.ORA file *.db_name=RAC1 *.cluster_database=true *.cluster_database_instances=2 rac1.instance_name='RAC1' rac1.instance_number=1 rac1.thread=1 rac1.undo_tablespace='UNDOTBS1' rac2.instance_name='RAC2' rac2.instance_number=2 rac2.thread=2 rac2.undo_tablespace='UNDOTBS2' rac1.local_listener='(ADDRESS=(PROTOCOL = TCP)(HOST = tpol)(PORT = 1521))' rac1.remote_listener='(ADDRESS=(PROTOCOL = TCP)(HOST = scotty)(PORT = 1521))' rac2.local_listener='(ADDRESS=(PROTOCOL = TCP)(HOST = scotty)(PORT = 1521))' rac2.remote_listener='(ADDRESS=(PROTOCOL = TCP)(HOST = tpol)(PORT = 1521))

RAC Specific Issues Preconditions for running the CREATE DATABASE script: –GSD should be running on both nodes –Listeners should be running on both nodes tnsping was used to verify the network configuration An UNDO tablespace should be created for each node in the cluster Cluster redo logs: –Each instance has its own thread of redo log files –There are an equal number of redo log groups for each node –All log files must be placed on the shared storage Run the catalog scripts for the RAC $ORACLE_HOME/rdbms/admin/catclust.sql

Add Another Instance The database should be started using the PFILE Create a password file, and a PFILE for the next instance Start the instance RAC2 from the PFILE Create an SPFILE from the PFILE of RAC1 In both PFILEs, remove everything but the pointer to the SPFILE Each instance has its own password file, but they MUST be synchronized across all instances

Configuring Transparent Application Failover (TAF) TAF instructs Oracle Net to transfer a failed connection to a different listener A user can continue to work using the new connection TAF configuration: –tnsnames.ora - on both server and client side –set up the 'Local Listener' and 'Remote Listener' in the init file Configuration that works: (CONNECT_DATA = (SERVICE_NAME=RAC1.eis.ca) (FAILOVER_MODE = (TYPE = SELECT) (METHOD = BASIC) (RETRIES = 64) (DELAY = 4))))

REDO Logs and $RAC_SCRIPTS/gvlog.sql INST_ID GROUP# THREAD# SEQ# BYTES MEMBERS ARC STATUS FIRST_CHAN FIRST_TIME YES INACTIVE :12: NO CURRENT :12: YES INACTIVE :06: YES INACTIVE :12: NO CURRENT :12: YES INACTIVE :10: YES INACTIVE :12: NO CURRENT :12: YES INACTIVE :06: YES INACTIVE :12: NO CURRENT :12: YES INACTIVE :10:01 12 rows selected.

Switching from NOARCHIVELOG to ARCHIVELOG mode There should be only one instance running, and the instance should be mounted in EXCLUSIVE mode The only way to mount the instance in EXCLUSIVE mode is to set the cluster_database parameter to FALSE)

Archivelog List archive log list Database log mode Archive Mode Automatic archival Enabled Archive destination /var/opt/oracle/RAC1/archive Oldest online log sequence 58 Next log sequence to archive 60 Current log sequence 60 archive log list Database log mode Archive Mode Automatic archival Enabled Archive destination /var/opt/oracle/RAC1/archive Oldest online log sequence 45 Next log sequence to archive 47 Current log sequence 47

Backups with RMAN connected to target database: RAC1 (DBID= ) Starting backup at :14:25:44 channel t1: starting full datafile backupset channel t1: specifying datafile(s) in backupset including current controlfile in backupset input datafile fno=00001 name=/var/opt/oracle/RAC1/system/system01.dbf input datafile fno=00002 name=/var/opt/oracle/RAC1/undo/undotbs1.dbf input datafile fno=00005 name=/var/opt/oracle/RAC1/undo/undotbs2.dbf input datafile fno=00004 name=/var/opt/oracle/RAC1/data/users01.dbf channel t1: starting piece 1 at :14:26:20 channel t1: finished piece 1 at :15:01:48 piece handle=/var/opt/oracle/RAC1/backup/rman/RAC1_28_1_ bak comment=NONE channel t1: backup set complete, elapsed time: 00:36:02 channel t1: starting full datafile backupset channel t1: specifying datafile(s) in backupset input datafile fno=00003 name=/var/opt/oracle/RAC1/index/index01.dbf channel t1: starting piece 1 at :15:01:49 channel t1: finished piece 1 at :15:02:04 piece handle=/var/opt/oracle/RAC1/backup/rman/RAC1_29_1_ bak comment=NONE channel t1: backup set complete, elapsed time: 00:00:15 Finished backup at :15:02:04

Summary Architecture components How does it work in practice? Creating and Maintaining a Real Application Cluster Database

Additional Sources of Information Oracle Online Documentation: Real Application Clusters Concepts Oracle Metalink document # : Step-By-Step Installation of RAC on Linux Linux SuSE 8.1 documentation ( Bill Garner, Wally Pereira, Gary Dodge: "Oracle9i Real Application Clusters (RAC) with Red Hat Linux Advanced Server* 2.1 Powered by Intel Itanium 2 Processors", Oracle World, San Francisco, 2003.

Our Thanks Enterprise Information Systems OOUG Contact Information –Emilija Knezevic –Trevor Crljenko

Questions & Answers