1. Oracle Cache Fusion Cache Fusion Concepts, Data Block Shipping, and Recovery with Cache Fusion

2. Objectives At the end of this module the student will understand the following tasks and concepts. Cache fusion concepts Block transfers using cache fusion Cache Fusion and Recovery

3. Overview Synchronization Past Images Cache Fusion Scenarios Recovery Methodology and steps Recovery Process Scenarios

4. Synchronization of Concurrent Tasks RAC synchronization through Cache Fusion: Maintains cluster-wide concurrency of resources Maintains cluster-wide concurrency of resources Ensures integrity of shared data Ensures integrity of shared data Data blocks and enqueues are synchronized when nodes within a cluster: Acquire block or enqueue ownership Acquire block or enqueue ownership Release block or enqueue ownership Release block or enqueue ownership

5. Minimize Synchronization Key is to optimally divide tasks among nodes, so that very little synchronization is necessary Minimize inter-node communication, because block access in local cache is cheapest: Block access in local cache ~ 0.01 msec Block access in local cache ~ 0.01 msec Block access in remote cache ~ 2.5 msec Block access in remote cache ~ 2.5 msec Block access on disk ~ 14 msec+ Block access on disk ~ 14 msec+

6. Past Images Past Images introduced in Oracle 9i RAC to maintain data integrity Dirty data block not written to disk immediately Another instance may request the same block for read or write Image of the block is created at owning instance, and is shipped to requesting block Backup image kept at owning block is a Past Image, and is kept in memory for local consistency

7. “Juggling” Data with Multiple Past Images Multiple Past Image versions of a data block may be kept by different instances Upon a checkpoint, only the current image is written to disk; Past Images are discarded In the event of a failure, current version of block can be reconstructed from PIs Since PIs are kept in memory, they aid in avoiding frequent disk writes This avoids “disk pinging” experienced with 8i OPS due to frequent writes to disk Data is “juggled” in memory, without touching down on the disk

8. Cache Fusion Scenario 1: Read/Read Cache Fusion – GCS Processing Global Cache Service (GCS) Block SGA Buffer Cache SGA Buffer Cache (3) Ship (1) Request (4) Inform GCS 1: (Shared, Local) 2: (Shared, Local) (2) Forward Locks: (None)  (Shared, Local) Locks: (Shared, Local) Instance 1Instance 2 Based on “ Oracle 9i RAC Oracle Real Application Clusters Configuration and Interals” Mike Ault, Madhu Tumma

9. Cache Fusion Scenario 2: Write/Write Cache Fusion – GCS Processing Global Cache Service (GCS) Block SGA Buffer Cache SGA Buffer Cache (3) Ship (1) Request (4) Inform GCS 1: (Exclusive, Global) 2: (Null, Global, Past Image) (2) Forward Locks: (None)  (Exclusive, Global) Locks: (Exclusive, Local)  (Null, Global, Past Image) Instance 1Instance 2 Based on “ Oracle 9i RAC Oracle Real Application Clusters Configuration and Interals” Mike Ault, Madhu Tumma

10. Cache Fusion Scenario 3: Write Blocks to Disk – GCS Processing Global Cache Service (GCS) Block SGA Buffer Cache SGA Buffer Cache (2) Forward (4) Notify GCS of write and inform GCS 1: (Exclusive Local) 2: (None) (5) Flush PI Locks: (Exclusive, Global)  (Exclusive, Local) Locks: (Null, Global, Past Image)  (None) Instance 1Instance 2 (1) Write Request (3) Write Based on “ Oracle 9i RAC Oracle Real Application Clusters Configuration and Interals” Mike Ault, Madhu Tumma

11. Online Instance Recovery Steps Instance Failure detected by Cluster Manager and GCS Reconfiguration of GES resources (enqueues); global resource directory is frozen Reconfiguration of GCS resources; involves redistribution among surviving instances One of the surviving instances becomes the “recovering instance”

12. Online Instance Recovery Steps (cont.) SMON process of recovering instance starts first pass of redo log read of the failed instance’s redo log thread SMON finds BWR (block written records) in the redo and removes them as their PI is already written to disk SMON prepares recovery set of the blocks modified by the failed instance but not written to disk Entries in the recovery list are sorted by first dirty SCN SMON informs each block’s master node to take ownership of the block for recovery

13. Online Instance Recovery Steps (cont.) Second pass of log read begins. Redo is applied to the data files. Global Resource Directory is unfrozen

14. RAC Recovery – Solving the Mystery Recovering from a failed RAC instance is like solving a Detective mystery Part of the evidence is missing – by definition, the failed instance is inaccessible at recovery time Part of the evidence is missing – by definition, the failed instance is inaccessible at recovery time There are clues – in this case the lock state and mode of the blocks on the surviving nodes. There are clues – in this case the lock state and mode of the blocks on the surviving nodes. Tip 1: Whichever node has an exclusive lock on a given block has the most recent version of that block Tip 2: If the mode of surviving blocks is local, any change made was on one node ony. If the mode of surviving blocks is global, changes have been made on multiple nodes. Tip 4: If the block mode is global and a node has a past image, there is a newer version of the block on another node.

15. Lock Remastering – (Scenario-1) Exclusive, LocalNone Exclusive, LocalNone Remove from recovery list Recovering Instance A Open Instance B Failed Instance C Lock Held Lock Aquired X Current copy of buffer Based on “ Oracle 9i RAC Oracle Real Application Clusters Configuration and Interals” Mike Ault, Madhu Tumma

16. Lock Remastering – (Scenario-2) Exclusive, Global Null, Global, Past Image Exclusive, Global Null, Global, Past Image Remove from recovery list Recovering Instance A Open Instance B Failed Instance C Lock Held Lock Acquired X Write to disk Based on “ Oracle 9i RAC Oracle Real Application Clusters Configuration and Interals” Mike Ault, Madhu Tumma

17. Lock Remastering – (Scenario-3) NoneExclusive, Local NoneExclusive, Local Remove from recovery list Recovering Instance A Open Instance B Failed Instance C Lock Held Lock Acquired X No need to write to disk Based on “ Oracle 9i RAC Oracle Real Application Clusters Configuration and Interals” Mike Ault, Madhu Tumma

18. Lock Remastering – (Scenario-4) Null, Global, Past Image Exclusive, Global Null, Global, Past Image Exclusive, Global Remove from recovery list Recovering Instance A Open Instance B Failed Instance C Lock Held Lock Acquired X Write to disk Based on “ Oracle 9i RAC Oracle Real Application Clusters Configuration and Interals” Mike Ault, Madhu Tumma

19. Lock Remastering – (Scenario-5) NoneExclusive, Global Null, Global, Past Image None Remove from recovery list Recovering Instance A Open Instance B Failed Instance C Lock Held Lock Acquired X write to disk Based on “ Oracle 9i RAC Oracle Real Application Clusters Configuration and Interals” Mike Ault, Madhu Tumma

20. Lock Remastering – (Scenario-6) None (Exclusive, Local Implied) Exclusive, LocalNone Keep in recovery list Recovering Instance A Open Instance B Failed Instance C Lock Held Lock Acquired X Based on “ Oracle 9i RAC Oracle Real Application Clusters Configuration and Interals” Mike Ault, Madhu Tumma

21. Lock Remastering – (Scenario-7) None Exclusive, Global Null, Global, Past Image Keep in recovery list Recovering Instance A Open Instance B Failed Instance C Lock Held Lock Aquired X Send Cr block to A (Exclusive, Global Implied) Null, Global, Past Image Based on “ Oracle 9i RAC Oracle Real Application Clusters Configuration and Interals” Mike Ault, Madhu Tumma

22. Lock Remastering – (Scenario-8) Null, Global, Past Image Null, Global Exclusive, Global, Past Image Null, Global Keep in recovery list Recovering Instance A Open Instance B Failed Instance C Lock Held Lock Aquired X Send Cr block to A Based on “ Oracle 9i RAC Oracle Real Application Clusters Configuration and Interals” Mike Ault, Madhu Tumma (Exclusive, Global Implied)

23. Review Name two times that data blocks and enqueues are synchronized. True or false: since communication across the private interconect is fast, you want to maximize the amount of inter-node communication. True or false: multiple instances may hold Past Images of the same block at the same time. All cache fusion requests must pass through the ___. True or false: an instance with an exclusive lock with a global resource role contains the most current version of a block.

24. Summary Cache Fusion synchronizes resources cluster-wide such as: Data blocks Data blocks Enqueues Enqueues Past Images are used to maintain data block integrity No immediate disk write required No immediate disk write required Avoids “disk pinging” Avoids “disk pinging” The GCS plays a key role in performing the necessary block transfers Including lock mode conversions Including lock mode conversions Upon recovery, GCS remasters cache resources of a failed node or nodes on one or more recovery nodes