1. Buffer Cache Waits

2. #.2 Copyright 2006 Kyle Hailey Buffer Cache Waits Waits Disk I/O Buffer Busy Library Cache Enqueue SQL*Net Free Buffer Hot Blocks Free lists IO Read Redo Buffer Cache Row Cache Write Complete Cache Buffer Chains Latch

3. #.3 Copyright 2006 Kyle Hailey Buffer Cache Waits  Latch - cache buffers chains  Latch - cache buffer chains LRU  Free Buffer Wait  Buffer Busy Wait  Index  Data  Free lists  Hot Spots  RBS  Write Complete Wait  Local Write Wait

4. #.4 Copyright 2006 Kyle Hailey REDO Log Files Data Files DBWR LGWR User2 User1 User3 Log Buffer Buffer Cache Log Buffer Buffer Cache SGA Library Cache Oracle Memory Stuctures

5. #.5 Copyright 2006 Kyle Hailey Buffer Cache Caches data blocks from disk that are heavily used Buffer Cache Disk ShadowProcess

6. #.6 Copyright 2006 Kyle Hailey Block in cache? How do you know if a block is cached? Do you search all the blocks? Could be 1000s of blocks to search. Shado w Proces s ?

7. #.7 Copyright 2006 Kyle Hailey Buffer Cache Find a block by Take Data file # Block# and Hash Them

8. #.8 Copyright 2006 Kyle Hailey Hashing I  Simple has could be a Mod function  1 mod 4 = 1  2 mod 4 = 2  3 mod 4 = 3  4 mod 4 = 0  5 mod 4 = 1  Using “mod 4” as a hash funtion creates 4 “buckets” to store things

9. #.9 Copyright 2006 Kyle Hailey Hashing II Empty Buckets 1 2 0 3

10. #.10 Copyright 2006 Kyle Hailey Hashing III Data Block 1 2 0 3 Hash Block’s file# block #’s Result in a bucket# Put Block in bucket

11. #.11 Copyright 2006 Kyle Hailey Hashing IV 1 2 0 3

12. #.12 Copyright 2006 Kyle Hailey Hashing V 1 2 0 3 After a while the buckets become populated with blocks

13. #.13 Copyright 2006 Kyle Hailey To Find a Block Buffer Headers Data Blocks Hash bucket 1.Hash the block address 2.Look for header 3.Found, read block in cache 4.Not Found Read block off disk ShadowProcess 124 3

14. #.14 Copyright 2006 Kyle Hailey Cache Buffers Chains 1 2 0 3 A different latch protects each bucket Note: multiple buckets are now protected by one latch 1 2 3 4

15. #.15 Copyright 2006 Kyle Hailey Cache Buffers Chains Hash Buckets Buffer Headers Well Balanced 1 2 3 4 5 6 7 8 9 10101010

16. #.16 Copyright 2006 Kyle Hailey Cache Buffers Chains Hash Buckets Buffer Headers Multiple Buckets protected by same latch v9 onwards 1 2 3

17. #.17 Copyright 2006 Kyle Hailey Cache Buffers Chains : Diag Hash Buckets s1 s1 s1 s1 s1 Sessions Contention if too accesses on a bucket 1 2 3

18. #.18 Copyright 2006 Kyle Hailey Cache Buffers Chains : Diag Hash Buckets Buffer Headers Contention: Too Many Buffers in Bucket 1 2 3

19. #.19 Copyright 2006 Kyle Hailey Cache Buffers Chains : Solution  Fine the SQL causing the problem  Contributing Factors  Updates, inserts on blocks while reading those blocks  Cause multiple copies  Hot root block of an index  Select for update  Using dual instead of x$dual

20. #.20 Copyright 2006 Kyle Hailey Need Free Block Block  If the data block isn’t in cache  Get a free block in the buffer cache  Read it off disk  Add a header  Read the block into the buffer cache Need Free Block to Read in New Data Block

21. #.21 Copyright 2006 Kyle Hailey Finding a Free Block ShadowProcess When a session reads a block Into the bufffer cache how does it find a FREE spot?

22. #.22 Copyright 2006 Kyle Hailey Finding a Free Block Buffer Headers Data Blocks Hash bucket Buffer Headers are also on a LRU linked List

23. #.23 Copyright 2006 Kyle Hailey Cache Buffers LRU list MRU LRU Buffer Headers LRU = Least Recently Used MRU = Least Recently Used Session Shadow

24. #.24 Copyright 2006 Kyle Hailey Free Buffer Wait  Finding a Free Block  Go to the LRU end of data blocks  Look for first non-dirty block  If search too many post DBWR to make free  Free Buffer wait

25. #.25 Copyright 2006 Kyle Hailey Free Buffer Wait  #17  Data Block Cache lack free buffers  Tune by  Increase data blocks  Try to tune DBWR  Inefficient SQL  requesting large # of blocks In V7, sorts could flood the cache with dirty blocks

26. #.26 Copyright 2006 Kyle Hailey Cache Buffers LRU list MRU LRU Used Block gets put on Front Unused blocks get pushed off Buffer Headers

27. #.27 Copyright 2006 Kyle Hailey Cache Buffers LRU Latch MRU LRU Buffer Headers One Latch protects the linked list during changes to the list

28. #.28 Copyright 2006 Kyle Hailey Cache Buffers LRU Latch MRU LRU Mid-Point Insertion Full Table Scan Insertion Point Buffer Headers

29. #.29 Copyright 2006 Kyle Hailey Cache Buffers LRU Latch MRU LRU Mid-Point Insertion Oracle Tracks the touch count of blocks. As the block is pushed to the LRU end, if it’s touch count is 3 or more, it’s promoted to the MRU end

30. #.30 Copyright 2006 Kyle Hailey Cache Buffers LRU Latch : Solution Buffer Headers Multiple Sets _db_block_lru_latches = 8 10gR2 with cpu_count = 2 Set 1 Set 2

31. #.31 Copyright 2006 Kyle Hailey Cache Buffers LRU Latch : Solution Other  Increase Size of Buffer Cache  Reduce Checkpointing Activity  Use Async I/O for DBWR  Increase DBWR write batch size

32. #.32 Copyright 2006 Kyle Hailey Buffer Busy Waits  User 1 tries to change a buffer header  User 2 already has buffer header “locked” 1 2 0 3 User1 User2

33. #.33 Copyright 2006 Kyle Hailey Buffer Busy  14 kinds of blocks Undo Header - more RBS segs data (index and table are different for bbw) segment header – free lists freelist blocks – free lists groups  9 kinds Buffer Busy Waits 100,110,120,130 (read I/O issues) 200,210,220,230, 231 (block change issues)

34. #.34 Copyright 2006 Kyle Hailey Buffer Busy Wait What’s Important:  What kind of block header  Rollback  Data  Index  Table  Segment Header  If Pre 10  Read (100,110,120,130 )  Write (200,210,220,230, 231 )

35. #.35 Copyright 2006 Kyle Hailey Buffer Busy Cases  Undo Header - Not enough rollback segments  Undo Block – hot spot in RBS  Data Block  Table Block w/ DML : Table lacks Free lists  Table I/O : Multiple sessions reading same block into data cache  Note: in 10g, becomes “read by other session”  Index : has hot block, partition or change SQL  Segment Header – add free Lists (with Datablock)  Free List – add free lists groups

36. #.36 Copyright 2006 Kyle Hailey Buffer Busy Solutions  IF Buffer Busy Waits high  First look at v$waitstat  Class  Time/Count SQL> select * from v$waitstat; CLASS COUNT TIME ------------------ ---------- ---------- data block 1 1 segment header 0 0 free list 0 0 undo header 0 0 undo block 0 0 SQL> select * from v$waitstat; CLASS COUNT TIME ------------------ ---------- ---------- data block 1 1 segment header 0 0 free list 0 0 undo header 0 0 undo block 0 0

37. #.37 Copyright 2006 Kyle Hailey Buffer Busy Wait Solutions Buffer wait Statistics DB/Inst: LINUX3/linux3 Snaps: 39-42 -> ordered by wait time desc, waits desc Class Waits Total Wait Time (s) Avg Time (ms) ---------------------- ----------- ------------------- ------------- undo header 1,159 18 16 data block 132 2 12 file header block 45 0 2 segment header 4 0 10 undo block 1 0 0 ------------------------------------------------------------

38. #.38 Copyright 2006 Kyle Hailey Load : Example  4 Sessions running Insert into toto values (null, ‘a’); Commit;

39. #.39 Copyright 2006 Kyle Hailey Lack of Free List : Diag S1 S2 S3 S4

40. #.40 Copyright 2006 Kyle Hailey With Free Lists : Diag S1 S2 S3 S4

41. #.41 Copyright 2006 Kyle Hailey 10g SQL> select * from v$waitstat; CLASS COUNT TIME ------------------ ---------- ---------- 1.data block 89640 13732 2.sort block 0 0 3.save undo block 0 0 4.segment header 1017 2558 5.save undo header 0 0 6.free list 0 0 7.extent map 0 0 8.1st level bmb 204874 18896 9.2nd level bmb 155 2 10.3rd level bmb 0 0 11.bitmap block 0 0 12.bitmap index block 0 0 13.file header block 0 0 14.unused 0 0 15.system undo header 0 0 16.system undo block 0 0 17.undo header 219 122 18.undo block 0 0 NAME P1 P2 P3 ---------------- -------- ------- ------ buffer busy waits file# block# class NAME P1 P2 P3 ---------------- -------- ------- ------ buffer busy waits file# block# class

42. #.42 Copyright 2006 Kyle Hailey write complete waits  #18  Usually happens in tandem with free buffer  Tune by  Increase data block cache  Happens because shadow wants to access blocks that are currently being written to disk by DBWR

43. #.43 Copyright 2006 Kyle Hailey Local Write Wait  #25  Truncating a table  Wait for Data Cache to be cleared of all blocks of truncated table  Wait by shadow for DBWR

44. #.44 Copyright 2006 Kyle Hailey Summary Buffer Cache Waits  Latch - cache buffers chains: find SQL  Latch - cache buffer chains LRU  Free Buffer Wait : increase cache size, tune SQL  Buffer Busy Wait  Index : aleveate hot spots  Data DML : add free lists  Data Read: change app, tune I/O  RBS : more RBS segs, use UNDO  Write Complete Waits –: increase cache size  Local Write Wait : truncates / reduce cache size