1. Monitoring and Testing I/O
Using Perfmon, Oracle tools and IOMeter for monitoring and load testing the I/O Subsystem

2. Objectives
At the end of this module the student will understand the following tasks and concepts.
Monitoring the I/O Subsystem using Perfmon
Monitoring the I/O Subsystem using Oracle Tools
Load Testing the I/O Subsystem using TTCP

3. Oracle and I/O I/O performance is crucial to Oracle Performance
Data buffers are sensitive to read I/O latency
Log buffers are sensitive to write I/O latency
Any slowdown in I/O performance can cause a significant performance loss for Oracle. Although Oracle uses Buffer Cache very effectively, all data must at some point be read into memory, andall transactions must be written out to the redo log files..

4. Oracle I/O Problems Performance Degradation
Index lookup
10 I/Os
10 x 6 ms = 60 ms (not bad)
10 x 100 ms = 1 sec (noticeable)
Table Scan
30,000 I/Os
30,000 x 6 ms = 90 sec (not bad for a Table Scan)
30,000 x 100 ms = 3,000 sec = 50 minutes
You are getting phone calls and s

5. Monitoring I/O Performance Perfmon
General Tips
Set the sample rate up
1 sec is too fast
Not accurate on all counters
Not enough time to see all the counters
Way to fast for saving the data
Heizenburg Uncertainty Principle
Use both graph and report as appropriate
Save data if possible

6. Monitoring I/O Performance Perfmon
Logical Disk Counters
Lists the Physical Disks by Drive Letter
Good for monitoring file I/O
Physical Disk Counters
Lists the Physical Disks by Physical Disk
Good for monitoring the LUNs or Volumes

7. Monitoring I/O Performance Perfmon
Latencies
IOPS (I/Os Per Second)
Throughput
I/O sizes

8. Monitoring I/O Performance Perfmon
Latencies
Avg. Disk sec/Read
Provides Read Latency Statistics
Avg. Disk sec/Write
Provides Write Latency Statistics

9. Monitoring I/O Performance Latency

10. Monitoring I/O Performance Perfmon
IOPS
Disk Reads/sec
Disk Transfers/sec
Useful for total volume IOPS (remember RAID overhead)
Disk Writes/sec

11. Monitoring I/O Performance IOPS

12. Monitoring I/O Performance Perfmon
Throughput
Disk Bytes/sec
Total Throughput
Disk Read Bytes/sec
Disk Write Bytes/sec

13. Monitoring I/O Performance Throughput

14. Monitoring I/O Performance Perfmon
I/O sizes
Avg. Disk Bytes/Read
Avg. Disk Bytes/Write
These counters provide information that allows you to get a little bit of an idea of what Oracle is doing. When Oracle is doing index lookups or random reads and writes the standard block size of 8 KB will be used and you will see an 8 KB Avg. Disk Bytes/Read and an 8 KB Avg. Disk Bytes/Write.
When Oracle is doing table scans the block size will be much larger. The maximum size is determined by the maximum I/O size allowable by the OS. In addition, the Oracle log writes are variable based on how much data is in the log buffer when the log writer thread is ready to write it out.

15. Monitoring I/O Performance I/O Sizes

16. Monitoring I/O Performance Perfmon
Miscellaneous
Current Disk Queue Length
The current disk queue length could give you an indication that there are performance problems with the I/O subsystem. With SAN storage the disk queue length should not be over 1 or 2. If this number is very high you should investigate further. This is not as good of an indication of a slowdown as the latency counters.

17. Monitoring I/O Performance Oracle Tools
OEM Performance Page
Instance Disk I/O
Instance Throughput

18. Monitoring I/O Performance Oracle Tools
Oracle Counters for Windows Performance Monitor
Add Oracle Data Files counter
physreads/sec
Physwrites/se
Choose data files to monitor
Shows I/O for database files

19. Monitoring I/O Performance Oracle Tools
STATSPACK Reports
Tablespace IO Stats
Data File IO Stats
Tablespace IO Stats for DB: DSSA Instance: dssa4 Snaps: 1 -2
->ordered by IOs (Reads + Writes) desc
Tablespace
Av Av Av Av Buffer Av Buf
Reads Reads/s Rd(ms) Blks/Rd Writes Writes/s Waits Wt(ms)
UNDOTBS4 ,
USERS ,
TEMP
TOOLS ,
SYSTEM
File IO Stats for DB: DSSA Instance: dssa4 Snaps: 1 -2
->ordered by Tablespace, File
Tablespace Filename
SYSTEM /u11/dssa/system01.dbf
TEMP /u11/dssa/temp01.dbf
TOOLS /u11/dssa/tools01.dbf ,
UNDOTBS /u11/dssa/undotbs04.dbf ,
USERS /u11/dssa/users01.dbf ,

20. Load Testing I/O
Used to determine I/O problems in a controlled and scientific manner
Used to identify the upper limits of the storage subsystem
Used to prove or disprove hypothesis or theories

21. Load Testing I/O IOMeter
IOMeter is a public domain utility
Available at
Generates I/O based on your specifications
Provides accurate results
Is repeatable

22. Load Testing I/O IOMeter
Made up of two major components
GUI
Used for configuration and results
Dynamo
Used to generate the load

23. Load Testing I/O IOMeter
Configuring IOMeter
Disk Targets
Network Targets
Access Specifications
Results Display
Test Setup

24. Load Testing I/O Disk Targets
Maximum Disk Size
For random I/Os this must be sufficiently large
# of Outstanding I/Os
Defines concurrency of I/Os
If set to one there will only be one I/O issued at a time and IOPS = 1 / latency
The maximum disk size should be sufficiently large to cause seeking on the disk drives. In some cases, this must be very large. A maximum disk size that is too small will skew your results. The number of outstanding I/Os must be sufficiently high to simulate Oracle asynchronous I/O. If you leave it at the default of 1, only one I/O will be outstanding at time, thus resulting in no queueing.

25. Load Testing I/O Network Targets
Used for network I/O testing

26. Load Testing I/O Access Specifications
Specifies the type of I/O
Block Size
Read/Write ratio
Sequential/Random Ratio
Create new spec for Oracle

27. Load Testing I/O Test Setup
Used to set up automated tests
Run time
Ramp up
Number of workers
Etc.

28. Load Testing I/O Results Display
View the performance during the test
IOPS
Throughput
Latency
CPU Utilization

29. Review
What type of I/O problem causes the biggest problem with Oracle Data Files ?
Name some Perfmon counters to monitor for I/O?
What counter does the Oracle Counters for Windows Performance Monitor add to Perfmon that measures Oracle I/O?
What are the two components of IOMeter?

30. Summary Monitoring the I/O Subsystem using Perfmon
Monitoring the I/O Subsystem using Oracle Tools
Load Testing the I/O Subsystem using TTCP