Key SharePoint Performance Metrics Marcus Erickson SharePoint Product Manager, Idera Software

What are we going to talk about? Basics of SharePoint performance for IT pros Basics of performance monitoring Key performance metrics What we are not going to talk about: Optimizing your SharePoint code Advanced capacity planning End-to-end SharePoint performance tuning Quick cover what we will talk about

SharePoint performance Goals Keep users happy and productive Solve problems - If content IS NOT loading fast you have to diagnose the underlying cause Capacity Planning - If content IS loading fast gather data for planning and staying ahead of the curve My goals today: Information targeted at IT Pros Which key counters to watch for every day administration Provide lists and resources for light reading at home so you can continue digging deeper Going to focus on basics so you can build on this

Who am I? Marcus Erickson My Background SharePoint Product Manager/Developer at Idera My Background Wrote first program in 1978 – written in Basic a paper feeding teletype machine with an acoustic coupler running at speeds up to 300 baud Plus we had to code uphill in the snow for 10 miles Code lots of things over the years: Handheld Operating System, Linker, Virtual Memory Library, .RTPatch, Voice Recognition Systems, Life Insurance Proposal Systems, etc, etc But for the last 15 years I have focused on the needs of IT: Mission Critical/NetIQ – Windows administration & security Idera - Designed and wrote several of Idera’s SQL Server products including SQL compliance manager, SharePoint diagnostic manager Currently directing my energies to helping SharePoint admins – this market needs tools

SharePoint to an End User Waiting for page to load Waiting SharePoint sucks – its so slow – wrong!!

SharePoint to an Administrator Lots of moving parts Need to architect; need to plan Need to react It’s a real system Image courtesy of SharePoint magazine

Behind the scenes Manage the hardware Keep SharePoint up and fast IT / SP Administrator Content Developers Manage the hardware Keep SharePoint up and fast Must figure out when to add hardware, faster machines etc Want to know about problems before end users Blame developers Create pages and other data – write code Tune/Profile code Don’t want to care about hardware Expect IT to be magical and treat the SharePoint farm as a “cloud” Blame Administrators IT – why cant we get real developers? Content developers – why do we get the old hand-me-down hardware?

Performance Data/Tools Performance Counters SharePoint Health Analyzer WMI PowerShell SharePoint object model MOM SCOM Lots of third party tools Some specific, for one purpose or product; some generic perf monitors There are 1000’s of performance counters out there from Windows, SharePoint and other apps to help you fine tune Around 100 on just SharePoint Indexer alone Luckily its easy for admins – there lots of good data for performance analysis

The key performance indicator CPU, CPU, CPU This is part of everything Processing page requests SQL Server queries Indexing Processes running on machine Etc. If the CPU is overloaded all is slow Causes IT just threw SharePoint on one of their old servers Underpowered machines Too many SharePoint services sharing same machine Too many users hitting same machine Easiest way to do basic scaling is add more power or split programs or content across servers Yes, there are 1000s of metrics available One is king Yes this is common sense but many people jump straight to complex stuff and forget about the basics. If you are lucky enough to have a big budget and can have everything you want then this may not be a problem – but you cant win the Indy 500 in a Model-T Adding servers to a SharePoint farm is not like trying to decrease the time it takes to have a baby by adding more women – adding more CPU power can work

CPU on system that is OK Random spikes are OK You care about average over time

Long periods of high CPU Long periods of high CPU signal that you should pay attention Look for scheduled processes running – building indexes, SQL maintenance , backups – should during off times Other patterns of activity (everyone logging in at 9am)

High Average CPU Check user load on machine (IIS connections) Check if other processes are sucking up CPU Consistently high CPU means you need higher powered machine or need to split things across machines Check for heavy pages

OK, so others are important too SharePoint servers are just Windows based computers So the basics are the top priority: CPU Disk Memory Network Yes they are just servers – but they do a lot of stuff IIS, Indexing, Searches, Crawlers, Excel Calculation, SQL Server, Page Rendering etc SharePoint’s job is to server up content – documents, html pages etc

What should I measure Metric Frequency Average % processor time Daily Disk space – available Disk space where you store backups Memory - Available MBs Memory Memory - % committed bytes SQL Data File Size / Log File Size Weekly – capacity planning Disk – Space used IIS connections Note: these are overtime; you should set up alerts for problems like disk secs/operation; asp requests queued or rejected

Suggested thresholds (over time) All is good Pay Attention Houston, we have a problem CPU Usage < 60% 60-90% > 90% Memory Free % > 50% free 10-50% free <10% free Disk Read/Writes/sec 0-15 15-25 25 Disk Time (use only to see use over time) Avg. Disk sec/Read Avg. Disk sec/Write Avg. Disk sec/Transfer < 15 ms > 15 ms > 20 ms Network bandwidth – bytes total < 40% 41-65% > 65% Network Latency – output queue length 1-2 > 2 These need to be averages over a long period of time – like a day or week – anything can spike Just watch CPU when you wiggle mouse

Example – monitoring disk space Which drives to monitor? Everywhere SharePoint lives Drives with SharePoint databases – data and log files Drives with SharePoint log files Disks with other SharePoint content Metrics Available space MB Available % free Total used SQL data and log file sizes Want to watch current size and growth over time

Watch the basics until there is a problem: Digging into problems will mean looking beyond one counter: Sample Questions Was user load high when problem occurred? Was CPU high while pages loaded slowly? Look at other processes to see what’s keeping CPU from SharePoint? e.g. building indexes during business day; SQL auto growth Was number of connections high while CPU was high? Your system may not scale for high loads Was disk activity high while pages were slow? Is SQL auto growing or defragmenting a database?

Levels of investigation Layer 1 CPU / Memory / Disk/Network Layer 2 IIS – Connections , Requests Queued/Rejected SQL Server – Transactions/sec, Write Transactions/Sec Too much resource contention – segregate data across databases and drives; maybe pages need to be better designed to reduce SQL hits Layer 3 Individual ASP service Perf Counters – e.g. Search, Indexing, Excel

Performance Monitor Windows Performance Monitor is a powerful tool In the old days you were limited to watching live or capturing in a log Now you can also: Basic alerting – start other collections, launch scripts etc Save collections of counters Reports Don’t use default 1 second interval – it will auto-average over time for you! Live doesn’t help you diagnose a problem that happened yesterday! Not your father’s perfmon

Observer Effect For most metrics you want to monitor from remote machines For example it takes CPU and Memory to run perfmon so it will skew your metrics One notable exception is network traffic since then you are sending performance data across network http://www.windowsnetworking.com/articles_tutorials/Run-Performance-Monitor-Locally-Remotely.html

Scripting Also available via scripts so you do what you want with the data PowerShell # List all of the performance counters you can monitor Get-Counter –listSet * | Select-Object -ExpandProperty Paths # Get one sample of the processor time Get-Counter '\Processor(*)\% Processor Time' # Get a sample of the processor time until you press CTRL + C Get-Counter '\Processor(*)\% Processor Time' –Continuous Available via WMI Monitoring Performance using WMI http://msdn.microsoft.com/en-us/library/ms974615.aspx

Suggested Readings Follow

General SP Metrics What deeper SharePoint metrics are available? Of course more and more computer metrics http://technet.microsoft.com/en-us/library/ff758658.aspx Metrics for individual SharePoint services

Suggested Computer Metrics Source: Joel Oleson’s Blog http://blogs.msdn.com/b/joelo/archive/2007/01/16/good-list-of-performance-counters.aspx Replace the server name $fe1$ with your front end and $be1$ with the back end server name.  Front End Performance Counters \\$fe1$\Processor(_Total)\% Processor Time \\$fe1$\Process(LSASS)\% Processor Time \\$fe1$\Process(w3wp)\% Processor Time \\$fe1$\Process(OWSTIMER)\% Processor Time \\$fe1$\Memory\Pages/sec \\$fe1$\Memory\Available Bytes \\$fe1$\System\Context Switches/sec \\$fe1$\Process(w3wp)\Working Set \\$fe1$\Process(w3wp)\Private Bytes \\$fe1$\Process(w3wp)\Page Faults/sec \\$fe1$\Process(w3wp)\Working Set Peak \\$fe1$\Process(w3wp)\Virtual Bytes \\$fe1$\Process(w3wp)\Virtual Bytes Peak \\$fe1$\Process(w3wp)\Page File Bytes \\$fe1$\Process(w3wp)\Page File Bytes Peak \\$fe1$\Process(OWSTIMER)\% Processor Time \\$be1$\PhysicalDisk(_Total)\Disk Write Bytes/sec   Back End (SQL Server) Performance Counters \\$be1$\\Processor(_Total)\% Processor Time \\$be1$\Memory\Pages/sec \\$be1$\System\Context Switches/sec \\$be1$\Process(sqlservr)\% Processor Time \\$be1$\Process(sqlservr)\Working Set \\$be1$\Process(sqlservr)\Private Bytes \\$be1$\SQLServer:SQL Statistics\Batch Requests/sec \\$be1$\PhysicalDisk(_Total)\Current Disk Queue Length \\$be1$\PhysicalDisk(_Total)\Disk Read Bytes/sec

IIS IIS is important to SharePoint – should I add WFEs etc. Some key metrics In addition to per process ones on previous slides: \\$fe1$\ASP.NET\Request Execution Time \\$fe1$\ASP.NET\Request Wait Time \\$fe1$\ASP.NET\Requests Queued \\$fe1$\ASP.NET\Requests Rejected \\$fe1$\ASP.NET\Worker Process Restarts \\$fe1$\ASP.NET\Application Restarts

SQL Server Counters See the load on each SQL Server to decide if you need to split site collections across machines General User Connections Number of logins/logouts Database level information Data file size Log file size Active transactions Transactions/sec Other interesting Locks Deadlocks/sec Lock waits/sec Average wait time Lock wait time Average latch time and more…

SharePoint Services Examples of services with metrics Search Gatherer Search Gather Projects Search Indexer Catalog Search Archival Plug-In Excel Services Web Front End Excel Calculation Services

Resources Marcus Erickson General PowerShell Marcus.Erickson@idera.com www.idera.com General Secrets of SharePoint 10 Steps to Optimize SharePoint Performance by Eric Shupps http://www.idera.com/Content/Resources.aspx Monitoring and Maintaining SharePoint 2010 http://technet.microsoft.com/en-us/library/ff758658.aspx Fast Search 2010 for SharePoint http://technet.microsoft.com/en-us/library/ff383282.aspx The Perfmon Dilemna http://www.infoworld.com/d/data-management/perfmon-dilema-725 Performance monitor – A practical approach http://ftp.dell.com/app/Perfmon.pdf PowerShell My first useful PowerShellscript – capturing performance counters http://sqlblogcasts.com/blogs/sqlandthelike/archive/2009/11/30/my-first-useful-powershell-script-capturing-performance-counters.aspx Get the number of SharePoint connections http://geekswithblogs.net/Lance/archive/2009/06/03/get-the-number-of-current-sharepoint-connections.aspx

Idera Solutions for SharePoint Performance and availability monitoring for SharePoint 11 Tools to simplify SharePoint management Enterprise-class backup and recovery for SharePoint sites Architect, organize and migrate SharePoint content Today’s webcast is sponsored by Idera Software, providing tools for all your SharePoint needs, including SharePoint diagnostic manager and SharePoint migration manager. Idera provides a full suite of powerful and easy-to-use tools to simplify the management of Microsoft SharePoint environments including tools for performance monitoring, administration, backup and recovery and migration. Try a free 14-day trial of any Idera product by visiting www.idera.com. Fast, effective SharePoint security administration Power tools for SharePoint administrators and developers 29

Thank you Marcus Erickson SharePoint Product Manager, Idera Software