1. System Performance Assessment Tools for Windows Longhorn Richard G. Russell Development Manager Windows Core Operating System Division rgr @ microsoft.com Microsoft Corporation

2. Introduction Large difference in platform capability moving forward High-end dual core 64-bit gaming system vs. low end value single core system Media center mobile system vs. thin and light or ultra-mobile system There are Microsoft Windows codenamed “Longhorn” client features that need to scale with platform capability: The window manager and themes Video playback Gaming TV Recording How does Longhorn and applications make scaling decisions?

3. Situation Today Scaling decisions are left to applications Microsoft Windows XP makes no material scaling decisions Windows XP provides few tools to help applications make decisions based on platform capability Graphics capability information from D3D is useful for games Application can enumerate features via WMI Only a few applications have their own system assessment code Games are the exception But, many games do a poor job of this – often leaving tunable settings up to the user

4. Longhorn Helps Solve These Problems * See detailed definitions at the end of the deck New Tool: the Windows System Assessment Tool (WinSAT) Built into Longhorn WinSAT assesses system Features* – something that can be enumerated or detected Attributes* – something that must be measured Capabilities* – a fundamental ability to perform a task or action WinSAT provides performance metrics and other information necessary for Longhorn and applications to make scaling decisions

5. WinSAT Helps Answer Scaling Questions Can the system efficiently run with desktop composition enabled? Can the system efficiently support advanced window management and themes? Aero-Express or Aero-Glass Can the system play high definition video without dropping frames? How many channels of TV can a system record simultaneously? How many enemy robot AI agents should a game configure? Can a game enable full water simulation?

6. Better Automatic Configuration is the Goal Thus avoiding: Making the user experiment or guess at the best settings The use of default “lowest common denominator” settings Assuming high-end systems – turning on too many features that will run poorly on some systems Developers avoid having to develop, test, deploy, and maintain custom system assessment code Allow the operating system and applications to better automatically configure themselves to provide the best user experience of which an individual PC is capable

7. What Does WinSAT Do? WinSAT assesses these subsystems: Graphics Memory Processor Storage WinSAT assess: Attributes Features Capabilities Only items that are relevant to Longhorn and application scalability are assessed

8. WinSAT and Assessment Details

9. WinSAT Component Diagram

10. WinSAT.exe WinSAT.exe is a command line tool that contains the individual assessments Graphics, D3D, Memory, Storage, Computation Video Playback, Feature Enumerator* (TBI) Easy to run Lots of parameters for each assessment Command line help built into Windows* (TBI) Must be run as administrator from the command line Canonical output is XML Human readable output to standard output Easy to script * To Be Implemented (TBI) – not present in WinHEC Longhorn Distribution

11. WinSAT.exe Full 32-bit and 64-bit parity Multi-Core/Multi-CPU aware where appropriate Memory assessment Computational assessment Runs on Windows XP (just copy WinSAT.exe) WinSAT will be updated over time to reflect changes and improvements in platform technology

12. Provided via a simple COM interface Provides simple and easy programmatic access to WinSAT and the data store Access to the XML data – returns an MSXML DOM Allows WinSAT.exe to be run With an arbitrary command line With a “formal assessment”* (TBI) From a limited user account* (TBI) A formal assessment is set of assessments that is executed with a pre-defined set of parameters Used to generate metrics used by Longhorn and reported by the WinSAT API The API can be disabled through group policies Off, or administrator only * To Be Implemented (TBI) – not present in WinHEC Longhorn Distribution WinSAT API

13. The Data Store Will only store data from formal assessments scheduled from the API Data from assessments run from user command lines are not managed or accessible through the API Such data is saved in normal XML files The XML schemes are very simple The Schemas will change! A history of formal assessments is kept* (TBI) up to 100 assessments (oldest deleted) Always keeps the first one generated during first user logon The data store files are read-only to all entities except the WinSAT API* (TBI) Data is provided to callers by providing an MSXML DOM * To Be Implemented (TBI) – not present in WinHEC Longhorn Distribution

14. The Graphics Assessment Designed to assess a system’s ability to efficiently run a composited desktop and the Aero themes Metric is Effective Frames Per Second Drives the hardware in a very similar way to the desktop composition engine* (TBI) Used to make decisions about enabling desktop composition and theme level Focused on evaluating shader texture load performance, system to graphics memory bandwidth and back-buffer read-back * To Be Implemented (TBI) – not present in WinHEC Longhorn Distribution

15. The D3D Assessment Designed to assess a system’s ability to render 3D gaming graphics Metric is Effective Frames Per Second Targeted to Pixel Shader v2.0 or better hardware Focused on three aspects: Shader ALU performance Shader texture load performance Post-pixel blend performance Evaluates 8-bit and 16-bit render targets

16. The System Memory Assessment Focused on throughput, not latency Deigned to assess How well large objects can be moved in system memory Metric is Mega bytes per second Methodology Uses MMX or SSE registers for copies Uses temporal reads Uses non temporal writes Uses explicit block pre-fetching (16K stride) All data is aligned

17. The Storage Assessment Is a “sweep” test – divides a physical disk into regions and evaluates the following for each Random read and write performance Sequential read and write performance Uses up to 16 regions Metric is megabytes (MB) per second Reports metrics for each region Reports an aggregated metric for the disk using the geometric mean of each region

18. The Processor Assessment Designed to assess the system’s computational ability Metric is Mega bytes per second processed Is not a synthetic test – uses native Windows components Data compression and decompression Data encryption and decryption Widows Media Video encoding and decoding* (TBI) Values from each sub-assessment aggregated using the geometric mean * To Be Implemented (TBI) – not present in WinHEC Longhorn Distribution

19. Windows XP vs. Longhorn Differences Some assessments may produce moderately different results on Longhorn vs. Windows XP Graphics and D3D: Due to changes in the driver model XPDM vs. LDDM Due to desktop composition Processor assessments: Due to differences in the Windows components used in those assessments

20. Demo

21. Scenarios

22. For the OEM Focus on platform performance as it relates to key Longhorn features Measuring performance consistency of similarly configured models Understand performance differences between models Diagnostics: misconfigured systems may perform poorly For example, the memory test is very good at picking out misconfigured DRAM

23. Independent Software Vendor ISV can use simple (or complex!) rules based on WinSAT metrics and built into an application to make scaling decisions Since WinSAT is built into Longhorn, good correlation data can be collected during internal testing and external ALPHA and BETA testing In short, ISVs can use WinSAT just as Microsoft uses it for Longhorn and our applications

24. Media Player Example User installs a new media player User then tries to play a high definition video clip The player checks the WinSAT data via the API and determines that the system is not capable of effectively playing HD video Note that this happens without running any tests, nor does the player need to guess – it happens immediately The player can then inform the user that they may see frame drops and other video artifacts – do they wish to continue?

25. Longhorn Upgrade User upgrades an existing system to Longhorn WinSAT runs as part of the first user logon experience (this used to be called OOBE) Longhorn uses WinSAT metrics to make configuration decisions and recommendations to the user “Your system is capable of running Aero-Glass, would you like this option configured?”

26. Longhorn Graphics Card Upgrade User installs a new graphics card in their system – upgrading from UMA graphics LH detects this after the reboot and recommends to the user that he or she should reassess the system (the user says yes) WinSAT runs and the system determines that it is now possible to run Aero-Glass The user is informed, much the same way as the previous example

27. Wrapping Up

28. Where is the Development Process? The WinSAT tool is in the WinHEC Longhorn build It is in an early ALPHA state Large correlation study is underway Not all assessments and features are implemented in Longhorn Beta 1 We may add further assessments The metrics may change Tuning and enhancements will happen through the Beta cycles We may make interim versions of WinSAT available on the beta site

29. Key Features and Updates to be Implemented Consumer suitable visuals for the Graphics and D3D assessments Windows Graphics Foundation (WGF) 2.0 support Video decode assessment System feature enumeration Integration with: The desktop composition system First user logon process LUA support WMI integration Detailed technical documentation In Windows Help format Group Policy Support

30. Call to Action Become familiar with WinSAT Correlate the performance of your product with WinSAT metrics Think about how WinSAT metrics can help your application make scaling and configuration decisions Help us make WinSAT better How does it rank systems and components? Is it statistically reliable? How will does it correlate with your product’s performance? Report bugs and problems

31. Community Resources Windows Hardware & Driver Central (WHDC) www.microsoft.com/whdc/default.mspx Technical Communities www.microsoft.com/communities/products/default.mspx Non-Microsoft Community Sites www.microsoft.com/communities/related/default.mspx Microsoft Public Newsgroups www.microsoft.com/communities/newsgroups Technical Chats and Webcasts www.microsoft.com/communities/chats/default.mspx www.microsoft.com/webcasts Microsoft Blogs www.microsoft.com/communities/blogs

32. Additional Resources See the WinSAT command line document on the WinHEC CD Please send your questions to Winsatfb @ microsoft.com We will notify anyone who sends e-mail to this alias of updates as they become available

34. © 2005 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

35. Key Definitions

36. An Attribute Is: A system characteristic that is measured Attributes cannot be predetermined accurately, they must be measured as the system is built WinSAT measures the performance attributes of: System memory The storage sub-system The CPU (or CPUs) The graphics sub-system

37. A Feature Is: A prominent or distinctive system part, component, or characteristic Features are either present or absent Features are not measured, they are detected or enumerated Examples: x64, Hyper Threading, memory size, optical drive type (DVD, CD, writer etc), MMX, SSE, 3DNow, 1394, number of processors (logical and physical), number of cores per processor package, cache size, hardware MPEG-II or WMV decode support, shader 2.0 support

38. A Capability Is: A system’s ability to effectively perform a specific function or task Capabilities are often present or absent but can also be present to a matter of degree, or some measure or metric To be present, a capability requires: A specified (or minimum) level of one or more attributes The presence of one or more specified features Some capabilities can be assessed directly For example, the ability to play high definition video Some must be inferred or determined by correlation with features and measured attributes

39. © 2005 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.