Testing And Tools For Print And Imaging Drivers

Outline Basic testing philosophy Where Microsoft finds bugs What testing does Microsoft currently do Microsoft provided test tools Importance of Debugging Summary

Basic Print and Imaging Testing Philosophy

Test Device Drivers Right amount of QA before you ship Fewer bugs for users to find Lower support costs Quality not just the responsibility of QA team Need good tools for developers Enable developers to run unit tests

Microsoft Investment Microsoft is interested in the test environment to improve the overall driver ecosystem Microsoft QA approach Inbox driver program Logo Program Provide tools for drivers we never see Driver education on tools and methods

Where Microsoft Finds Bugs and How They’re Triaged

Bug Buckets Main areas bugs are found (highest to lowest) Through code analysis (PREfast) Printable regions Basic printing features Stress testing Font handling Advanced features Setup Communications and I/O Microsoft supplies tools that target each bucket

What Happens if Microsoft Finds a Bug? Reporting the bug What did you see? What were you expecting? Exact steps to reproduce Triage Is it in Microsoft code? Assign to a developer for investigation Fix Developer builds private for test to verify Ship team decides whether to accept the change to the product Check in Resolve Fix is verified against live build by test Tester who opened original bug is the only person allowed close the issue

Microsoft Provided Test Tools

Microsoft Testing of Printing and Imaging Device Drivers Tool Distributions Winparty 2005 Used for device and driver testing today Used for inbox driver check in Included in upcoming Windows Driver Kit (WDK) WDK Uses WTT Test Shell Makes it very simple for anyone to run tests and collect results

What Architectures Are Supported? Tools work on a variety of architectures x86 x64 Itanium Inbox check-in requires testing on all three architectures by the IHV and Microsoft To test a 64bit operating system (OS) Run 64-bit tools Run 32-bit tools Get 64-bit machines today Required for WOW64 testing

Microsoft Provided Imaging Test Tools Data transfer tool Error Handler test tool Filters test tool Properties and Item Tree test tool Imaging Driver Robustness tool Twain Spec Compliance tool Power Management Stress test tool Twain Datasource Reliability/Robustness tool WIA Navigation and Property tool WIA Trace Viewer

Microsoft Provided Print Test Tools Print Device Capabilities Print Area Photo Print test Print Gremlin Office Tests Inbox App tests DevMode Corruptor Print Driver Stress Font Test

Importance of Debugging

How Does a Debugging Environment Help? Increases stability of product Increases customer satisfaction Increases tester output Increases developer output Lowers cost of development Lowers cost of testing Lowers support costs

Invest in Your QA Process Example of testing without debugging: Tester notices applet crashes every few minutes, disregards break Example with debugging Tester notices applet crashes, creates debugger break Passed on to developer, developer finds heap corruption ( developer finds code bug ) Fixed before driver is released Avoids support costs Increases stability of product Avoids customer dissatisfaction Every single debugger break is a bug to investigate

Kernel vs. Local Debugging Easy to set up:Two approaches User mode debugging Easy to set up Lots of cmd windows on screen Can’t catch kernel-mode bugs Kernel mode debugger Harder to set up (separate computer needed) No visible testing machine effects Catches kernel-mode bugs Debugging user mode drivers can be a problem e.g. Spooler context and loading symbols

Debugging Tools PREfast for drivers Performed mostly by developers You need to have the commitment to fix all the errors Zero bug count required for inbox check-in Configurable through filters Application Verifier, Driver Verifier, clean test systems, Pageheap, NTSD etc. Expands scope of testing Performed by developers and testers Very important for quality testing

AppVerif vs. Verifier Application Verifier = appverif.exe Driver Verifier = verifier.exe Verifier is a kernel-mode driver debugging tool Verifer was and continues to be widely adopted to find and fix kernel mode problems IHVs have found and fixed lots of bugs because of Verifer since Windows 2000 AppVerif is a user-mode driver and application debugging tool Used when testing user mode components

Good Test Debug Setup Development resources to debug a break User mode or kernel mode debugger “Clean” test machine Make use of imaging software Latest debug tools on debugger and test machine We have a new tool to make debugging set-up easier! Winparty 2005

Good Test Debug Setup (con’t) Application Verifier (user mode debugger) Spooler Explorer WIA IHV spawned process All test tools Driver Verifier All installed kernel-mode components NTSD is the Microsoft standard user-mode debugger Enable NTSD on spooler Enable NTSD on WIA service Enable NTSD on all IHV-spawned processes Debugging is for test and development

PREfast Overview Static code analysis tool Run against source code Automated tool to find code defects Not limited by test cases Identify location of bug (easy to fix) Run on source code, not binaries Analyzes code for bugs before expensive testing begins

PREfast Overview (con’t) Checks for common coding bugs Bounds (array bounds violations) Precedence & stack hogs HRESULT (abuses of HRESULT type) Divide by zero errors Null pointers/invalid pointers It’s just a tool, not a magic bullet Complements other tools such as Application Verifier, Pageheap, etc. Does this tool find all the defects? No!

PREfast Buffer Over-run Example extern TCHAR g_szName[MAX_PATH + 1]; static TCHAR c_szServerName[] = "SERVER_NAME"; DWORD dwSize = sizeof(g_szName); TCHAR szAnsiName[MAX_NAME_LENGTH + 1]; pECB->GetVariable (pECB->ConnID, c_szServerName, szAnsiName, &dwSize); Security bulletin (and patch) MS01-023 Detected by PREfast 4.0: Fixed in Windows XP

Summary Microsoft provides many printing and imaging test tools Proper debugging saves time and money and increases customer satisfaction Testing is not just for testers Invest in your QA org

Call to Action Use Microsoft tools and methods to improve the efficiency of your QA org Setup proper debugging to find bugs Work with Microsoft to improve the stability of your device drivers

Additional Resources Community Sites Web resources http://www.microsoft.com/communities/default.mspx MSDN Developer Community Chats: Windows Printer Driver Development http://msdn.microsoft.com/chats/transcripts/windows/windows_092104.aspx Windows Drivers: Printer Drivers http://msdn.microsoft.com/chats/windows/windows_101602.asp Windows Drivers Printing and Networking http://msdn.microsoft.com/chats/windows/windows_022002.asp Web resources WinFX http://msdn.microsoft.com/downloads/ WS-Devices Profile http://msdn.microsoft.com/webservices/understanding/specs/default.aspx?pull=/library/en-us/dnglobspec/html/devprof.asp Printer Working Group http://www.pwg.org WHDC Printing home page: http://www.microsoft.com/whdc/device/print/default.mspx WHDC Still Imaging / WIA home page: http://www.microsoft.com/whdc/device/stillimage/default.mspx

Additional Resources E-mail For developer print questions: prninfo @ microsoft.com For developer scan questions: wiainfo @ microsoft.com For developer color questions: mscolor @ microsoft.com

© 2005 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.