1. Testing And Tools For Print And Imaging Drivers

2. Outline Basic testing philosophy Where Microsoft finds bugs
What testing does Microsoft currently do
Microsoft provided test tools
Importance of Debugging
Summary

3. Basic Print and Imaging Testing Philosophy

4. Test Device Drivers Right amount of QA before you ship
Fewer bugs for users to find
Lower support costs
Quality not just the responsibility of QA team
Need good tools for developers
Enable developers to run unit tests

5. Microsoft Investment
Microsoft is interested in the test environment to improve the overall driver ecosystem
Microsoft QA approach
Inbox driver program
Logo Program
Provide tools for drivers we never see
Driver education on tools and methods

6. Where Microsoft Finds Bugs and How They’re Triaged

7. Bug Buckets Main areas bugs are found (highest to lowest)
Through code analysis (PREfast)
Printable regions
Basic printing features
Stress testing
Font handling
Advanced features
Setup
Communications and I/O
Microsoft supplies tools that target each bucket

8. What Happens if Microsoft Finds a Bug?
Reporting the bug
What did you see?
What were you expecting?
Exact steps to reproduce
Triage
Is it in Microsoft code?
Assign to a developer for investigation
Fix
Developer builds private for test to verify
Ship team decides whether to accept the change to the product
Check in
Resolve
Fix is verified against live build by test
Tester who opened original bug is the only person allowed close the issue

9. Microsoft Provided Test Tools

10. Microsoft Testing of Printing and Imaging Device Drivers
Tool Distributions
Winparty 2005
Used for device and driver testing today
Used for inbox driver check in
Included in upcoming Windows Driver Kit (WDK)
WDK
Uses WTT Test Shell
Makes it very simple for anyone to run tests and collect results

11. What Architectures Are Supported?
Tools work on a variety of architectures
x86
x64
Itanium
Inbox check-in requires testing on all three architectures by the IHV and Microsoft
To test a 64bit operating system (OS)
Run 64-bit tools
Run 32-bit tools
Get 64-bit machines today
Required for WOW64 testing

12. Microsoft Provided Imaging Test Tools
Data transfer tool
Error Handler test tool
Filters test tool
Properties and Item Tree test tool
Imaging Driver Robustness tool
Twain Spec Compliance tool
Power Management Stress test tool
Twain Datasource Reliability/Robustness tool
WIA Navigation and Property tool
WIA Trace Viewer

13. Microsoft Provided Print Test Tools
Print Device Capabilities
Print Area
Photo Print test
Print Gremlin
Office Tests
Inbox App tests
DevMode Corruptor
Print Driver Stress
Font Test

14. Importance of Debugging

15. How Does a Debugging Environment Help?
Increases stability of product
Increases customer satisfaction
Increases tester output
Increases developer output
Lowers cost of development
Lowers cost of testing
Lowers support costs

16. Invest in Your QA Process
Example of testing without debugging:
Tester notices applet crashes every few minutes, disregards break
Example with debugging
Tester notices applet crashes, creates debugger break
Passed on to developer, developer finds heap corruption ( developer finds code bug )
Fixed before driver is released
Avoids support costs
Increases stability of product
Avoids customer dissatisfaction
Every single debugger break is a bug to investigate

17. Kernel vs. Local Debugging
Easy to set up:Two approaches
User mode debugging
Easy to set up
Lots of cmd windows on screen
Can’t catch kernel-mode bugs
Kernel mode debugger
Harder to set up (separate computer needed)
No visible testing machine effects
Catches kernel-mode bugs
Debugging user mode drivers can be a problem
e.g. Spooler context and loading symbols

18. Debugging Tools PREfast for drivers
Performed mostly by developers
You need to have the commitment to fix all the errors
Zero bug count required for inbox check-in
Configurable through filters
Application Verifier, Driver Verifier, clean test systems, Pageheap, NTSD etc.
Expands scope of testing
Performed by developers and testers
Very important for quality testing

19. AppVerif vs. Verifier Application Verifier = appverif.exe
Driver Verifier = verifier.exe
Verifier is a kernel-mode driver debugging tool
Verifer was and continues to be widely adopted to find and fix kernel mode problems
IHVs have found and fixed lots of bugs because of Verifer since Windows 2000
AppVerif is a user-mode driver and application debugging tool
Used when testing user mode components

20. Good Test Debug Setup Development resources to debug a break
User mode or kernel mode debugger
“Clean” test machine
Make use of imaging software
Latest debug tools on debugger and test machine
We have a new tool to make debugging set-up easier!
Winparty 2005

21. Good Test Debug Setup (con’t)
Application Verifier (user mode debugger)
Spooler
Explorer
WIA
IHV spawned process
All test tools
Driver Verifier
All installed kernel-mode components
NTSD is the Microsoft standard user-mode debugger
Enable NTSD on spooler
Enable NTSD on WIA service
Enable NTSD on all IHV-spawned processes
Debugging is for test and development

22. PREfast Overview Static code analysis tool
Run against source code
Automated tool to find code defects
Not limited by test cases
Identify location of bug (easy to fix)
Run on source code, not binaries
Analyzes code for bugs before expensive testing begins

23. PREfast Overview (con’t)
Checks for common coding bugs
Bounds (array bounds violations)
Precedence & stack hogs
HRESULT (abuses of HRESULT type)
Divide by zero errors
Null pointers/invalid pointers It’s just a tool, not a magic bullet
Complements other tools such as Application Verifier, Pageheap, etc.
Does this tool find all the defects? No!

24. PREfast Buffer Over-run Example
extern TCHAR g_szName[MAX_PATH + 1];
static TCHAR c_szServerName[] = "SERVER_NAME";
DWORD dwSize = sizeof(g_szName);
TCHAR szAnsiName[MAX_NAME_LENGTH + 1];
pECB->GetVariable (pECB->ConnID,
c_szServerName, szAnsiName, &dwSize);
Security bulletin (and patch) MS01-023
Detected by PREfast 4.0: Fixed in Windows XP

25. Summary Microsoft provides many printing and imaging test tools
Proper debugging saves time and money and increases customer satisfaction
Testing is not just for testers
Invest in your QA org

26. Call to Action
Use Microsoft tools and methods to improve the efficiency of your QA org
Setup proper debugging to find bugs
Work with Microsoft to improve the stability of your device drivers

27. Additional Resources Community Sites Web resources
MSDN Developer Community Chats:
Windows Printer Driver Development
Windows Drivers: Printer Drivers
Windows Drivers Printing and Networking
Web resources
WinFX
WS-Devices Profile
Printer Working Group
WHDC Printing home page:
WHDC Still Imaging / WIA home page:

28. Additional Resources E-mail
For developer print questions: microsoft.com
For developer scan questions: microsoft.com
For developer color questions: microsoft.com

29. © 2005 Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.