Quality Engineering in Microsoft Office Jeffrey Murray Test Manager PowerPoint, Microsoft Silicon Valley

Over 1600 employees, plus 400 vendors/contractors. Approx 450 employees in San Francisco. Teams on campus: –Microsoft Research –Windows Live Operations –Live Search Labs –MSTV/IPTV/Mediaroom –Tellme –MSN Mobile –MSN Video –Xbox Hardware –Hotmail –MacBU –PowerPoint

Goals for today Learn about the Office product cycle Career Tips Fun Stories

About Me 1990 Graduated SUNY Albany BS in Computer Science and Mathematics Straight to Microsoft Internal tools Tester of no compile tools Bug tracking tools Common Dialog tools Windows Arabic/Hebrew Windows 3.1 Japanese Windows 95 Office Word97,XP,2000 OneNote 2003 Outlook 2007 PowerPoint 14

What Challenges face Office?

Office Challenges World Wide deployment Users want more! New Competitors Hiring and Retaining great talent

Basic Project Management Variables Time –How much time do we have? Resources –Who are they and how many? Scope –What are you going to do and how risky is it?

The Cast Software Development Engineer Software Development Engineer in Test Program Management

Scope – What Features? Feature team makes the decisions –Must align with Office “Pillars” –Must fit into allowed development time –Must be fully resourced Management will approve features via –Adds/Cuts –Product priorities and opportunities –Manage risk

Office Schedule is fixed to Months RTM 8-16 weeks 2 betas 10 weeks each 12 weeks dev + 12 weeks of testing (x3 milestones) 4 weeks of planning followed by 12 weeks of design

Overlapping for Efficiency PlanningCodingValidationPlanningCodingValidation Milestone 1 Milestone 2

Quality Engineering Over the wall Black box VCR

Testing grows up ~ Big Button – can the dev check in? BVT – Release the build to testing? Acceptance – Does my feature work enough to continue Performance – Key sign of something major wrong Comprehensive -- All tests possible

Pushing Quality Further Upstream Testers involved in spec and design phase Feature not “checked in” till beta quality More formal review of test/dev collateral Code reviews earlier in product cycle Advancement in dev tools and auto validation programs

When are you ready to ship? Your set of risky features need to be under control Statistics and trends are all doing the right thing Checklists Bug Charts Dogfood Watson Beta feedback “Show Stopper” Triage committee Automation Performance Accessibility International Localization Planned testing Stress User Scenarios Backward compatibility Configuration Interoperability Political sensitivity UA approval Visual Freeze Programmability Printing Setup and Upgrade Code Optimization Logo

Typical Office Product Bug Trend

Watson 1.We don’t have user steps or data 2.We know what line of code caused the crash and can often guard against it

Career Tips Companies are better at identifying talent within you than you are at picking the best company. Make sure you don’t hold back at an interview. Don’t plan your whole career all at once, you will miss out on interesting opportunities Use that open door You are a professional, act like it When you screw up (and you will) what you do next is critical Ask yourself each week, what do I like about my job? Interview the company beyond the job, a good part of your life will be there.

Stories How I got my Job at Microsoft Copy protection Steve’s laptop OneNote Office pranks –Elevator –Beach –Peanuts –Disco