1. © 2006 Macrovision Corporation | Company Confidential Continuous Testing in an Agile Environment Nan Krull Manager, Software Quality

2. 2© 2006 Macrovision Corporation | Company Confidential Macrovision enables businesses reliant on the deployment of software or content to secure, adapt and optimize their offering among digital distribution channels and destination devices. Who is Macrovision? NASDAQ: MVSN, founded 1983, public since 1997 Global leader in Digital Value Management Solutions Best-of-breed solutions include technologies such as: –FLEXnet (Software Licensing) –InstallShield/InstallAnywhere (Software Distribution) –ActiveReach / Trymedia / eMeta (DRM) 70,000+ customers 1,200+ patents Offices in the US, UK, France, Germany, Korea, Hong Kong, Japan

3. 3© 2006 Macrovision Corporation | Company Confidential FLEXnet AdminStudio Enterprise suite of applications for repackaging, customizing, testing and deploying application packages in Windows Installer (MSI) environments Enterprise clients include international banks, airlines, software companies – their needs are complex and urgent. AdminStudio must maintain the highest quality standards in order to exceed client expectations.

4. 4© 2006 Macrovision Corporation | Company Confidential Approaches to Quality Linear, Sequential QA planning occurs late No test case review Emphasis on bug fixing

5. 5© 2006 Macrovision Corporation | Company Confidential Approaches to Quality – Continuous Testing QA begins at Requirements, Design QA test planning parallels Dev Feature Test Plan reviewed Feature Test Execution starts sooner Cost-effective bug removal Requires integrated QA and Dev effort

6. 6© 2006 Macrovision Corporation | Company Confidential Approaches to Quality

7. 7© 2006 Macrovision Corporation | Company Confidential Good – but not good enough Defects fixed after Feature Development is complete The “big picture” view of certain risk areas may not be clear until the end of Feature Development Late defect fixing creates risk of breakage Late identification of defects and breakage means a higher number of defects in the released product

8. 8© 2006 Macrovision Corporation | Company Confidential Approaches to Quality – Agile Environment QA begins at Requirements, Design QA test planning parallels Dev Feature Test Plan reviewed Feature Test Execution starts sooner Cost-effective bug removal Requires integrated QA and Dev effort In an Agile environment, as the process is compressed these benefits are accelerated.

9. 9© 2006 Macrovision Corporation | Company Confidential Agile Environment Granular: User Stories, not Features Daily stand-up meetings keep information flowing Code is tested as soon as it’s done Code is fixed before more code is written Agile may appear more casual – but executing “Agile” requires discipline, communication and teamwork.

10. 10© 2006 Macrovision Corporation | Company Confidential Continuous Testing in an Agile Environment

11. 11© 2006 Macrovision Corporation | Company Confidential What difference does really it make? Significantly improved product quality Reduced Risk Production Cost Savings

12. 12© 2006 Macrovision Corporation | Company Confidential Assessing Quality 83%* of bugs originate in the Requirements and Design stages of a project. Approx. half of software development effort is spent on testing. *Quality Assurance Institute, 2006 **Economics of Continuous Testing, Stephen A. Bender, The Quality Connection, 2006

13. 13© 2006 Macrovision Corporation | Company Confidential AdminStudio – Improved Product Quality

14. 14© 2006 Macrovision Corporation | Company Confidential AdminStudio – Improved Product Quality

15. 15© 2006 Macrovision Corporation | Company Confidential Economics of Continuous Testing* Requirements (50%) 20 errors injected Defect cost=$1 Design (50%) 20 errors injected Defect cost=$1 Code (50%) 20 errors injected Defect cost=$1 Test 80% errors detected Defect cost=$10 Production ”0” Defects Defect cost=$100 Continuous Testing Accumulated Accumulated Errs Remaining Total Cost 10 $10 15 $25 18 $42 4 $182 0 $582 Traditional Testing Accumulated Test Cost Errs Remaining $0 20 0 40 0 60 $480 12 $1680 0 *Economics of Continuous Testing, Stephen A. Bender, The Quality Connection, 2006

16. 16© 2006 Macrovision Corporation | Company Confidential Agile maximizes Continuous models Early QA involvement prevents defects: –Fixing defects costs less than releasing them –Preventing defects cost less than fixing them! Agile results in earliest possible ID of: –Critical Issues –Breakage Development gets immediate feedback on quality of each piece of code – and fixes it before coding the next piece! System Test results in more stable builds and less time handling issues

17. 17© 2006 Macrovision Corporation | Company Confidential Results Higher quality product Lower risk of releasing defects Lower defect handling costs Earlier flagging of issues which impact the schedule Better Quality = Satisfied Customers, Increased Demand Everybody Wins!