1. Quality is Everything Geoffrey Bessin Market Manager, Software Quality IBM Rational

2. 2

3. 3 What is “quality”? FURPS+ Functionality Usability Reliability Performance Supportability + others Acceptable GOALRESULT

4. 4 Software “engineering” Software bugs cost the US economy an estimated $59.5 billion in 2002. 1 On average, professional coders make 100 to 150 errors in every thousand lines of code they write. 2 In 2002, canceled projects cost firms $67 billion; overruns on other projects racked up another $21 billion. 3 1 NIST 2 Watts Humphrey, Carnegie Mellon 3 Standish Group

5. 5 Room for improvement DEFECT COUNT RESULTING QUALITY Process Problems Waterfall-like development Poor asset management and measurement Product Problems Lack of proactive validation techniques Data disconnect between IT Operations and Development People Problems No shared responsibility for quality Over-reliance on test team 70% of organization s are only CMM Level 1 50% of total project cost goes toward software maintenance 25% of projects are cancelled; another 25% are late

6. 6 It’s the other person’s fault! “….we estimate that 40 percent of unplanned application downtime is caused by application failures; 40 percent by application errors; and only 20 percent by hardware (server and network), environmental factors (heating, cooling and power failures) and disasters” “70%-80% of project failures are tied to poor analysis”

7. 7 If you’re thinking, “Not my problem” – then you’ve got a problem.

8. 8 Solution Selling 101 SQUEAK!!

9. 9 Solution Selling – Graduate Course

10. 10  Is this problem related to hardware?  Is an obvious application failure logged?  Is this problem related to memory or performance issues of the software?  Which additional data/traces should be captured to accelerate problem determination?  What is the severity of the problem?  Is this problem related to hardware?  Is an obvious application failure logged?  Is this problem related to memory or performance issues of the software?  Which additional data/traces should be captured to accelerate problem determination?  What is the severity of the problem? Operations  Is this problem a defect?  What scenario reproduces the defect?  What is the customer priority?  Is this defect related to a specific configuration?  Is this problem a request for enhancement?  What is the requirement or use case?  What is the customer priority?  Is this problem a defect?  What scenario reproduces the defect?  What is the customer priority?  Is this defect related to a specific configuration?  Is this problem a request for enhancement?  What is the requirement or use case?  What is the customer priority? Tech Support  Is it possible to reproduce the defect?  Which version does it apply to?  Is there a need to configure an environment for defect reproduction?  Which use case does this defect relate to?  Is it possible to reproduce the defect?  Which version does it apply to?  Is there a need to configure an environment for defect reproduction?  Which use case does this defect relate to? Tester DeploymentTransitionConstructionElaborationInceptionDeploymentTransitionConstructionElaborationInception Packaged Application RDBMS Web Service Jservice JSPs, struts, Servlets Enterprise A CICS Maintenance of Existing Systems and Software App Servers EJBs App Servers Web Server EJBs Edge Server Enterprise B Traceability Test Case Execution Trace Test Case Defect Code

11. 11  Is it possible to reproduce the defect?  Which version does it apply to?  Is there a need to configure an environment for defect reproduction?  Which use case does this defect relate to?  Is it possible to reproduce the defect?  Which version does it apply to?  Is there a need to configure an environment for defect reproduction?  Which use case does this defect relate to? Tester  What is the business priority of the defect?  What is the impact on existing customers?  How quickly is a fix needed?  What is the relative importance of this defect compared to high-priority enhancements?  What is the business priority of the defect?  What is the impact on existing customers?  How quickly is a fix needed?  What is the relative importance of this defect compared to high-priority enhancements? Analyst  What is the impact of this defect?  Which components realize the defective scenario?  Who should be working on the defect?  How much time should be allocated to fix it?  Is there any foreseeable architectural impact?  What is the impact of this defect?  Which components realize the defective scenario?  Who should be working on the defect?  How much time should be allocated to fix it?  Is there any foreseeable architectural impact? Architect InceptionDeploymentTransitionConstructionElaboration Packaged Application RDBMS Web Service Jservice JSPs, struts, Servlets Enterprise A CICS Maintenance of Existing Systems and Software App Servers EJBs App Servers Web Server EJBs Edge Server Enterprise B Traceability Test Case FEATreq Architecture Code Execution Trace Test Case Defect Planning

12. 12  What is the impact of this defect?  Which components realize the defective scenario?  Who should be working on the defect?  How much time should be allocated to fix it?  Is there any foreseeable architectural impact?  What is the impact of this defect?  Which components realize the defective scenario?  Who should be working on the defect?  How much time should be allocated to fix it?  Is there any foreseeable architectural impact? Architect  Which components are suspect?  Are there any clues in the captured traces?  Is this defect reproducible in the development or test environment?  Is there a need for additional traces?  What is the foreseeable impact and risk?  What other components are impacted?  How much time is needed to fix the issue?  Which components are suspect?  Are there any clues in the captured traces?  Is this defect reproducible in the development or test environment?  Is there a need for additional traces?  What is the foreseeable impact and risk?  What other components are impacted?  How much time is needed to fix the issue? Developer  Is there a significant architectural impact on the system?  How many components are impacted?  Is there a need to refactor the architecture?  Is there a significant architectural impact on the system?  How many components are impacted?  Is there a need to refactor the architecture? Architect  How much regression testing is required?  Which use cases are impacted?  Which components are impacted?  Which tests need to be re-executed?  What is the impact on existing plan?  How many resources need to be engaged?  Is it possible to stay on original schedule?  How much regression testing is required?  Which use cases are impacted?  Which components are impacted?  Which tests need to be re-executed?  What is the impact on existing plan?  How many resources need to be engaged?  Is it possible to stay on original schedule? Tester  What is the business case to fix the defect?  How many customers might be impacted?  What is the $$ loss associated with the defect?  Which features/use cases might be de-scoped due to time spent in fixing the issue?  What is the potential revenue impact of these features?  What is the business case to fix the defect?  How many customers might be impacted?  What is the $$ loss associated with the defect?  Which features/use cases might be de-scoped due to time spent in fixing the issue?  What is the potential revenue impact of these features? Analyst ElaborationTriageDeploymentTransitionConstruction Packaged Application RDBMS Web Service Jservice JSPs, struts, Servlets Enterprise A CICS Maintenance of Existing Systems and Software App Servers EJBs App Servers Web Server EJBs Edge Server Enterprise B Traceability Architecture Test Case FEATreq Code Execution Trace Test Case Defect Planning Driver

13. 13  What is the business case to fix the defect?  How many customers might be impacted?  What is the $$ loss associated with the defect?  Which features/use cases might be de-scoped due to time spent in fixing the issue?  What is the potential revenue impact of these features?  What is the business case to fix the defect?  How many customers might be impacted?  What is the $$ loss associated with the defect?  Which features/use cases might be de-scoped due to time spent in fixing the issue?  What is the potential revenue impact of these features? Analyst  What is the impact of these changes to existing components?  Which components are impacted by the changes?  Is there any regression on these components?  What is the impact of these changes to existing components?  Which components are impacted by the changes?  Is there any regression on these components? Developer  What needs to be tested now?  Which fixes have been delivered?  Do they fix the primary use case?  What are the additional use cases to verify?  What is the stability of this build?  How many regressions were introduced?  Does the defect trend match projections?  What needs to be tested now?  Which fixes have been delivered?  Do they fix the primary use case?  What are the additional use cases to verify?  What is the stability of this build?  How many regressions were introduced?  Does the defect trend match projections? Tester ConstructionAnalysisTriageDeploymentTransition Packaged Application RDBMS Web Service Jservice JSPs, struts, Servlets Enterprise A CICS Maintenance of Existing Systems and Software App Servers EJBs App Servers Web Server EJBs Edge Server Enterprise B Traceability Architecture Test Case FEATreq Code Execution Trace Test Case Defect Planning

14. 14  What needs to be tested now?  Which fixes have been delivered?  Do they fix the primary use case?  What are the additional use cases to verify?  What is the stability of this build?  How many regressions were introduced?  Does the defect trend match projections?  What needs to be tested now?  Which fixes have been delivered?  Do they fix the primary use case?  What are the additional use cases to verify?  What is the stability of this build?  How many regressions were introduced?  Does the defect trend match projections? Tester  Is the fix delivery vehicle working?  Does the install work?  Does the fix solve the issue?  Is the fix delivery vehicle working?  Does the install work?  Does the fix solve the issue? Tester  Does this solve the issue? Operations  Is the customer satisfied with the fix? Tech Support TransitionFixAnalysisTriageDeployment Packaged Application RDBMS Web Service Jservice JSPs, struts, Servlets Enterprise A CICS Maintenance of Existing Systems and Software App Servers EJBs App Servers Web Server EJBs Edge Server Enterprise B Traceability Architecture Test Case FEATreq Code Execution Trace Test Case Defect Planning Driver

15. 15 Don’t be intimidated

16. 16 The IBM Software Development Platform AnalystArchitectDeveloperTester Visually construct, transform, integrate and generate code Design, create, and execute tests Model, simulate, assemble, and monitor business processes Model applications and data Industry Standards Training & Mentoring Microsoft IBM and 3 rd Party Servers DB2 Lotus Rational Tivoli WebSphere Pervasive and Embedded Partners  Follow a common process  Track project status  Manage requirements  Manage change and assets  Manage quality Project Manager Deployer Provision, configure, tune and troubleshoot applications Develop Iteratively Focus on Architecture Manage Change and Assets Continuously Ensure Quality

17. 17 Building a quality ecosystem for flexibility and growth Role-customized software quality tools and processes Global deployment awareness Seamless asset unification Open, extensible, technology/platform- independent infrastructure Execution Trace UCreq FEATreq Architecture Code UC Realizations Test Case Test Plan H H H H HYADES Analyst Tester Architect Developer IT Operations Verify and validate from the beginning Maintain traceable, actionable cross-lifecycle linkages Incorporate post- deployment analysis

18. 18 Application Lifecycle Management Closed-loop problem determination, repair, and redeployment  Reduce business downtime while accelerating business value throughput… 1.Quickly discover and understand application- level errors even after deployment 2.Speed Tivoli-aware application fix and (re)build 3.Optimize and accelerate (re)deployment  …by bridging development and operations teams Sample Technologies  Tivoli Monitoring  Tivoli Configuration Management  Tivoli Provisioning/Orchestration  Rational Test and Runtime Analysis  Rational Version and Build Control  Rational Modeling Monitor-ready Application Development Problem Determination Operations Team Development Team 1 1 2 2 3 3 Rapid Deployment

19. 19 Quality is FREE - A well-defined process with unanimous buy-in from team members - Focused attention to value-add, detail and quality across the lifecycle - Constant evaluation of end-user feedback to ensure reinvigoration of the product line and satisfied, repeat customers - Poorly defined process with inconsistent buy-in - Front-loaded requirements gathering without reassessment or validation - Absent or limited architectural and code quality assurance - Late cycle system testing - Limited post-deployment monitoring and assessment

20. 20 Quality enables differentiation Through 2006, 80 percent of development organizations will not follow strict software engineering practices and will fail in 70 percent of their application delivery efforts

21. 21 Quality enables innovation

22. 22  An IBM & Gartner “rule of thumb” for the relative costs to fix defects:  Requirements1.5  Design:1.0  Construct: 1.5  Prior to test:10.0  Testing:60.0  Production:100.0 It’s your choice Cost of a Defect (when identified) 0 20 40 60 80 100 120 Requirements Design Construct Prior to test Testing Production Development Phase Relative Cost Cost of a Defect Note: Does not include business implications of lost revenue and long term impact to business image You can wait…..

23. 23 ….or you can act Take personal responsibility Work together Establish a right-sized, iterative process Automate whenever, wherever Never assume and don’t settle

24. 24