Presentation on theme: "Audience Line of Business, including Product, Project and Quality Managers (concerned about reliability and transparency) CEOs, (concerned about quality."— Presentation transcript:
Audience Line of Business, including Product, Project and Quality Managers (concerned about reliability and transparency) CEOs, (concerned about quality and agility) CIOs (concerned about efficiency) Objective An introduction of the HP Test Factory Services offering - ensuring that applications deliver targeted business results while lowering overall testing costs and risks. This presentation shows how we apply our proven HP Quality Methodology and market-leading HP Software, that enable applications to meet clearly defined expectations. Scenario 60 minute customer presentation to provide an initial overview of the HP Test Factory services. Also to engage the customer in the next step – an Envisioning Workshop or a Test Factory Outline Version info Draft version 1, July 28th, 2008 Do not present this slide Control sheet
Agenda Why care about quality? Professional testing in a factory? How do I to get there? Why HP? Who has done it?
Why care about quality?
What is the impact of a project delay? −Project impact A four week delay for a 75 person project = $1.3M in cost −75 people x 900/person/day x 20 days = $1.3M −Business impact In September 2004, HP struggled with SAP consolidation project resulting in a three week delay for their supply-chain software. Results: HP's ordering system became chaotic $400M loss in Q3 revenue Damaged reputation with customers Three executives were dismissed HP's shares dropped 15% on the news If all that can happen is a project delay…
Source: CHAOS Report 2007, Standish Group International, Inc. Success of IT projects
88/25/2014HP CONFIDENTIAL The gap between the specification... And the user accceptance test
98/25/2014HP CONFIDENTIAL The gap between improvements... And the retest
10 November 1, 2006HP Confidential Introduction of defects Detection of defects Source: NIST 2002 RTI Project Majority of defects are introduced during the requirements and design phase! However majority of defects are actually detected during user acceptance testing and in production! Understanding defects
Industry References: 3 B. Boehm and V. Basili, "Software Defect Reduction Top 10 List," IEEE Computer, IEEE Computer Society, Vol. 34, No. 1, January 2001, pp This industry average is used as a baseline to calculate the cost savings Assessing cost of defects
The HP Quality Methodology (STLC) Deployed Application Business Requirement Design Effort and Cost time Traditional Cost Idea Implement Developer Test Production Requirement Verification Functional Test / E2E Test System Test User Acc. Test Operation Test preparation Planned Cost
Defect (ISTQB): A flaw in a component or system that can cause the component or system to fail to perform its required function, e.g. an incorrect statement or data definition. A defect, if encountered during execution, may cause a failure of the component or system. Inserted defect: Defect created (inserted) in a phase Contained defect: Defect found in the phase it originated Introduced defect: Defect not found in the phase it originated Detected defect: Finding of an Introduced defect Defect detection rate & cost
Profesional testing in a factory?
Customer success roadmap defined Level 0 Quality Agnostic Level 1 Quality Initialising Level 2 Quality Conscious Level 3 Quality Savvy Level 4 Quality Expert Project-based technology, processes, & personnel The majority of testing, if not all, is still manual No documented QA processes or recognition of the need for them Testing, if done at all, is manual Centralised technology, licenses, admin and support QA process in place, but mainly in individual projects Centralized technology Centralized service bureau for QA personnel Centralized methodology and best practices for QA procedures Centralized technology & personnel QA processes govern testing and quality initiatives QA is thought leader and has enterprise influence Product Utility Service Utility Centre of Excellence Increasing Maturity – m 1 of 2 Most organizations Introducing the HP Quality Model
Design Implement Unit Test IT Supplier Production Business User Business Analyst Design Requirements Test Execution Test Environment Testing in the Lifecycle - today
Test definition Test data specification Test automation DesignImplementUnit Test Production Business User Business Analyst Design Requirements Acceptance Criteria IT Supplier Manual Test Test execution Test environment Quality Factory Professional Testing Roll Out
Test Factory Service Catalog (1) 1825 August 2014 Lifecycle PhaseServiceKPI Idea Impact analysis Number of businesses Functions affected Function change rate Analysis Business Impact Analysis Test Effort calculation Test Case Change effort Regression Test effort Design Acceptance criteria / Test rule definition Test environment design Performance test scenario definition Number of issues found Number of issues accepted Functional coverage per business function
Test Factory Services Catalog (2) 1925 August 2014 Lifecycle PhaseServiceKPI Implementation Functional Test – Test environment setup – Driver and Stub development – Test data defintion – Test script development / maintenance – Test execution Non Functional Test – Test environment setup – Driver and Stub development – Test data defintion – Test script development / maintenance – Test execution Number of defects found Number of defects accepted Functional coverage per business function Number of concurrent users during performance test System response times Number of concurrent users during Load test
Test Factory Services Catalog (3) 2025 August 2014 Lifecycle PhaseServiceKPI Deployment Go Live test AD hoc regression Training environment provisioning Functional coverage during ad hoc regression Maintenance Impact Analysis Test Case maintance Regression Test Change Rate per function Number of regression test cases Function coverage during regression test Average system response time during performance / load test
How to get there?
Building a Test Factory Envisioning Workshop Present HP as solution Partner Agreement on milestones for engagement Solution Blueprint & Business Case Development Pilot installation Time 1-2 days days days >100 days Build blueprint for first test factory line Set up first factory line and run one cycle Application Quality Roll-out Service
Sample Timeline Q308Q408Q109Q209Q309Q409 Phase 1: Preparation Phase 2: Setup the first line Phase 3: Operate the first line Phase 4: Evaluate the results and adjust Phase 5: Setup the next lines
Providing real time status information Dashboards as Program “Cockpit” Application Management Dashboard - EriDoc Demand / Project Status Performance Test Status Functional Test Status
Controlling the Factory Consolidated top-down planning & bottom-up execution Project proposals Operational work Budgeting/costing Resource capacity analysis Project scheduling HP PPM Center ™ Visibility ControlsFlexibility Governance structuresBusiness rules WorkflowNotificationsEscalations Audit trail Issues, risks, scope changes Testing issues (defects) Change management Service desk or 3 rd party integrations Time tracking
Improve performance Reduce backlog Ensure quality 278/25/2014HP CONFIDENTIAL Simplify Standardize Optimize Strategic quality methodology Market- leading HP Software Global delivery capabilities Quality HP Quality Services tri-dimensional approach aligns business with IT
Simply with HP Quality Methodology 288/25/2014HP CONFIDENTIAL Simplify Standardize Optimize Strategic quality methodology Market- leading HP Software Global delivery capabilities Quality Supports the application lifecycle, requirement to deployment Integrates with every development methodology Supports reuse of existing artifacts Incorporates the business Bridges the communication gap between IT and business
Blueprint - Based on real world experience
Standardize with HP Software Tools - Market leading product (60% +) Requirements management, risk- based testing, and cycle management First real-time link between business requirements and quality in a single enterprise platform. Delivers risk-based release decisions and end-to-end traceability. Introduces dimension of time and release cycles for agile quality Web 2.0 Performance validation in a world of AJAX and FLEX Accelerated testing SAP “TAO” HP Quality Management Ecosystem™ 308/25/2014HP CONFIDENTIAL Simplify Standardize Optimize Strategic quality methodology Market- leading HP Software Global delivery capabilities Quality
318/25/2014HP CONFIDENTIAL Optimize with HP Global Delivery 400+ plus testing professionals across GDAS Centers Skilled professionals: Test Managers, Test Architects, Test consultants, Test Leads, Manual Test Engineers, Automated Test Engineers and Performance Test engineers, etc… 150+ certified professionals - CSTE, CSTM, CSTP ISTQB, Brain bench. etc… Strong people retention & development practices Simplify Standardize Optimize Strategic quality methodology Market- leading HP Software Global delivery capabilities Quality
Learning and Development Program −Certified Tester Foundation Level −Certified Tester Advanced Level (Test Management) −Certified Professional for Requirements Engineering Certification Program Qualification
Who has done it?
Total Cost of Defect Detection Project Example I – Automative Industrie (Experience) 34 Testing Addendum Average Defect Number in comparable Projects (Experience) Standard Software (SAP/Siebel) /high Complexity due to Functionality / Interfaces HP approachClassic approach Cost $/ Defect Defect % Defect absolute Cost Defect % Defect absolute Cost Requirements $ 139,0065%2500 $ , $ ,00 Unit Testing Functional Testing $ 977,0028%1500 $ , $ ,00 Integration Testing 5%200 $ ,00 System Testing UAT $ 7.136,002%50 $ , $ ,00 Production $14.102,0 01%40 $ , $ ,00 Defect Costs (total) $ ,00 $ ,00 Savings (total) $ ,00
KMD Information Technology Services 3525 August 2014 IT improvements Business Outcomes ObjectiveApproachResults To remain one of Denmark’s top three IT service providers To assure customer satisfaction by improving operational quality 2004: Initiated IT lifecycle management plan Created a Quality Center of Excellence built on HP Quality Center software Now accommodates users and 150 test projects Established an Operational Quality Center powered by HP Performance Center software and HP Business Availability Center software HP Universal CMDB integration for 3 rd party help desk solution and all HP management software 96% customer satisfaction with testing services has led to referrals and new business 18% revenue growth: 2006 Reduced time to repair from hours to minutes Reduced risk through − Functional and performance testing and monitoring − 99.5% - 100% availability Rapid root cause analysis saves ~$300k, or 2200 work hours, per year Able to offer new services to its customers