Presentation is loading. Please wait.

Presentation is loading. Please wait.

Test Automation Success: Choosing the Right People & Process Kiran Pyneni, Automation Manager Aetna, Inc.

Similar presentations

Presentation on theme: "Test Automation Success: Choosing the Right People & Process Kiran Pyneni, Automation Manager Aetna, Inc."— Presentation transcript:

1 Test Automation Success: Choosing the Right People & Process Kiran Pyneni, Automation Manager Aetna, Inc.

2 Agenda Automation Lifecycle Automation Team Metrics Of Our Success Questions Automation Coverage, Efforts Automated, Utilization & Cost Avoidance Company Background Strategy Team Delivery Team Benefits of Teams Strategy Benefits of Process Integration Our Journey… Automation Objectives

3 Aetna Background Aetna is one of the nation's leading diversified health care benefits companies, serving members with information and resources to help them make better informed decisions about their health care. Products and management Programs: –Aetna offers a broad range of insurance and employee benefits products. –The first national, full-service health insurer to offer a consumer-directed health plan, Aetna continues to lead the way with its Aetna HealthFund2 line of products, including HSA, HRA and RRA options. –Aetna offers a wide array of programs and services that help control rising employee benefits costs while striving to improve the quality of health care, such as case management; disease and patient safety programs; integrated medical, dental, pharmaceutical, behavioral health and disability information. –Aetna provides members with access to convenient tools and easy-to-understand information that can help them make better-informed decisions about their health and financial wellbeing. 3

4 Automation Team Improves efficiencies and quality at our organization by automating consistent and repeatable manual tasks using industry standard technologies / tools / frameworks. Automation Team Assessments & Designs Reports & Metrics Standards & Best Practices Automation Development & Maintenance

5 Automation Objectives Automation Objectives Automating Right Tasks Sustaining and Growing Automation Suite Utilization of existing automation

6 Our Journey… Benefits –Easy identification of automation opportunities –Higher collaboration between application & automation teams Challenges –Automation development has become expensive –Usage of different tools & solutions –Standards & Best practices are not shared across applications Benefits –Standardization of automation tool set at enterprise level –Sharing of coding best practices, templates and standards –Cost of automation development is lowered Challenges –Slower automation coverage progression –Lower automated scripts utilization –Funding Challenges Decentralized ApproachCentralized Approach

7 Our Journey Cont’d. – Challenges & Solutions Challenges Identification of automation opportunities Prioritization of automation projects based on ROI Cost of automation Implementation of optimized designs Solution Structuring automation as strategy & delivery team Challenges Early completion of maintenance for automation scripts Funding for automation projects Collaboration between automation & business teams Solution Create automation life cycle process and integrate with company's overall software development life cycle

8 Automation Team Structure Automation Team Automation Strategy Team Automation Delivery Team

9 Automation Strategy Team Definition: –To provide automation strategic direction and identify potential new automation opportunities Characteristics: –Project management experience –Various automation frameworks/tools knowledge –Ability to manage multiple projects in parallel –Very strong written and verbal communication skills –Very strong leadership and analytical skills Activities: –Process & Best Practices –Assessment & Strategies –Design Solutions & Techniques –Estimations & Planning –ROI Cost/Benefit Analysis –Reports & Metrics –Tool Technical Support & Analysis

10 Automation Delivery Team Definition: –To develop, deliver, and maintain automation solutions Characteristics: –Understanding of programming principles and best practices –Knowledge of various automation tools –Knowledge of multiple programming languages –Strong communication skills Activities: –Scripts Development –Scripts Maintenance –Coding Standards –Scripts Planning & Execution –Day to Day interactions w/End User

11 Benefits of Teams Strategy Benefits Easy identification of automation opportunities Prioritization of automation projects based on ROI Improved automation design Optimized the cost of automation Clearly defined roles & responsibilities Better communication process Optimized automation project planning

12 Preliminary Estimations Automation Test Strategy Business Case - ROI Selected automation tool Detailed Estimations Go, No Go Automation Project Plan Framework documentation – Re-usable components, Portability, test data grid design, etc. Business Process Test (BPT) components Automation Scripts Test Data Tested automation scripts Automated test execution results Lessons Learned Sign Off G1 G2G3 Automation Evaluation Understand functionality of the project / application Understand testing processes Understand target environments Understand test data details Verify automation tool compatibility Identify Automation opportunities Phases Activities Deliverables Automation Preparation Identify the resources needed. Understand the requirements with the help application demos. Create automation framework / design Identify timelines and milestones Training required for automation tools Tool installation if required Access & Environment Setup Automation Generation Create Business Process Test (BPT) components Parameterize test data in BPT components Review components for quality and correctness String the BPT components together to create a test script Develop automation scripts for BPT components Identify test data to conduct unit & acceptance testing Unit testing of automation Deliver automation to application team Execute automation by application team Validation of ROI Retrospection Sign off Execution & Acceptance Automation Lifecycle

13 Team Collaboration Automation Evaluation Automation Preparation Automation Generation Execution & Acceptance Automation Strategy Team Automation Delivery Team

14 Automation ROI Calculator


16 Integration of Automation Lifecycle & Software Development Lifecycle Automation Evaluation Feasibility/Analysis Automation Preparation Design Automation Generation Implementation Execution & Acceptance Testing & Maintenance

17 Benefits of Process Integration Benefits Improved understanding on automation process across the entire organization Better collaboration between business and automation teams Funding challenges are removed Improved on-time delivery of automation scripts Helped to promote the automation across entire organization Formal escalation process developed Initialized the creation of automation reports & metrics

18 Automation Coverage – Regression Testing 42% Q1-2009 45% Q2-2009 47% Q3-2009 52% Q4-2009 35% Year-2008

19 Manual Testing Efforts Automated 7,047 hrs 8,570 hrs 10,200 hrs 11,750 hrs 13,803 hrs YearAs of 2008As of 20092009 only efforts Total Efforts Automated7047 hrs13803 hrs6756 hrs Year-2008Q1-2009Q2-2009Q3-2009Q4-2009

20 Utilization & Cost Avoidance Q1-2009Q2-2009Q3-2009Q4-2009 Cost Avoidance 7,200 hrs 10,810 hrs 9,618 hrs 9,867 hrs Q1-2009Q2-2009Q3-2009Q4-2009 Utilization 45% Year-2008 87% 150% 134% 130%

21 Savings Vs. Cost Avoidance Cost Savings –Efforts difference between manual and automation testing – Manual (Not automation) execution frequency –Produced by automation strategy team after assessment –Automation ROI Calculator Cost Avoidance –Test coverage difference between manual & automation testing –Efforts difference between manual and automation testing –Automation execution frequency –Generated from Quality Center after release implementation Helped to improve the credibility of automation metrics significantly

22 Automation Metrics – Best Practices Automation is a way of Testing and process followed for manual testing can be applied Best Practices followed in Test Management Tool – Quality Center –Define automation work flow Regression Testing identifier Automation Assessment status Automation Development status –Store automation inventory –Manage automated test scripts execution

23 Questions

Download ppt "Test Automation Success: Choosing the Right People & Process Kiran Pyneni, Automation Manager Aetna, Inc."

Similar presentations

Ads by Google