Planning Test Automation (Wisely). Introductions Thomas Messerschmidt Test Automation Specialist Paramount Pictures Avner Uzan Test Automation Analyst.

Slides:



Advertisements
Similar presentations
Delta Confidential 1 5/29 – 6/6, 2001 SAP R/3 V4.6c PP Module Order Change Management(OCM)
Advertisements

You have been given a mission and a code. Use the code to complete the mission and you will save the world from obliteration…
2017/3/25 Test Case Upgrade from “Test Case-Training Material v1.4.ppt” of Testing basics Authors: NganVK Version: 1.4 Last Update: Dec-2005.
© 2008 Oracle Corporation – Proprietary and Confidential.
Advanced Piloting Cruise Plot.
The 4 T’s of Test Automation:
Requirements Engineering Process
Chapter 24 Quality Management.
Chapter 1 The Study of Body Function Image PowerPoint
By Rick Clements Software Testing 101 By Rick Clements
Business Transaction Management Software for Application Coordination 1 Business Processes and Coordination.
Copyright CompSci Resources LLC Web-Based XBRL Products from CompSci Resources LLC Virginia, USA. Presentation by: Colm Ó hÁonghusa.
Click to edit Master title style Page - 1 OneSky Teams Step-by-Step Online Corporate Communication Support 2006.
1 Validation & Measurement Methods for the PHARE Demonstrations R A Whitaker Validation Project Leader.
Jeopardy Q 1 Q 6 Q 11 Q 16 Q 21 Q 2 Q 7 Q 12 Q 17 Q 22 Q 3 Q 8 Q 13
Jeopardy Q 1 Q 6 Q 11 Q 16 Q 21 Q 2 Q 7 Q 12 Q 17 Q 22 Q 3 Q 8 Q 13
Title Subtitle.
My Alphabet Book abcdefghijklm nopqrstuvwxyz.
Leading for High Performance. PKR, Inc., for Cedar Rapids 10/04 2 Everythings Up-to-Date in Cedar Rapids! Working at classroom, building, and district.
FACTORING ax2 + bx + c Think “unfoil” Work down, Show all steps.
Addition Facts
Year 6 mental test 10 second questions
Making the System Operational
1 A Tool-box for Web-site Maintenance Manjula Patel UKOLN University of Bath Bath, BA2 7AY UKOLN is funded by the Library and Information Commission, the.
For Translators and Translation Editors Note-Taking presents... by Riccardo Schiaffino CTA 3rd Annual Conference Boulder, May © Riccardo Schiaffino,
Introduction Lesson 1 Microsoft Office 2010 and the Internet
Proud Members of the Consulting Group, LLC
1 SESSION 5- RECORDING AND REPORTING IN GRADES R-12 Computer Applications Technology Information Technology.
Spoofing State Estimation
Configuration management
1 A Test Automation Tool For Java Applets Testing of Web Applications TATJA Program Demonstration Conclusions By Matthew Xuereb.
ABC Technology Project
Microsoft Office Illustrated Fundamentals Unit K: Working with Data.
Session 2: Introduction to the Quality Criteria. Session Overview Your facilitator, ___________________. [Add details of facilitators background, including.
1 Contract Inactivation & Replacement Fly-in Action ( Continue to Page Down/Click on each page…) Electronic Document Access (EDA)
1 Undirected Breadth First Search F A BCG DE H 2 F A BCG DE H Queue: A get Undiscovered Fringe Finished Active 0 distance from A visit(A)
Microsoft Office Illustrated Fundamentals Unit C: Getting Started with Unit C: Getting Started with Microsoft Office 2010 Microsoft Office 2010.
VOORBLAD.
1 Breadth First Search s s Undiscovered Discovered Finished Queue: s Top of queue 2 1 Shortest path from s.
Factor P 16 8(8-5ab) 4(d² + 4) 3rs(2r – s) 15cd(1 + 2cd) 8(4a² + 3b²)
© 2012 National Heart Foundation of Australia. Slide 2.
Lets play bingo!!. Calculate: MEAN Calculate: MEDIAN
Chapter 10 Software Testing
Understanding Generalist Practice, 5e, Kirst-Ashman/Hull
Chapter 5 Test Review Sections 5-1 through 5-4.
GG Consulting, LLC I-SUITE. Source: TEA SHARS Frequently asked questions 2.
1 Overview of Testing Methodology Obtain generic Test Scripts Populate Self Testing Work Paper Template Prepare actual test scripts Determine sample Combine.
Addition 1’s to 20.
Model and Relationships 6 M 1 M M M M M M M M M M M M M M M M
25 seconds left…...
Equal or Not. Equal or Not
Slippery Slope
RTI Implementer Webinar Series: Establishing a Screening Process
Januar MDMDFSSMDMDFSSS
Week 1.
Level 1:Shows the total opportunities worldwide. Size of pie is proportional to size-of-prize. Shown in black color. Revenue breakup: How to read the slides.
We will resume in: 25 Minutes.
©Brooks/Cole, 2001 Chapter 12 Derived Types-- Enumerated, Structure and Union.
12 January 2009SDS batch generation, distribution and web interface 1 ExESS IT tool for SDS batch generation, distribution and web interface ExESS IT tool.
A SMALL TRUTH TO MAKE LIFE 100%
PSSA Preparation.
Immunobiology: The Immune System in Health & Disease Sixth Edition
ACD Training.
Immunobiology: The Immune System in Health & Disease Sixth Edition
CpSc 3220 Designing a Database
© Paradigm Publishing, Inc Access 2010 Level 2 Unit 2Advanced Reports, Access Tools, and Customizing Access Chapter 8Integrating Access Data.
Presentation transcript:

Planning Test Automation (Wisely)

Introductions Thomas Messerschmidt Test Automation Specialist Paramount Pictures Avner Uzan Test Automation Analyst 2Planning Test Automation

Disclaimer The views expressed in this presentation are those of the presenters & do not necessarily reflect the views of their current or past employers; their subsidiaries or affiliated companies! 3Planning Test Automation

4 Planning Automation - Premise Set the hook Basic Rules Methodology: Keyword-Driven Spreadsheet-Test Cases Framework Construction Scope: Smoke, Functional, System, Performance Prioritizing Test Construction Best Kept Secrets (Trackers & Analysis Masters) Test Automation Best Practices Wisely Planning Test Automation 4 (Does Test automation Find Bugs?)

5 Premise: Create a testing framework that will use easy-to-read spreadsheets to enumerate the actual test cases Create & maintain generic & application specific functions & subroutines to support the automation Planning Test Automation 5 Create maintainable automated test suites

Does Test Automation Find Bugs?

7 Basic Rules Plan your automation process thoroughly Establish & follow naming conventions Establish & persist a directory structure * Build automation for platform independence Solicit developments cooperation to develop for testability Expand your knowledge with other SMEs MEASURE OR DIE Planning Test Automation 7

8 Methodology Smoke Testing – Keyword-driven Spreadsheets Functional Testing – Keyword-driven Spreadsheets System Testing – Functional Decomposition Performance Testing – Weighted Functionality Planning Test Automation 8

Zambelich Method 9Planning Test Automation 9

10 Spreadsheet-Test Cases Planning Test Automation 10

11 Generic Functions Planning Test Automation 11

12 App Specific Functions Planning Test Automation 12

13 Sample Keyword Action Planning Test Automation 13

14 Parameter & Data File Planning Test Automation 14

15 Smoke Test Objectives Primarily validate builds Open and close as many GUIs as possible Some windows are acct data dependant Manually test GUIs that could not be automated Smoke Test Execution Metrics Covers 115/120 L1 GUIs - 105/110 L2 GUIs Execution time: 1:25 / L1, 1:10 L2 This was run 200+ times on the WR version This has runs 200+ times on the QTP version Currently demonstrates 100% GUI Pass rate Planning Test Automation 15

16 GUI Functional GUI Functional Objectives Validates the functionality of Each GUI Verify screen functionality as per spec, NOT business processes Validates objects exist on UI per spec Verifies object operations, calculations & navigations Verifies data formats ($-Amts, SSN, Phone #, etc.) Verifies field/object data comparisons across screens Verifies all error processing (invalid values, pop-ups, etc.) GUI Functional Scope Targeted two languages Planning Test Automation 16

17 System Test System Test Objectives Focus on core functionality – align w/ project team &/or customer Regression test regular release & special projects Future plans include converting our system test to BPT & auto. Scope Execution Metrics 200 scripts of which about 175 are active - take about 4-5 days 50 full suite runs, 10,000 tests run over 5 year period Use Functional Decomposition Planning Test Automation 17

18 Performance Test Planning Test Automation

19 Prioritize Test Construction Target low hanging fruit to get early ROI Prioritize & build common functions first Create tests for most frequented GUIs Planning Test Automation 19

20 Optimizing Automation Optimize automation for maximum coverage Increase number of executing machines The downside Brake automation into byte-size pieces If one function can do the work of two Planning Test Automation 20

21 Other Considerations Number of test platform / environments Availability of test platform (hours/day) Comparability of test environment to prod. Environment refresh Test data availability / harvesting (method of) Test data accuracy Product or application stability Planning Test Automation 21

22 Trackers (Measure or Die) Planning Test Automation 22

23 Analysis Masters Demo Analysis Masters Planning Test Automation 23

24 Performance Analysis Master Planning Test Automation 24

Automation Best Practices Setup directory structure & use it consistently Define & establish filing conventions Publish changes to subroutines & functions. Keep test assets under version control Choose automation tools carefully based on compatibility to your development platform, resource knowledge, budget. Include resource, time & process allowances for test asset maintenance. Keep an inventory list of functions & routines easily accessible. If possible cross-reference use of functions & subroutines in scripts to more predicatively understand the impact changes to these assets might have. Prioritize tests to be automated carefully to avoid marginalizing ROI 25Planning Test Automation

Get management buy-in & commitment early Set realistic expectations – automation is not just the push of a button Identify a set of relevant metrics & measure regularly & consistently Include regular progress reviews to ensure that the automation team is in fact focused on the business or customer needs Share the wealth – the more people use it the richer youll be! Automation Best Practices 2 Additional References: TOTALLY DATA-DRIVEN AUTOMATED TESTING – Keith Zambelich– Keys to Functional Test Automation Success, by Thomas Murphy / The Gartner Group - Thanks to Microfocus an HP Partner for the link LoadRunner Test Analysis Master – A. Uzan 26 Planning Test Automation

27 Automation Demo Demo Automation (time permitting) Planning Test Automation 27

Contact us: Avner Uzan: Thomas Messerschmidt: 28Planning Test Automation

Parting Thought On Automation Why Automate? Automation, fundamentally, is the mechanization of a manual task. Love, education & human interaction should not be automated. However, there is no evil in taking a manual process & allowing a machine to do that which would take a person longer, work a person harder or take the pleasure out of doing a task over & over. However, if by automating something, the existence of life, the values of morality, the sensibilities of ones existence or the livelihood of those who would otherwise perform the task manually is usurped, than automation has defeated its very purpose. Automation should be carefully conceived so that future generations are not burdened with undoing the damage of unwitting or well intended but misguided endeavors. 29Planning Test Automation