1

Chair, ISTQB Expert Level Test Automation Working Group ASTQB Webinar Series ISTQB and Test Automation How ISTQB Certification Prepares you for Automated Testing Presented by ANDREW L POLLNER Vice President, ASTQB Chair, ISTQB Expert Level Test Automation Working Group 2

Introduction ISTQB and ASTQB Importance of Automation in Testing Automation Topics in ISTQB Syllabi Business Outcomes Expert Level Syllabi

WORLD-WIDE COVERAGE ASTQB Countries covered by Member Boards Countries covered by Exam Providers

Goals for Automation in Testing Fast execution of tests Repeatable process Increased functional coverage Lots of variations and permutations Automatic reporting of errors Low maintenance requirements Make test team more productive Stay on time and on budget

Body of Knowledge (BOK) What Documents Cover Automation? Foundation Level Advanced Level Test Analyst Advanced Level Technical Test Analyst Advanced Level Test Manager Expert Level Glossary

Foundation Level Chapter 6 – Tool Support for Testing: Types of Test Tools Effective use of Tools Introducing a Tool into an Organization

Advanced Level Test Analyst Chapter 7 – Test Tools: Test Data Preparation Tools Automated Test Execution Tools Test Automation Implementation Improving the Success of the Automation Effort Keyword-Driven Automation Causes for Failures of the Automation Effort

Advanced Level Technical Test Analyst Chapter 6 – Test Tools and Automation: Integration and Information Interchange Between Tools Defining the Test Automation Project Selecting the Automation Approach Modeling Business Processes for Automation Specific Test Tools Tools for Web-Based Testing

Advanced Level Test Manager Chapter 6 – Test Tools and Automation: Tool Selection Selection Process Return on Investment (ROI) Tool Lifecycle Tool Metrics

Expert Level Test Automation Two Modules: Test Automation Engineering Test Automation Management

Business Outcomes – TAE Contribute to the development of a plan to integrate automated testing within the testing process Evaluate tools and technology for automation best fit to each project and organization Create an approach and methodology for building a test automation architecture Design and develop new or modified test automation solutions that meet the business needs Create automated test reporting and metrics collection Manage and optimize testing assets to facilitate maintainability and address evolving test systems

Business Outcomes – TAM Develop a test automation policy and strategy appropriate for the organization Create a business case which outlines the costs and benefits to be expected from introducing and sustaining test automation Develop a plan to integrate automated testing within the testing process Oversee the evaluation of tools and technology for automation best fit to each project and organization Manage the test automation resources to implement and sustain an effective test automation program Define and gather metrics to effectively report progress of the test automation program Apply findings to improve the test automation process

Test Automation Engineering (TAE) In Scope Out of Scope Tasks performed by a test automation engineer Designing, developing and maintaining concrete test automation solutions Concepts, methods, tools, and processes Automating dynamic functional tests Test Management Automation of non-functional tests Teaching of software engineering methods and programming Teaching of software technologies Selection of software testing products and services

Topics in TAE Introduction and Objectives for Test Automation Preparing for Test Automation The Generic Test Automation Architecture Deployment Risks and Contingencies Test Automation Reporting Metrics Transitioning Manual Testing to an Automated Environment Verifying the TAS Continuous Improvement

SDLC processes for SUT and TAS

K LEVELS: Syllabus & Exam Questions are classified according to the cognitive level, the K- level (also known as level of knowledge): K1 – Remember (recognize a term or concept) K2 – Understand (able to explain a concept) K3 – Apply (select correct application of concept or technique) K4 – Analyze (can distinguish between facts and inferences for deeper understanding) K5 – Evaluate (can make judgments based on criteria and standards) K6 – Create (can put elements together to form a coherent or functional whole) The number of questions for each topic is related to the length of the topic in the syllabus

Learning Objectives (Chap 1-2) 1.1 Purpose of Test Automation ELTA-E-1.1.1 (K2) Explain the objectives, advantages and limitations of test automation 2.1 Test Automation Requirements for the SUT and its Context ELTA-E-2.1.1 (K4) Analyze a system under test to determine the optimal automation solution

Learning Objectives (Chap 3-4) 3.2 TAA Design ELTA-E-3.2.1 (K4) Design the appropriate TAA for a given project 3.3 TAS Development ELTA-E-3.3.1 (K6) Construct a purpose-built TAA based on the generic TAA (gTAA) using predefined components 4.3 Technical Risk Assessment and Mitigation Strategies ELTA-E-4.3.1 (K6) Devise mitigation strategies to address technical risks, including framework approach, that could lead to failure of the test automation project

Generic Test Automation Architecture

Learning Objectives (Chap 5-6) 5.3 Logging of the TAS and the SUT ELTA-E-5.3.1 (K4) Implement logging of test executions and test results (of both TAA and SUT data) 6.1 Criteria for Automation ELTA-E-6.1.1 (K4) Analyze factors relevant to determining the appropriate criteria for suitability of tests for automation

Learning Objectives (Chap 7-8) 7.1 Verifying Automated Test Environment Components ELTA-E-7.1.1 (K3) Apply validity checks to the automated test environment and test tool setup 8.2 Planning the Implementation of Test Automation Improvement ELTA-E-8.2.1 (K4) Analyze the test environment components, including tools, in order to understand where consolidation and updates need to be made following a given set of test environment changes

Test Automation Management Tentative Topics: Test Automation Policy Test Automation Strategy Managing Test Automation Test Automation Resources

ISTQB LEVELS AND SYLLABI

Syllabus in Progress Syllabus Released Exam in Progress Certification Prerequisite

ISTQB CERTIFICATES ISSUED

Conclusion Test Automation is more similar to software development than to testing ISTQB syllabi provide a wealth of information on Test Automation topics and a forthcoming certification exam Certification criteria: 7 Years of practical testing experience Foundation Level Certification Advanced Level TTA Certification (for TAE)

Questions?