High Volume Automated Testing for Software Components Harry RobinsonDoug Szabo Microsoft.

Slides:

Advertisements

Similar presentations

Presentation by Prabhjot Singh

Advertisements

Catching Bugs in Software Rajeev Alur Systems Design Research Lab University of Pennsylvania

A System to Generate Test Data and Symbolically Execute Programs Lori A. Clarke September 1976.

SOFTWARE TESTING. INTRODUCTION  Software Testing is the process of executing a program or system with the intent of finding errors.  It involves any.

Chapter 17 Software Testing Techniques

1 Today Another approach to “coverage” Cover “everything” – within a well-defined, feasible limit Bounded Exhaustive Testing.

1 These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 5/e and are provided with permission by.

1 Advanced Material The following slides contain advanced material and are optional.

Testing an individual module

Software Engineering Lecture 12 Software Testing Techniques 1.

1 Software Testing Techniques CIS 375 Bruce R. Maxim UM-Dearborn.

Chapter 13 & 14 Software Testing Strategies and Techniques

Zichao Qi, Fan Long, Sara Achour, and Martin Rinard MIT CSAIL

What Exactly are the Techniques of Software Verification and Validation A Storehouse of Vast Knowledge on Software Testing.

CS527: (Advanced) Topics in Software Engineering Overview of Software Quality Assurance Tao Xie ©D. Marinov, T. Xie.

Dr. Pedro Mejia Alvarez Software Testing Slide 1 Software Testing: Building Test Cases.

Expert System Presentation On…. Software Certification for Industry - Verification and Validation Issues in Expert Systems By Anca I. Vermesan Presented.

Chapter 2 What is software quality ?. Outline What is software? Software errors, faults and failures Classification of the causes of software errors Software.

Web Technologies COMP6115 Session 2: Planning, Designing, Constructing and Testing Static Web Sites Dr. Paul Walcott Department of Computer Science, Mathematics.

1 These courseware materials are to be used in conjunction with Software Engineering: A Practitioner’s Approach, 5/e and are provided with permission by.

Software Testing. Definition To test a program is to try to make it fail.

1 Software Testing (Part-II) Lecture Software Testing Software Testing is the process of finding the bugs in a software. It helps in Verifying and.

© Andrew IrelandSoftware Design F28SD2 Software Design (F28SD2): Life-Cycle Perspective - Part 2 Andrew Ireland School of Mathematical & Computer Sciences.

Verification and Validation Overview References: Shach, Object Oriented and Classical Software Engineering Pressman, Software Engineering: a Practitioner’s.

Lecture 11 Testing and Debugging SFDV Principles of Information Systems.

1 Software testing. 2 Testing Objectives Testing is a process of executing a program with the intent of finding an error. A good test case is in that.

Software Testing The process of operating a system or component under specified conditions, observing and recording the results, and making an evaluation.

Agenda Introduction Overview of White-box testing Basis path testing

Survey on Trace Analyzer (2) Hong, Shin /34Survey on Trace Analyzer (2) KAIST.

Testing Testing Techniques to Design Tests. Testing:Example Problem: Find a mode and its frequency given an ordered list (array) of with one or more integer.

1 Software Engineering: A Practitioner’s Approach, 6/e Chapter 14a: Software Testing Techniques Software Engineering: A Practitioner’s Approach, 6/e Chapter.

WARNING These slides are not optimized for printing or exam preparation. These are for lecture delivery only. These slides are made for PowerPoint 2010.

Testing Techniques Software Testing Module ( ) Dr. Samer Hanna.

Software Testing Definition Software Testing Module ( ) Dr. Samer Odeh Hanna.

Lopamudra Kundu Reg. No. : of Roll No.:- 91/RPE/ Koushik Basak

Testing, Testing & Testing - By M.D.ACHARYA QA doesn't make software but makes it better.

Chapter 8 Testing. Principles of Object-Oriented Testing Å Object-oriented systems are built out of two or more interrelated objects Å Determining the.

Software Engineering Saeed Akhtar The University of Lahore.

LOGO TESTING Team 8: 1.Nguyễn Hoàng Khánh 2.Dương Quốc Việt 3.Trang Thế Vinh.

Software Testing and Quality Assurance 1. What is the objectives of Software Testing?

Software Testing. System/Software Testing Error detection and removal determine level of reliability well-planned procedure - Test Cases done by independent.

Software Quality Assurance and Testing Fazal Rehman Shamil.

 Software Testing Software Testing  Characteristics of Testable Software Characteristics of Testable Software  A Testing Life Cycle A Testing Life.

Dynamic Testing.

These slides are designed to accompany Software Engineering: A Practitioner’s Approach, 7/e (McGraw-Hill 2009). Slides copyright 2009 by Roger Pressman.1.

SOFTWARE TESTING SOFTWARE TESTING Presented By, C.Jackulin Sugirtha-10mx15 R.Jeyaramar-10mx17K.Kanagalakshmi-10mx20J.A.Linda-10mx25P.B.Vahedha-10mx53.

Increase Test Coverage in Linux-based OS Guillermo Ponce: AWK maniac Victor Rodriguez : Linux fan.

Cs498dm Software Testing Darko Marinov January 24, 2012.

TESTING BASED ON ERROR GUESSING Rasa Zavistanavičiūtė, IFME-0/2.

WARNING These slides are not optimized for printing or exam preparation. These are for lecture delivery only. These slides are made for PowerPoint 2010.

Chapter 17 Software Testing Techniques

Introduction to Software Quality Assurance & Testing

SOFTWARE TESTING Date: 29-Dec-2016 By: Ram Karthick.

Software Engineering (CSI 321)

Software Testing.

Software Testing.

TQS - Teste e Qualidade de Software (Software Testing and Quality) Introduction To Software Testing Concepts João Pascoal.

Verification and Validation Overview

Chapter 13 & 14 Software Testing Strategies and Techniques

الأستاذ المساعد بقسم المناهج وطرق التدريس

Software Testing (Lecture 11-a)

Software Verification and Validation

Software Verification and Validation

Software Testing “If you can’t test it, you can’t design it”

Software Testing.

Software Verification and Validation

Introduction to Software Quality Assurance & Testing

RANDOM NUMBERS SET # 1:

Chapter 13 & 14 Software Testing Strategies and Techniques 1 Software Engineering: A Practitioner’s Approach, 6th edition by Roger S. Pressman.

Introduction to Software Testing

Presentation transcript:

High Volume Automated Testing for Software Components Harry RobinsonDoug Szabo Microsoft

High Volume Test Automation The essence of HVTA techniques is automated execution and evaluation of large numbers of tests, for the purpose of exposing functional errors that are otherwise hard to find. Experiments with High Volume Test Automation, 2004 McGee & Kaner

Subtle Bugs Only weak bugs have a logic to them and are amenable to exposure by strictly logical means. Subtle bugs have no definable pattern - they are wild cards. Software Testing Techniques 2 nd Edition, 1990 Beizer

Bounded Exhaustive Testing Bounded exhaustive testing (BET) is a verification technique in which software is automatically tested for all valid inputs up to specified size bounds. Software assurance by bounded exhaustive testing, 2005 Coppit et al

Typical Biases Developers – “happy paths” Testers – “a nose for bugs”

CodeProject: Sorting Algorithms in C#

The Sorting Algorithms

Popular …

Recommended …

… and it looks cool!

… but dark clouds appear on the horizon

...

Reasonable Unit Tests

… Even Some Randomized Unit Tests

Pre-Oracled Data shuffle sort compare

The Loop

Shearsort demo #1

Shearsort demo #2

Shearsort bug

Shearsort bug - fixed

Heapsort bug

Heapsort bug - fixed

OddEvenTransport bug

OddEvenTransport bug - fixed

Distribution of Buggy Array Sizes

Bug is not detected at array size 1000

Number of Random Arrays to Detect a Bug

Generated Unit Tests