Testing – A Methodology of Science and Art

Agenda To show, A global Test Process which work Like a solution Black Box for an Software Implementation Life cycle Bottom line: “Create a Box of Logically Related Best Practices which will Test any Software Product and define the Degree of Quality”

Known Health of Global Software Software is the only product where large number of defects seems acceptable US products – 5.9 to 7 defects per 1000 lines of code By software assessments, benchmarks and Best Practices by Capers Jones Software defects rate have increased 15% in compared to By Meta Group 2002 Over 15,000,000,000 lines of code are written per year and at 5 errors/1000 lines of code ( in production) will produce nearly 75 million bugs released/year

Let’s check the Generic Life Cycle of Software Implementation

Software Implementation Lifecycle

High Level Test Life Cycle

Quality Requirements

Overall Activities Involved Initiat ion Start up Meeting Client presentation Estimation Resource and capacity planning Resource Selection Start up Meeting Client presentation Estimation Resource and capacity planning Resource Selection Study KT Test Closu re Test Closu re Exec ution Desig n Plann ing Understand the Product Application Plan for knowledge transfer sessions. Deployment of key resource at client side and/or audio visual conference calls Transferring knowledge ownership Create Stake holders Understand the Product Application Plan for knowledge transfer sessions. Deployment of key resource at client side and/or audio visual conference calls Transferring knowledge ownership Create Stake holders Define logic complexity & Test Strategy Plan Testing design & development Activities Define mechanism to track changes Define bug tracking mechanism Define reporting mechanism. Evaluation and selection of test tool if applicable Define logic complexity & Test Strategy Plan Testing design & development Activities Define mechanism to track changes Define bug tracking mechanism Define reporting mechanism. Evaluation and selection of test tool if applicable Create Test scripts manual and automated Test data preparation Create STD, CED, DT Review and confirm test cases Test Bed preparation and verification for readiness Create Test scripts manual and automated Test data preparation Create STD, CED, DT Review and confirm test cases Test Bed preparation and verification for readiness Test case execution Test Execution result log Defect log and status and reporting Defect verification and closure Code coverage matrix Test summary report. Test case execution Test Execution result log Defect log and status and reporting Defect verification and closure Code coverage matrix Test summary report. Client acceptance and sign off Project related matrices Project closure Maintenance and support Post Test Analysis Defect Prevention Decision Client acceptance and sign off Project related matrices Project closure Maintenance and support Post Test Analysis Defect Prevention Decision Study and Understand business rules, process and product Design Identification of core and non core functionality Gap analysis and report preparation Study and Understand business rules, process and product Design Identification of core and non core functionality Gap analysis and report preparation

What is Important?

Global Strategy Rigorous Testing Process & Best Practices which will make a product/project highly deterministic Timely: Integrate throughout with the lifecycle Manageable: Measurable and Predictable Efficient: Optimum number of test cases & heavily automated Effective: Ensure optimum validation, Right test definition

Define Quality Attributes – Earliest Defining Quality Qualifiers is a must for Software Determination of it’s distributed functionalities and it’s behavior, accessed by 100s or 1000s of users, Customers are from different background, Users use variable environments for application usage, Application should consider the best possible transaction process time Information Security Quality Factors, which can define the product/project profile Functionality Usability Reliability Performance Supportability

Determine Quality Attributes The specification of the FURPS+ characteristics needs to go into the Systems Requirements. Each Attributes should have quantities Measure Some of the FURPS+ characteristics, i.e. Functionality and Usability can be tested by executing the actual software. Some, however, like Supportability and Adaptability can only be verified by code inspection or dry running What if? scenarios. It is important to note that neither the SQA nor SQC group should have the responsibility of putting the desired FURPS+ characteristic into the product. They (SQC) should only test the presence or absence of the FURPS+ characteristics. With an established practice of defining and measuring the FURPS+ (or similar model) characteristics it is possible to implement Software QA\QC along similar lines to manufacturing QA\QC.

Understand Risk Create Criteria Understanding the Risk and Challenges involved in each phase of the testing is the most important analysis QCON creates a set of Best practices to identify and analyze the incomplete areas of Development/Enhancement/Defect fix on the product before next phase of Testing Unit-Integration  Functional  System  User Acceptance

What all Testing we should do?

Requirement Ambiguity Identification Complexity analysis Unit and Integration Testing Automating Test Harness Black box Testing Black Box Automation Non-Functional Testing Static Testing Test Management Post Test Data Analysis Orthogonal Testing

How They Are Related ?

Requirement Ambiguity Identification Complexity analysis Unit and Integration Testing Automating Test Harness Black box Testing Black Box Automation Non-Functional Testing Static Testing Test Management Post Test Data Analysis Orthogonal Testing

We Understand, Let’s Relate them Logically

Requirement Ambiguity Identification Complexity analysis Unit and Integration Testing Automating Test Harness Black box Testing Black Box Automation Non-Functional Testing Static Testing Test Management Post Test Data Analysis Orthogonal Testing

What All methodology we’ll Apply?

Methodologies mixed up with Science and Art 1.Understanding Complexities of System 2.Requirement Ambiguity Analysis 3.Test Strategy/Approach Creation before Testing 1.Direct/Regression/Progression Analysis 4.Understanding Test coverage 5.Logic Based Testing for 100% coverage 6.Orthogonal Array based testing for identification of obvious defects with 90% (approx.) effort and Test case reduction 7.Periodic Test Metric collection and Analysis 8.Usage of Automation Frameworks for Optimum Automation 9.Identify why Performance and create model for Performance Analysis 10.Effective Performance Testing and Analysis for Tuning 11.Process improvements through Root Cause analysis 12.Effective Test Management

And, QCON can build end to end of Each of the Methodology

Thank You