1. DONE-01: Applying Quality Assurance and Testing to Application Development Ademola Taiwo (AT) Empowerment Consultant

2. 2© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Agenda n Introduction n Quality Defined n Why Quality Assurance n Implementing QA Infrastructure n Ensuring Quality n Summary n Questions

3. 3© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Achieving Quality A Quality product meets or exceeds customer expectations Goal: Implement Quality

4. 4© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. What is Quality? n Quality is –Based on customers’ perceptions of a product’s design and how well the design matches the original specifications. –The ability of a product and service to satisfy stated or implied needs. –Achieved by conforming to established requirements within an organization. http://www.asq.org/portal/page Quality is All about the Customer

5. 5© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. What is Quality Control n Control = An evaluation to indicate needed corrective responses; the act of guiding or the state of a process in which the variability is attributable to a constant system of chance causes. n Quality control = The operational techniques and activities used to fulfill requirements for quality. Quality Control is NOT Quality Assurance

6. 6© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. What is Quality Assurance? n Assurance = The act of giving confidence, the state of being certain, or the act of making certain. n Quality assurance = All the planned and systematic activities implemented within the quality system that can be demonstrated to provide confidence a product or service will fulfill requirements for quality Quality Assurance is NOT Quality Control

7. 7© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Cost of Poor quality Costs associated with defects found before the customer receives the product or service Internal Failure Cost

8. 8© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Cost of Poor quality Costs associated with defects found after the customer receives the product or service External Failure Cost

9. 9© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Cost of Poor quality Costs incurred to determine the degree of conformance to quality requirements Appraisal Cost

10. 10© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Cost of Poor quality Costs incurred to keep failure and appraisal costs to a minimum Prevention Cost

11. 11© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Cost of Poor Quality n Quality Variables –Defects –Time To Market –Features –Internal Cost / Affordability Quality Variables Drive Priorities

12. 12© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Why is QA Necessary? RequirementsCodeRelease TIME “It takes 5 to 17hrs to fix bugs in released code, as compared to 1 to 1.5 hrs during code reviews and inspections” US National Institute for Standards and Technology, May 2002, Economic Impact of Inadequate Infrastructure of Software Testing. The Boehm Curve

13. 13© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Cost of Poor Quality n The National Institute of Standards and Technology study shows that Software defects cost the U.S. economy some $59.5 billion a year. (2002) n http://www.nist.gov/public_affairs/releases/n02-10.htm http://www.nist.gov/public_affairs/releases/n02-10.htm To the US Economy

14. 14© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Cost of Poor Quality “On average developers make 100 – 150 errors every thousand lines. These statistics are different developer to developer and project to project and language to language.” If 10% of the errors produced are serious in a small application of 20,000 lines, then we will have 200 serious errors!! Watts. S. Humphrey. Multiyear Study by Software Engineering Institute (SEI) at Carnegie Mellon University Internal Cost to the Development Organization

15. 15© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Cost of Poor Quality n Blue Cross Insurance, US – Sent $60 million in unwarranted and duplicate payments n Pepsi Promotion, Mexico – Instead of issuing 1 x $40,000 winning bottle, 800,000 were produced n Trader in London sold shares 100 times larger than he intended. Bank had to pay out millions to retrieve these shares. Validation requirements were not catered for External Failure Cost (To You)

16. 16© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Benefit of Good Quality n NIST's estimated that about $22 billion of the sunk costs could be recovered through better software testing n http://www.nist.gov/public_affairs/releases/n02-10.htm http://www.nist.gov/public_affairs/releases/n02-10.htm To Everyone (the Economy)

17. 17© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. What is Testing Testing is the act of gathering metrics in pursuit of Quality. It generates supporting data for assuring quality. Testing is just another source of Quality metrics Testing is NOT Quality Assurance

18. 18© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Why Test? n “ Software defects cost the U.S. economy almost $60 billion annually. 80% of the total development costs are consumed by developers identifying and correcting defects” National Institute of Standards and Technology (NIST). www.nist.gov/public_affairs/releases/n02-10.htm www.nist.gov/public_affairs/releases/n02-10.htm n “Cancelled projects due to quality issues in the US, cost $55 billion dollars” The Standish Group. CHAOS Chronicles v3.0 March 2003 A relatively inexpensive way to gather metrics

19. 19© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Agenda n Introduction n Quality Defined n Why Quality Assurance n Implementing QA Infrastructure n Ensuring Quality n Summary n Questions

20. 20© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Implementing QA Infrastructure n Verification n Is it error free? n Does it do what was specified? n Validation n Is it valid? n Is this what you really, really want? n Risk Mitigation n Expect the Unexpected http://www.its.bldrdoc.gov/fs-1037/dir-029/_4291.htm NOT Test Implementation

21. 21© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Benefits of Quality Assurance n Reduced Time to Market n Lowered Risk n Reduced Rework n Increased Job Satisfaction n Serving as a Differentiator n Efficient Framework for Outsourcing “IT outsourcing will increase by 130 percent in the next 5 years to $23 billion in 2008” Information Week 22 nd October 2003 Better Customer Experience at Reduced Cost

22. 22© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Quality is Everyone’s Job! Quality Performance Interoperability IntegrityEfficiency Maintainability Reliability Adaptability Usability Quality - or Lack of - is Ubiquitous

23. 23© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Implementing Quality Assurance Ensuring Quality

24. 24© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Integrate QA into your Software Development Life Cycle (SDLC)  QA Process should begin with requirement gathering and continue through to the application release.  Quality improvements require discipline. People, processes and technology are only successful with proper follow through.  QA processes must be supported by technology to help troubleshoot and analyse problems.  Testing should be a function of lowering risk (risk-based testing). QA begins at the Requirements Gathering Stage

25. 25© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Measuring Quality n Understand what metrics you want to measure –Define and communicate n Clear defect severity rating n Quality encompasses more than tests and defects! Think about: –Customer satisfaction n Features n Platforms n Defects n Costs – internal and external n Time to Market n Process n Feature / Functionality Definition n Spec reviews n Code reviews n Test scenario reviews n Project plan reviews n Measure and allow for improvements throughout the process

26. 26© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Keys to Achieving High Quality n People n Process n Technology Quality Product must Meet or Exceed Customer Expectation

27. 27© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Key to Achieving High Quality n Development & QA Managers: Peers n General Engineering Managers (GEM) –Monitors Critical Quality Goals –Cross Dept Communication –Maintains Product Release Calendar –Monitors & Improves Development Process People: Quality / Development Organization

28. 28© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Key to Achieving High Quality n Quality ownership: engineers and designers must think about how to test and validate –Design time requirements n Product feature design n Test scenario design n Quality needs to be a ‘lifestyle’ not a checkbox item! –Design quality into the product NOT –Test to see if it is a quality product Process: Guidelines

29. 29© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. QA in Development Methodologies n QA Approach is determined by the Software Development Life Cycle (SDLC) n There are 3 main SDLC Models: Requirements Specification Functional Specification Technical Specification Program Specification Code Acceptance Test Business Objectives 1)Waterfall Process: Guidelines

30. 30© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Development Methodologies Cont…. Check 2) V Model 3) RAD Process: Guidelines

31. 31© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Current View Point – Application Transformation Applying Quality Assurance and Testing to App. Dev.

32. 32© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Key to Achieving High Quality n Testing –Functional Testing –Boundary testing –Negative testing –Platform Testing –Multi-user Testing –Stress / Performance Testing Technology: Automated Data Gathering

33. 33© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Test Plans n Define test scenarios in the design specs n Define test cases in the test plans and automation n Intent needs to be clear n Review all test cases n Include multiple test types n Define measurable milestones –Feature Freeze, Code Complete, Code Freeze n Don’t forget performance testing –5X slower is a bug ! Technology: Automated Data Gathering

34. 34© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Data Gathering n Three phase of testing –Phase 1 – Planning through Feature Freeze –Phase 2 – Feature Freeze through Code Freeze –Phase 3 – Code Freeze to customer delivery Technology: Test, Test and Test again

35. 35© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Summary n Quality is meeting customer expectation –Good quality is exceeding customer expectation n QA enhances IT effectiveness –Increase Customer Satisfaction –involves People, Processes and Technology n Immediate ROI –Lower cost / Increased revenue n Found Money –Start NOW Quality is Everyone’s Job

36. 36© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Questions?

37. 37© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Thank you for your time!

38. 38© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. You Can Begin NOW!

39. 39© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev.

40. 40© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Agenda n Introduction n Quality Defined n Why Quality Assurance n Implementing QA Infrastructure n Ensuring Quality n Summary n Questions

41. 41© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Summary n Quality is meeting and exceeding customer expectations n QA involves People, Processes and Technology n Immediate ROI Quality Assurance

42. 42© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. The Quality Health Check n Quality Health Check Focuses on Four Key Area’s: –Software Testing –Software Configuration Management (SCM) –Incident Management –Process Management across QA

43. 43© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Quality Matters n Bullets: Arial Bold, size 28, sentence case –Second level: Arial Bold, size 26 n Third level: Arial Bold, size 24 n Maximum 7 bullets per slide n Maximum 7 words per bullet n Type in charts and graphics –Arial Bold, size 20 or above, initial caps n Source code in examples –ABSOLUTELY NO SMALLER THAN SIZE 20 –See sample slides Applying Quality Assurance and Testing to App. Dev.

44. 44© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Proposed Break Down Software Testing (IEEE 7925-2) Review and Highlight Improvements in Processes, Procedures and Strategies:  Incorporate Reviews – Walkthroughs, Informal Reviews, Technical Reviews and Inspections  Conduct a Risk Assessment's  Look at Functional (Black Box), Structural (White Box) and Non-Functional Testing in line with SDLC  Incorporate a Test Strategy  Incorporate a Test Plan (IEEE 829-1998)

45. 45© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Proposed Break Down Software Configuration Management (SCM)  Auditing  Release Management  Build Management  Process Control  Investigate the Workflow

46. 46© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Proposed Break Down Software Testing (IEEE 7925-2) Review and Highlight Improvements in Processes, Procedures and Strategies:  Incorporate Reviews – Walkthroughs, Informal Reviews, Technical Reviews and Inspections  Conduct a Risk Assessment's  Look at Functional (Black Box), Structural (White Box) and Non-Functional Testing in line with SDLC  Incorporate a Test Strategy  Incorporate a Test Plan (IEEE 829-1998)

47. 47© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Implementing Test Plans n Test at API Level as well as User Level n Automate versus manual test n Allow for multiple phases of development & testing –Phase 1 – development of features, unit testing & execution of automated regression tests. –Phase 2 – regression testing of existing functionality, performance, platform, and stress testing –Phase 3 – Final regression testing

48. 48© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Regression Testing (example) n Criteria to Ship –Automation Passing at 95% or greater ( understand the failures and evaluate the risk versus time to market) –Rerun of failed tests n Regression testing validates –No defects introduced –New features function n Weekend Automated Regression Tests –Execution on all platforms –2 days to complete testing n Nightly Automated Regression Tests –Execution on all platforms –1 night to complete testing

49. 49© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Proposed Break Down n Software Configuration Management (SCM) –Auditing –Release Management –Build Management –Process Control –Investigate the Workflow

50. 50© 2005 Progress Software Corporation DONE-01 - Applying QA and Testing to App. Dev. Measuring Quality n Understand what metrics you want to measure –Define and communicate n Clear defect severity rating n Quality encompasses more than tests and defects! Think about: –Customer satisfaction n Features n Platforms n Defects n Costs – internal and external n Time to Market n Process n Feature / Functionality Definition n Spec reviews n Code reviews n Test scenario reviews n Project plan reviews n Measure and allow for improvements throughout the process