1. Quality Assurance IS 101Y/CMSC 101Y November 12, 2013 Carolyn Seaman University of Maryland Baltimore County

2. Data Analysis Assignment Stacked bar charts

3. Data Analysis Assignment Aggregate and focus

4. Quality Assurance Refers to any activity whose goal is to make sure that the system or application being built is free from error and of high quality That it works That it solves a problem That it’s usable

5. A process view of QA Problem Identification Analysis Design Implementation QA Maintenance SDLC: Systems Development Lifecycle Installation CMSC, CMPE, IS

6. QA includes: Testing Exercises a system or part of a system to make sure it produces the correct output or behaves in expected ways Cannot happen until some part of the system is implemented Reviews and inspections Visual “reading” of some artifact Can be done early by reviewing early documents, e.g. design

7. Testing Terms Coverage ideally, testing will exercise the system in all possible ways not possible, so we use different criteria to judge how well our testing strategy “covers” the system Test case consists of data, procedure, and expected result represents just one situation under which the system (or some part of it) might run

8. Test Planning A test plan includes: test objectives schedule and logistics test strategies test cases procedure data expected result procedures for handling problems

9. Testing Phases Unit testing - does this piece work by itself? Integration testing - do these two pieces work together? System testing - do all the pieces work together? Alpha acceptance testing - try it out with in- house “users” Installation testing - can users install it and does it work in their environment? Beta acceptance testing - try it out with real users In development/ maintenance organization In user organization

10. Testing Techniques Structural testing techniques “white box” testing based on statements in the code or individual hardware and connections coverage criteria related to physical parts of the system tests how a program/system does something Functional testing techniques “black box” testing based on input and output coverage criteria based on behavior aspects tests the behavior of a system or program

11. System Testing Techniques Goal is to evaluate the system as a whole, not its parts Techniques can be structural or functional Techniques can be used in any stage that tests the system as a whole (acceptance, installation, etc.) Techniques not mutually exclusive

12. System Testing Techniques (cont.) stress testing - test larger-than-normal capacity in terms of transactions, data, users, speed, etc. execution testing - test performance in terms of speed, precision, etc. recovery testing - test how the system recovers from a disaster, how it handles corrupted data, etc. operations testing - test how the system fits in with existing operations and procedures in the user organization compliance testing - test adherence to standards security testing - test security requirements

13. System Testing Techniques (cont.) requirements testing - fundamental form of testing - makes sure the system does what it’s required to do regression testing - make sure unchanged functionality remains unchanged error-handling testing - test required error-handling functions (usually user error) manual-support testing - test that the system can be used properly - includes user documentation historical test data - tests until the number of defects found approaches the average number of defects in the products produced under similar circumstances

14. Unit Testing Goal is to evaluate some piece (file, program, module, component, etc.) in isolation Techniques can be structural or functional In practice, it’s usually ad-hoc and looks a lot like debugging More structured approaches exist

15. Unit Testing Techniques input domain testing - pick test cases representative of the range of allowable input, including high, low, and average values equivalence partitioning - partition the range of allowable input so that the program is expected to behave similarly for all inputs in a given partition, then pick a test case from each partition boundary value - choose test cases with input values at the boundary (both inside and outside) of the allowable range

16. Unit Testing Techniques (cont.) statement testing - ensure the set of test cases exercises every statement at least once branch testing - each branch of an if/then statement is exercised path testing - every path is exercised (impossible in practice) fault seeding - put a certain number of known faults into the code, then test until they are all found

17. Reviews and Inspection: Goal and Motivation By detecting defects early, and preventing their leakage downstream, the higher cost of later detection and rework is eliminated.

18. Basic Steps Using a reading technique, Perform a visual examination of the document Detect and correct: Defects Violation of design standards Other problems

19. Examples of artifacts that can be reviewed Include: Contracts Installation plans Progress reports Design descriptions Release notes Requirements specifications Source code

20. What Is a Defect? Any occurrence in a work product that is determined to be incomplete, incorrect, or missing Any instance which a requirement is not satisfied (Fagan, 1986) Informal synonyms: bug, fault, issue, problem, anomaly

21. Common Attributes of Systematic Reviews and Inspections Systematic reviews and inspections have these attributes in common: Team participation Documented results of the review Documented procedures for conducting the review

22. Management Reviews Performed by those directly responsible for the system Monitor progress Determine status of plans and schedules Confirm requirements and their system allocation Or, evaluate management approaches used to achieve fitness or purpose

23. Technical Reviews Confirms that a product Conforms to specifications Adheres to regulations, standards, guidelines, plans Changes are properly implemented Changes affect only those system areas identified by the change specification

24. Inspections (Formal Peer Reviews) Confirms that the software product satisfies Specifications Specified quality attributes Regulations, standards, guidelines, plans Identifies deviations from standards and specification results in logging a defect

25. Walk-throughs Evaluate a product or artifact or document Sometimes used for educating an audience Major objectives: Find anomalies Improve the product Consider alternative implementations Evaluate performance to standards and specs

26. Audits The purpose of an audit is to provide an independent evaluation of conformance of products and processes to applicable Regulations Standards Guidelines Plans Procedures

27. Inspection Process Most popular is the Fagan method Inspection is separated into 5/6 phases (Planning) Overview Preparation Inspection Meeting Rework Follow-up

28. Review and Inspection Pitfalls Insufficient Preparation Moderator Domination Incorrect Review Rate Ego-involvement and Personality Conflict Issue Resolution and Meeting Digression Recording Difficulties and Clerical Overhead

29. Other QA Activities Process conformance Making sure that quality procedures are followed Risk analysis Planning for adverse events Making the unexpected expected Hiring and training Developing guidelines and standards Identifying new training needs

30. Project demos This Thursday!!! No slides Everyone must participate and answer questions Show your program Running Code Talk about what it does and what it doesn’t do YET 10 minutes, including questions Turn in.pde file and document BEFORE class on Blackboard

31. Peer Evaluations Take a look at the Peer Evaluation form and think about the following two options: 1. Each student gets the actual sheets filled out by their teammates 2. Each student gets a summary of the feedback written by the instructor Evaluations due in class on THURSDAY (rewards to those who bring them to class)