Product Quality?

What is product quality? Quality, simplistically, means that a product should meet its specification The software product should deliver the required functionality (functional requirements) with the required quality attributes (non–functional requirements)

Problematic for software systems Some quality requirements are difficult to specify in an unambiguous way Software specifications are usually incomplete and often inconsistent Quality attributes are frequently conflicting and increase development costs, so there is a need for weighting and balancing. Different users may have different opinions about what constitutes software quality.

Quality attributes Correctness : the degree to which the software product performs the required functions accurately. Reliability is the degree to which the software behaves well and in the way the users expect. Robustness implies that the software is unlike to fail or break. performance relates to the response time of the software system. Usability relates to the user friendness of the software.

Quality attributes Maintainbility :Correcting software errors and deficiencies Scalibility : Ease with which the software can be scaled up or evolved in response to the growth in demand for its functionality. Reusability : Defines the level to which the software components can be used for construction of other products. Portability : Can software run on various hardware/software platforms without any modifications or after undergoing minor customization or parameterization ?

Quality control Quality control is mostly about testing the quality of a product ( to eliminate problems ) Quality assurance is about building quality into the product.

Testing – V model

“V” Model Each phase has corresponding test or validation counterpart Requirements Analysis Acceptance Test Integration Test System Design Program Design Unit Test Implementation

Sawtooth Model (Brugge) Requirements Analysis Demo Prototype 1 Demo Prototype 2 Acceptance Test Integration Test System Design Program Design Unit Test Implementation Quality is guaranteed at each project stage.

Unit testing The most ‘micro’ scale of Testing A unit = smallest testable software component Objects and methods Procedures / functions Performed by Programmer A tester can help. Requires detailed knowledge of the internal program design and code. The units are tested in isolation. Ensures the component is working according to the detailed design/build specifications of the module. Also known as component, module, or program testing.

Sometime called structural testing Unit Test ( white box) White-box testing Sometime called structural testing Tester studies the code and decide on data inputs to exercise all program statements (not all path combinations). Test coverage measures ensure that all statements have been executed at least once Derivation of test cases according to program structure. Knowledge of the program is used to identify additional test cases. Also suitable for design models and specification documents (walkthrough and inspections)

Integration Testing Testing of more than one (tested) unit together to determine if they function correctly. Focus on interfaces Communication between units It is done using the integration test design prepared during the architecture design phase. Helps assembling incrementally a whole system, ensuring the correct ‘flow’ of data from the first through the final component. Done by developers/designers and testers in collaboration Also called Interface Testing or Assembly Testing.

System testing Testing the system as a whole - Black-box type testing that is based on overall requirements specifications; covers all combined parts of a system. Ensures that system meets all functional and business requirements. Focus Verifying that specifications are met Validating that the system can be used for the intended purpose The system test design is derived from the system design documents and is used in this phase. It can involve a number of specialized types of tests to check performance, stress, documentation etc. Sometimes testing is automated using testing tools. Done by Independent testing group

System testing Black-box testing (testing to specifications) An approach to testing where the program is considered as a ‘black-box’ , taking some inputs and produces some outputs. Tester does not know or choose to ignore the internal workings of the program. The program test cases are based on the system specification Test planning can begin early in the software process Testing is done by feeding the test unit with data inputs and verifying that the expected output is produced. Also applicable to constraint testing ( performance and security ) and missing functionalities.

Acceptance testing To determine whether a system satisfies its acceptance criteria and business requirements or not. Similar to System testing in that the whole system is checked, but the important difference is the change in focus. Done by real business users. It enables the customer to determine whether to accept the system or not. Also called as Beta Testing, Application Testing or End User Testing. Approach Should be performed in real operating environment . Customer should be able to perform any test based on their business processes. Final Customer sign-off.

Testing techniques It is never possible to test for all possible data inputs or code execution paths. Testing techniques can be classified according to 5 criteria : Visibility Automation partitioning Coverage scripting

Testing techniques Visibility Automation

Partitioning -Equivalence partitioning Groups data inputs (and implicitly , data outputs)into partitions constituing homogenoues test targets ( testing with one member implies test with other member in the same partition ) Supported by Black box testing

Boundary value Additional data analysis technique Bounadary values are extreme cases withing equivalence partitions. Ex: Partition of integer from 1 to 100. Bounadry value analysis recommends tests to be done on the values on the edges that is : -1 , 0 , +1 as well as for 99,100, 101

Coverage Path coverage Determine how much code is going to be exercised by a white box test Operation coverage Ensure that each operation in the code is exercised at least once by the white box test Path coverage Numbering possible execution paths ( infinite in large program ) in the program Exercising them one by one. For large program , choose the most critical and frequently used ones. Testing can be manual or automatic

Manual testing Human tester interracts with the application under test conducts , according to a predefined test script and observe the results. Test script defines step-by-step testing actions and expected outcomes. Use cases are used to write test scripts. Problems: Freuqently output is not presented to the creen Live data are not predefined Expensive

Automated testing Use software testing tools to execute large volumes of test without human participation Tools can produce post test reports Automated testing can be divided into : Regression testing Exercising testing

Regression testing Repetitive execution of the same test scripts on the same data to be sure that the system has not been broken by successive changes to the code (changes not related to the tested functionality) . Execution of the script at scheduled test times.

Exercising testing Tool generates automatically and randomly various possible actions instead of the user. Mad user hitting any possible key on keyboard , selecting any possible menu item , ….etc.

Test planning Part of the quality management plan. Defines testing schedule, budget , tasks (test cases) and resources. Test Plan include code and other project artifacts testings. Specify which test cases should be conducted Human and material ressources should be allocated. Test database created and test software tools installed

Testing Approaches We will look at a small sample of approaches for testing White-box testing Control-flow-based testing Loop testing Data-flow-based testing Black-box testing Equivalence partitioning

Control-flow-based Testing A traditional form of white-box testing Step 1: From the source, create a graph describing the flow of control Called the control flow graph The graph is created (extracted from the source code) manually or automatically Step 2: Design test cases to cover certain elements of this graph Nodes, edges, paths

Example of a Control Flow Graph (CFG) 1 s:=0; d:=0; if (x+y < 100) s:=s+x+y; else d:=d+x-y; } 2 while (x<y) { x:=x+3; y:=y+2; 3 4 5 6 7 8

Statement Coverage Basic idea: given the control flow graph define a “coverage target” and write test cases to achieve it Traditional target: statement coverage Need to write test cases that cover all nodes in the control flow graph Intuition: code that has never been executed during testing may contain errors Often this is the “low-probability” code

Example Suppose that we write and execute two test cases Test case #1: follows path 1-2-exit (e.g., we never take the loop) Test case #2: 1-2-3-4-5-7-8-2-3-4-5-7-8-2-exit (loop twice, and both times take the true branch) Do we have 100% statement coverage? 1 2 3 T F 4 5 6 7 8

Branch Coverage Target: write test cases that cover all branches of predicate nodes True and false branches of each IF The two branches corresponding to the condition of a loop All alternatives in a SWITCH statement In modern languages, branch coverage implies statement coverage

Branch Coverage Statement coverage does not imply branch coverage Can you think of an example? Motivation for branch coverage: experience shows that many errors occur in “decision making” (i.e., branching) Plus, it subsumes statement coverage.

Example Same example as before Test case #1: follows path 1-2-exit Test case #2: 1-2-3-4-5-7-8-2-3-4-5-7-8-2-exit What is the branch coverage? 1 2 3 T F 4 5 6 7 8

Black-box Testing Unlike white-box testing, here we don’t use any knowledge about the internals of the code Test cases are designed based on specifications Example: search for a value in an array Postcondition: return value is the index of some occurrence of the value, or -1 if the value does not occur in the array We design test cases based on this spec

Test cases Design Project Name: Test Case Template   Test Case ID: Fun_10 Test Designed by: <Name> Test Priority (Low/Medium/High): Med Test Designed date: <Date> Module Name: Google login screen Test Executed by: <Name> Test Title: Verify login with valid username and password Test Execution date: <Date> Description: Test the Google login page Pre-conditions: User has valid username and password Dependencies: Post-conditions: Step Test Steps Test Data Expected Result Actual Result Status (Pass/Fail) Notes 1 Navigate to login page User= example@gmail.com   User should be able to login User is navigated to Pass 2 Provide valid username Password: 1234 dashboard with successful 3 Provide valid password login 4 Click on Login button