1. Product Quality?

2. What is product quality?
Quality, simplistically, means that a product should meet its specification
The software product should deliver the required functionality (functional requirements) with the required quality attributes (non–functional requirements)

3. Problematic for software systems
Some quality requirements are difficult to specify in an unambiguous way
Software specifications are usually incomplete and often inconsistent
Quality attributes are frequently conflicting and increase development costs, so there is a need for weighting and balancing.
Different users may have different opinions about what constitutes software quality.

4. Quality attributes
Correctness : the degree to which the software product performs the required functions accurately.
Reliability is the degree to which the software behaves well and in the way the users expect.
Robustness implies that the software is unlike to fail or break.
performance relates to the response time of the software system.
Usability relates to the user friendness of the software.

5. Quality attributes
Maintainbility :Correcting software errors and deficiencies
Scalibility : Ease with which the software can be scaled up or evolved in response to the growth in demand for its functionality.
Reusability : Defines the level to which the software components can be used for construction of other products.
Portability : Can software run on various hardware/software platforms without any modifications or after undergoing minor customization or parameterization ?

6. Quality control
Quality control is mostly about testing the quality of a product ( to eliminate problems )
Quality assurance is about building quality into the product.

7. Testing – V model

8. “V” Model Each phase has corresponding test or validation counterpart
Requirements Analysis
Acceptance Test
Integration Test
System Design
Program Design
Unit Test
Implementation

9. Sawtooth Model (Brugge)
Requirements Analysis
Demo Prototype 1
Demo Prototype 2
Acceptance Test
Integration Test
System Design
Program Design
Unit Test
Implementation
Quality is guaranteed at each project stage.

10. Unit testing The most ‘micro’ scale of Testing
A unit = smallest testable software component
Objects and methods
Procedures / functions
Performed by Programmer
A tester can help.
Requires detailed knowledge of the internal program design and code.
The units are tested in isolation.
Ensures the component is working according to the detailed design/build specifications of the module.
Also known as component, module, or program testing.

11. Sometime called structural testing
Unit Test ( white box)
White-box testing
Sometime called structural testing
Tester studies the code and decide on data inputs to exercise all program statements (not all path combinations).
Test coverage measures ensure that all statements have been executed at least once
Derivation of test cases according to program structure. Knowledge of the program is used to identify additional test cases.
Also suitable for design models and specification documents (walkthrough and inspections)

12. Integration Testing
Testing of more than one (tested) unit together to determine if they function correctly.
Focus on interfaces
Communication between units
It is done using the integration test design prepared during the architecture design phase.
Helps assembling incrementally a whole system, ensuring the correct ‘flow’ of data from the first through the final component.
Done by developers/designers and testers in collaboration
Also called Interface Testing or Assembly Testing.

13. System testing
Testing the system as a whole - Black-box type testing that is based on overall requirements specifications; covers all combined parts of a system.
Ensures that system meets all functional and business requirements.
Focus
Verifying that specifications are met
Validating that the system can be used for the intended purpose
The system test design is derived from the system design documents and is used in this phase.
It can involve a number of specialized types of tests to check performance, stress, documentation etc. Sometimes testing is automated using testing tools.
Done by Independent testing group

14. System testing Black-box testing (testing to specifications)
An approach to testing where the program is considered as a ‘black-box’ , taking some inputs and produces some outputs.
Tester does not know or choose to ignore the internal workings of the program.
The program test cases are based on the system specification
Test planning can begin early in the software process
Testing is done by feeding the test unit with data inputs and verifying that the expected output is produced.
Also applicable to constraint testing ( performance and security ) and missing functionalities.

15. Acceptance testing
To determine whether a system satisfies its acceptance criteria and business requirements or not.
Similar to System testing in that the whole system is checked, but the important difference is the change in focus.
Done by real business users.
It enables the customer to determine whether to accept the system or not.
Also called as Beta Testing, Application Testing or End User Testing.
Approach
Should be performed in real operating environment .
Customer should be able to perform any test based on their business processes.
Final Customer sign-off.

16. Testing techniques
It is never possible to test for all possible data inputs or code execution paths.
Testing techniques can be classified according to 5 criteria :
Visibility
Automation
partitioning
Coverage
scripting

17. Testing techniques
Visibility
Automation

18. Partitioning -Equivalence partitioning
Groups data inputs (and implicitly , data outputs)into partitions constituing homogenoues test targets ( testing with one member implies test with other member in the same partition )
Supported by Black box testing

19. Boundary value Additional data analysis technique
Bounadary values are extreme cases withing equivalence partitions.
Ex: Partition of integer from 1 to 100. Bounadry value analysis recommends tests to be done on the values on the edges that is : -1 , 0 , +1 as well as for 99,100, 101

20. Coverage Path coverage
Determine how much code is going to be exercised by a white box test
Operation coverage
Ensure that each operation in the code is exercised at least once by the white box test
Path coverage
Numbering possible execution paths ( infinite in large program ) in the program
Exercising them one by one.
For large program , choose the most critical and frequently used ones.
Testing can be manual or automatic

21. Manual testing
Human tester interracts with the application under test conducts , according to a predefined test script and observe the results.
Test script defines step-by-step testing actions and expected outcomes.
Use cases are used to write test scripts.
Problems:
Freuqently output is not presented to the creen
Live data are not predefined
Expensive

22. Automated testing
Use software testing tools to execute large volumes of test without human participation
Tools can produce post test reports
Automated testing can be divided into :
Regression testing
Exercising testing

23. Regression testing
Repetitive execution of the same test scripts on the same data to be sure that the system has not been broken by successive changes to the code (changes not related to the tested functionality) .
Execution of the script at scheduled test times.

24. Exercising testing
Tool generates automatically and randomly various possible actions instead of the user.
Mad user hitting any possible key on keyboard , selecting any possible menu item , ….etc.

25. Test planning Part of the quality management plan.
Defines testing schedule, budget , tasks (test cases) and resources.
Test Plan include code and other project artifacts testings.
Specify which test cases should be conducted
Human and material ressources should be allocated.
Test database created and test software tools installed

26. Testing Approaches
We will look at a small sample of approaches for testing
White-box testing
Control-flow-based testing
Loop testing
Data-flow-based testing
Black-box testing
Equivalence partitioning

27. Control-flow-based Testing
A traditional form of white-box testing
Step 1: From the source, create a graph describing the flow of control
Called the control flow graph
The graph is created (extracted from the source code) manually or automatically
Step 2: Design test cases to cover certain elements of this graph
Nodes, edges, paths

28. Example of a Control Flow Graph (CFG)1
s:=0; d:=0;
if (x+y < 100) s:=s+x+y; else d:=d+x-y; } 2
while (x<y) {
x:=x+3; y:=y+2; 3 4 5 6 7 8

29. Statement Coverage
Basic idea: given the control flow graph define a “coverage target” and write test cases to achieve it
Traditional target: statement coverage
Need to write test cases that cover all nodes in the control flow graph
Intuition: code that has never been executed during testing may contain errors
Often this is the “low-probability” code

30. Example Suppose that we write and execute two test cases
Test case #1: follows path 1-2-exit (e.g., we never take the loop)
Test case #2: exit (loop twice, and both times take the true branch)
Do we have 100% statement coverage? 1 2 3 T F 4 5 6 7 8

31. Branch Coverage
Target: write test cases that cover all branches of predicate nodes
True and false branches of each IF
The two branches corresponding to the condition of a loop
All alternatives in a SWITCH statement
In modern languages, branch coverage implies statement coverage

32. Branch Coverage Statement coverage does not imply branch coverage
Can you think of an example?
Motivation for branch coverage: experience shows that many errors occur in “decision making” (i.e., branching)
Plus, it subsumes statement coverage.

33. Example Same example as before Test case #1: follows path 1-2-exit
Test case #2: exit
What is the branch coverage? 1 2 3 T F 4 5 6 7 8

34. Black-box Testing
Unlike white-box testing, here we don’t use any knowledge about the internals of the code
Test cases are designed based on specifications
Example: search for a value in an array
Postcondition: return value is the index of some occurrence of the value, or -1 if the value does not occur in the array
We design test cases based on this spec

35. Test cases Design Project Name: Test Case Template
Test Case ID: Fun_10
Test Designed by: <Name>
Test Priority (Low/Medium/High): Med
Test Designed date: <Date>
Module Name: Google login screen
Test Executed by: <Name>
Test Title: Verify login with valid username and password
Test Execution date: <Date>
Description: Test the Google login page
Pre-conditions: User has valid username and password
Dependencies:
Post-conditions:
Step
Test Steps
Test Data
Expected Result
Actual Result
Status (Pass/Fail)
Notes 1
Navigate to login page
User=
User should be able to login
User is navigated to
Pass 2
Provide valid username
Password: 1234
dashboard with successful 3
Provide valid password
login 4
Click on Login button