1. Teaching slides Chapter 9

2. Chapter 9 Software Testing (Verification & Validation) Introduction Software testing & software engineering methodologies Introduction to different types of software testing Introduction to verification & validation Introduction to levels of software testing (validation) Verification Unit testing Integration testing System testing User acceptance testing Other important tests Test case design Test preparation Test lifecycle

3. Chapter 9 Software Testing (Verification & Validation) Software testing is the process of finding defects in a software product so that they can be fixed. Software testing originally involved testing the finished software product to uncover software defects. Later many people realized that even software requirement specifications and software design also contain defects and if these defects can be removed at the point of occurrence then they will not propagate to down the line processes. This will result in producing better quality software products. At the same time, effort required in fixing software defect will come down since fewer software defects will need to be fixed. The entire process of finding and fixing defects in all artifacts (requirement specifications, software design, source code etc.) during software development lifecycle is known as verification and validation.

4. Chapter 9 Software Testing (Verification & Validation) Verification & validation & levels of software testing

5. Chapter 9 Software Testing (Verification & Validation) There are many types of software testing (verification & validation) are done during the entire software development lifecycle. The above figure depicts most of the types of testing done on software projects. Verification tests are also known as static tests as no execution of artifact is required to run it during these tests. Verification tests include requirement specification reviews, design reviews, source code reviews etc. Validation tests are also known as dynamic tests as execution of artifact under test is required to run it during these tests. Validation tests are further divided into white box and black box tests. White box tests are named so as the source code is used during software testing. White box tests are also known as structural tests as they test small parts (structures) of the software product. Unit tests and integration tests fall under these tests. Black box tests are named so as the source code is not used during software testing. Instead the executable binary machine code is used. Black box tests are further divided into functional and non functional tests. Non functional tests are further divided into performance, security, usability etc. tests. Black box tests are done both at system level as well as at deployment (user acceptance) level. Regression tests are done for both white box as well as black box tests.

6. Chapter 9 Software Testing (Verification & Validation) Verification & validation and associated software testing types

7. Chapter 9 Software Testing (Verification & Validation) Verification is that part of software testing where the testing is done on any software project artifact which has no executable part available. Software requirement specifications, software designs, source code (testing to view only the source code and not running it) are project artifacts which are tested using verification techniques. Requirement specifications are tested using requirement specification reviews, software designs are tested using design reviews and source code (construction) is tested using source code reviews. Validation is that part of software testing where the testing is done on software product by executing its source code. Business logic at class (component) level is tested using unit testing, integration of component is tested using integration testing, the software product is tested using system testing and finally the deployment of the software product is tested by end users using user acceptance testing.

8. Chapter 9 Software Testing (Verification & Validation) Levels of software testing (validation)

9. Chapter 9 Software Testing (Verification & Validation) Software testing (validation) is done at many levels. These levels include unit, integration, system and deployment. All of these levels come under validation. Software components are tested for business logic correctness at unit tests level. These components are then tested for integration with source code build using integration testing. The completely built software product after integration of all components is again tested at system level. At this level functional tests are done to test if software product features are working as per software requirement specifications. When the software product is ready then it is deployed at client site. The end users do user acceptance testing to ensure that the software product is working as per requirement specifications.

10. Chapter 9 Software Testing (Verification & Validation) Barry Boehm’s V Model for software testing

11. Chapter 9 Software Testing (Verification & Validation) Barry Boehm’s V Model is all about linking artifacts developed during software development lifecycle with levels of testing. Software requirement specifications are used to perform user acceptance testing. Software design specifications are used to perform system testing. Software unit construction (software component) is used to perform unit testing. Software module or sub system is used to perform integration testing.

12. Chapter 9 Software Testing (Verification & Validation) Software verification Verification tests include requirement specification reviews, software design reviews, source code reviews etc. Requirement specification reviews are done after these artifacts get completed. Software testers or business analysts review the requirement specifications for completeness, ambiguity and errors. If any errors or incompleteness is found then it is rectified. Software design reviews are done to check if the software design is correct as per requirement specifications. Software design reviews also include reviewing software component structure. If large classes were with many methods are designed then it will lead to difficult to understand source code. Similar structural problems can be found during design reviews. The identified design problems will need to be rectified. Source code reviews are done to check for dead code, wrong implementation of business logic etc. If any of these defects are found then they are rectified. Code walkthroughs are also used for source code checking. Here the developer presents his/her source code in front of other project team members. Team members review the source code and find any defects are there in the code. Software developer later rectifies those defects.

13. Chapter 9 Software Testing (Verification & Validation) Unit testing Unit testing is done after a class or component is constructed. Each class implements some business logic. The source code in the class is checked to verify if the business logic is implemented properly. Unit testing is also done for extended testing when database access is also involved. In such cases business logic as well as database access aspect also need to be considered. Testing database connection and testing if database operation actually happens are also done on these tests.

14. Chapter 9 Software Testing (Verification & Validation) class addition { public boolean adding (integer one, integer two){ if one >= 1 and one = 1 and two <= 100 { integer x = one + two; print (x); return true; } else return false; } The above class implements a business logic which needs to be tested. The test code can be as following: class testAddition { public method void testAdding () { boolean x = addition.adding(-3, 5); } The above test case is a negative test case as the expected result should fail. Using positive and negative tests, you can find out if your business logic works fine.

15. Chapter 9 Software Testing (Verification & Validation) class testAddition { public method void testAdding () { boolean x = addition.adding(-3, 5); } The class for addition operation can be tested using the above test class. For positive and negative testing, this test class can be provided with various values and these values will be passed to the class under test through parameters.

16. Chapter 9 Software Testing (Verification & Validation) public class interest_compute { public integer interest_compute (float interest) { integer principal = 293; integer accrued_amount; accrued_amount = principal + principal x interest/100; return accrued_amount; } In some cases, the business logic seems to be fine but its implementation is incorrect. The above example will compute the result correctly but the final result is incorrect as due to faulty variable declaration, the final computation is truncated.

17. Chapter 9 Software Testing (Verification & Validation) class data_access { public delete_row () { string y = ‘Alice’; integer x; search in the database table and find the record number where customer name is Alice and assign it to x; if x = record number of Alice then delete row from customer table where customer name = y; } When a class involves database access then apart from unit testing, database connection needs to be checked as well as to find out if database operation also took place. In the above example, you will also need to find out if the mentioned record has been deleted from the database. Database operations work on the principle of first searching a particular record and then doing database operations. If a particular record is changed or deleted after a test then the same test script will not work next time. Since the particular database record was changed after execution of test script the first time, next time you may need to change test script. If database connection and database operations are not needed to be tested then the database environment can be simulated using tools.

18. Chapter 9 Software Testing (Verification & Validation) Integration testing Integration testing is used to find out if a newly created class or component or module can be integrated correctly with the rest of the software product. Integration testing thus is a test of the interfaces of a class or component or module. If the interface of the class or component or module is correct then the class or component or module will integrate with the rest of the software product.

19. Chapter 9 Software Testing (Verification & Validation) Integration testing in continuous integration of source code build environment

20. Chapter 9 Software Testing (Verification & Validation) Continuous integration of source code build is used on agile projects. Here when a unit of source code (class or component) is written by developers is integrated with the source code build. Integration testing is performed before a piece of source code is integrated with the source code build. If integration testing fails then the unit piece of source code is checked if it has any interface problems due to which it is not able to integrate. Corrected piece of source code is again tested for integration. This continues till there are no integration failures. Once integration test passes then this piece of source code is integrated with the source code build.

21. Chapter 9 Software Testing (Verification & Validation) Frequency of integration of source code

22. Chapter 9 Software Testing (Verification & Validation) On Waterfall projects, separate modules of software product are developed first. They are later integrated with each other. On agile projects however, source code is continuously integrated. These 2 strategies imply that frequency of integration of source code vary considerably. Whereas in case of Waterfall projects, frequency of integration of source code is infrequent; in case of agile projects, the frequency is very high. In fact on agile projects, frequency of integration of source code with the source code build is 100%. Every time a new class is created, it is immediately integrated with the source code build. This discussion implies that integration testing on agile projects is done frequently. This leads to a better software product with less number of integration software defects. When modules of software product are tried to be integrated with each other then a large number of integration problems will arise. In this scenario, it is also difficult to pin point the cause of integration problems. Thus fixing integration problems is also difficult and it takes time in fixing them. This is the reason, software projects are increasingly using continuous integration of source code.

23. Chapter 9 Software Testing (Verification & Validation) System, regression and user acceptance testing Many types of testing are done on software projects during system and user acceptance testing. Functional testing is done to test if software product features are working as per software design and requirement specifications. Non functional testing is done to test if software product is working under acceptable limits even when environmental conditions are deteriorated. If a website receives tremendous web traffic but even with so much traffic, the website is able to respond user page requests within acceptable limit then performance of this website is considered good. Similarly if a website receives malware attacks but is still able to overcome these attacks well and keeps running then security of this website is considered good. Non functional tests include performance, security, usability, localization etc. types of tests. Regression tests are also performed during system testing for software products which are built incrementally. Regression tests check if existing functionality of a software product features still work after integration of increment of software product features.

24. Chapter 9 Software Testing (Verification & Validation) Software testing done in software development, deployment and maintenance

25. Chapter 9 Software Testing (Verification & Validation) Software testing is not only done during software development but also during deployment and maintenance. During software development all verification and validation tests are performed (requirement specification reviews, design review, code reviews, unit, integration, system (functional and non functional tests). Regression tests are also performed. But during deployment, only user acceptance testing is performed. During deployment, all black box testing (functional and non functional) tests are performed. The goal here is to validate that the software product works as per the requirement specifications. After deployment a software product is used by software users. This is the production phase of the software product. During production, software product is tested for performance and sanity tests. When the software product is taken for maintenance then regression tests are also done.

26. Chapter 9 Software Testing (Verification & Validation) Alpha-Beta testing cycle

27. Chapter 9 Software Testing (Verification & Validation) There are some other tests which are also done on software products. If alpha and beta release of a software product are done then alpha and beta testing is done for the software product. Smoke testing is done when a piece of source code is integrated with the source code build. Planned testing involves writing test cases and then executing them to find software defects. Planned testing has a limitation that only limited testing can be done within the allowed time limits for testing. Exploratory testing removes this barrier. Instead of spending too much time in writing test cases, exploratory testing does not use test cases at all. A software tester will just run various commands to execute business processes and find out if they are working correctly. If the tester finds any incorrect behavior then he/she will file a test defect with the details. Using this technique more testing of the software product can be done within a short span of time. A coverage based testing strategy is used when it is known in advance as to which part of the software product needs to be tested. This strategy is adopted when it is known that it is not possible to test the entire software product in the allotted time for software testing on a software project. A good strategy in such situation is to prioritize test cases and do testing only for test cases which are high on priority.

28. Chapter 9 Software Testing (Verification & Validation) A test case consists of test case description, entry criteria, exit criteria, expected result, actual result and test execution result (pass/fail). Test case description contains steps which are needed to be done during execution of a test case. Entry criteria is the conditions which should be valid before a test case can be executed. Exit criteria is the conditions which should be valid after a test case is executed. Expected result is the expected outcome of computation after execution of a test case. Actual result is the actual outcome of computation after execution of a test case. If actual result is same as expected result then the test case passes and it is assumed that the tested business process has worked as per expected. Thus there is no software defect. In contrast if the actual result is not the same as expected result then it is assumed that the tested business process has not worked as per expected. Thus there is a software defect and it needs to be fixed.

29. Chapter 9 Software Testing (Verification & Validation) Test case design Designing good test cases is important because all testing needs to be carried out within time limits. If test designs are not good then it may lead to ineffective testing. Even with limited number of good test cases it is possible to test a software product effectively. Using boundary value analysis it is possible to test a business process with just 3 test cases. In one test case, a test value which is above the allowed range of valid input can be used for a negative result and assess if the business logic works correctly. Similarly a value below the allowed range of valid input can be used for a negative result and assess if the business logic works correctly. Finally a value within the allowed range of valid input can be used for a positive result and assess if the business logic works correctly. Using decision trees it is possible to test complex business logic where a matrix of values are needed to test a business process.

30. Chapter 9 Software Testing (Verification & Validation) Test environment A good test environment is important for finding software defects. If a good test environment is not provided then it can lead to difficulty in finding defects. Once a software release needs to be tested then a stable and isolated test environment is created on a computer. Exact replica of the software product is installed on this environment. Test data is also created. Apart from the test environment, supporting facilities like defect tracking system, configuration management system etc. also need to be prepared for a testing cycle.

31. Chapter 9 Software Testing (Verification & Validation) Test cycle

32. Chapter 9 Software Testing (Verification & Validation) Test cycle is the process which is followed from test planning to test case writing to test execution to defect logging. Test planning involves finding out which software features need to be tested, how testing will be done and for how long testing activities will be carried out. Test case design involves finding out what testing techniques (boundary value analysis, decision trees etc.) will be used and how much test coverage will be done during test cycle. Test cases are written by testers after test design is decided. How many test cases needs to be written will depend on test coverage, testing techniques used etc. Test case execution starts when test preparation is done and test cases have been written. All the failed test cases will result in pointing to software defects. These software defects are logged so that they can be fixed.

33. Chapter 9 Software Testing (Verification & Validation) Software defect cycle

34. Chapter 9 Software Testing (Verification & Validation) When test case execution starts then some of the test cases will pass while others will fail. The failed test case point to software defects. These software defects are logged in a defect tracking system. The software developer who owns the software product feature which contains the software defect is notified. The software developer fixes the software defect and notifies the software tester who had logged the software defect. The tester then verifies if the software defect indeed has been fixed. If the defect is found to be fixed then the tester will close the software defect in the defect tracking system. This is the complete lifecycle of software defect and its fixing.