1. © 2012 IBM Corporation Rational Insight | Back to Basis Series Chao Zhang 06-07-2012 Unit Testing

2. © 2012 IBM Corporation Rational Insight | Back to Basis Series 2 06/19/12 Agenda Overview Unit Test introduction – Objective – Value – Approaches – Test Planning, Test Area and Test Cases – Test Process Unit Test Example Discuss

3. © 2012 IBM Corporation Rational Insight | Back to Basis Series 3 06/19/12 Overview: Unit Test Scope Unit Testing covers various aspects of verifying the functionality implemented for a feature activity. Debugging Phase Boundary Conditions Error Paths Functional Testing Stability/Memory Leaks Feature Compliance Error Paths Integration Testing Feature Integration Feature Verification FVT/SVTFeature design/Defect fixing Formal Test Capture - Rational Quality Management White/Grey box tests (automated) Grey/Black Box Functional Tests Unit Test

4. © 2012 IBM Corporation Rational Insight | Back to Basis Series 4 06/19/12 Unit Test Why is it so important? A good way to exercise your API and make sure it's usable and practical. Effective way to catch low level defects in modified code. Feeling good about what you deliver. Deliver the best product possible. Effective way to verify error paths that cannot be verified with the system test. Prevent you from wasting time on bugs that are easily caught and identified at unit test time. Throwing untested functionality "over the wall" is unacceptable.

5. © 2012 IBM Corporation Rational Insight | Back to Basis Series 5 06/19/12 Unit Test Value Best time to validate the error paths. Find bugs in smaller amount of code in a more controlled environment. Integration with other components is easier when everyone uses same methodology for Unit Test. Proper unit test can deliver reduced costs, increased quality, and better adherence to project schedule. Defects found in unit test are less expensive than those found in system test

6. © 2012 IBM Corporation Rational Insight | Back to Basis Series 6 06/19/12 Unit Test Testing approaches White Box Testing - Code is tested for execution correctness at a very detailed level. This type of testing often requires the use of tools and simulators to help with the testing. White box testing requires knowledge of and access to the code. Grey Box Testing - Testing is done more at the component level for code execution correctness but also functional correctness. This may require tools and simulators to complete the testing. Black Box Testing - Code is treated as a "black box". Tests are conducted from a functional point of view only for functional correctness against the functionality specified in plan item, enhancement or defect description. The implementation details are irrelevant at this stage. Black box testing does not require knowledge or access to the code.

7. © 2012 IBM Corporation Rational Insight | Back to Basis Series 7 06/19/12 Unit Test Test Planning Are the functional requirements well documented and understood? It is very important to test components according to provided specifications/requirements. What scope of testing is required? Which hardware and software should be covered? Consider three perspectives when planning a test activity: Isolated testing - testing of a specific component in an isolated matter Off-target testing - simulating the functionality without real h/w On-target testing - a must, testing using the real h/w in the lab environment What metrics or data, if any, need to be prepared? What metrics or data, if any, need to be collected? What codes is being included in the coverage? What codes is being excluded from the coverage?

8. © 2012 IBM Corporation Rational Insight | Back to Basis Series 8 06/19/12 Unit Test Test Areas Consider what you will be testing and identify which area of your design will be tested. This may include specific software modules or components that will be included in test. Consider the following test plan aspects when determining what tests are needed. These are suggestions and may not apply to all situations: – Ensure all affected user actions/notifications for the component(s) under test are exercised. – Ensure there are no SWERRs (software error reports) emitted under normal circumstances by the component(s) under test. – Exercise all success paths. – Evaluate all boundary conditions. – Exercise all error and exception paths within the code. – Exercise and validate variable initialization

9. © 2012 IBM Corporation Rational Insight | Back to Basis Series 9 06/19/12 Unit Test Test Cases A test case is a specific set of data and associated procedures (or test script) developed for testing of a particular objective. For example all these paths are exercised – Success tests - Exercise success paths – Error handling tests - Exercise error handling for both critical errors as well as errors that only occur under extreme/rare conditions. – Boundary tests - Exercise all boundary values Test cases should have enough information in it so that it can be run by a person other than the author.

10. © 2012 IBM Corporation Rational Insight | Back to Basis Series 10 06/19/12 Unit Test Test Process of White Box White Box Testing: Write test cases document Coding your test suite/ test case with some language framework: Junit, CppUnit Running your test suite and test case, check the output to make sure the test results are our expected. If error found, correct and run test case again until the issue is solved.

11. © 2012 IBM Corporation Rational Insight | Back to Basis Series 11 06/19/12 Unit Test Test Process of Grey/Black Box Grey/Black Box Testing: Write test cases document. Build your personal build or special component jar which will be tested Install your personal build or replace the component jar in an available environment Simulate user normal and exception actions to execute the functionality and make sure the test results are our expected. Test it on different platform or hardware which we need support If error found, correct and repeat test steps until the issue is solved

12. © 2012 IBM Corporation Rational Insight | Back to Basis Series 12 06/19/12 Unit Test Example Memory Leak of ReqPro DataService - White Box Testing White Box Testing: Create a single project with the new Java class files and make sure there is no compilation error. Prepare the server environment: RequisitePro Write the test cases to cover the following testing scenarios: – Cover all methods of class – Cover all codes path as could as possible – Cover all basic functions – Cover all exception cases – Cover the single thread and multi-thread in concurrence scenario Running test cases and make sure the test results are our expected, if not, correct codes and run again.

13. © 2012 IBM Corporation Rational Insight | Back to Basis Series 13 06/19/12 Unit Test Example Memory Leak of ReqPro DataService - Black Box Testing Black Box Testing: Create a personal build. Install the personal build and prepare the testing environment. Run the test cases to cover the following scenarios: – Make sure the RePro Dataservice start/stop normally and the basic functions are working. – Make sure the other Dataservices are working. – Simulate the multi-user to retrieve the data from ReqPro DataService and make sure all data can be returned successfully. – Long run to retrieve the data in concurrence scenario, observe the memory changes of RqProx.exe, JVM process and make sure the memory is ok. – Make sure the ETL can be run successfully. – After long run with multi-user scenario and shut to the DataService server, make sure all RqProxy can be existed and the memory can be released – Build exception environment and make sure the DataService work as our expected. – Check the ReqPro DataService logs, Make sure our need information is logged in different log level and the logs are enough if exception or error is happened.

14. © 2012 IBM Corporation Rational Insight | Back to Basis Series 14 06/19/12 Discussion What's your view about the unit test? What's your problem in your unit test? What's your suggestion about how to do unit test for RRDI/Insight components? What's your suggestion about the test case? What's your suggestion about the test process?

15. © 2012 IBM Corporation Rational Insight | Back to Basis Series 15 06/19/12 www.ibm/software/rational

16. © 2012 IBM Corporation Rational Insight | Back to Basis Series 16 06/19/12 Revision History AuthorDateVersionComment Chao Zhang2012-05-300.01Initial Draft Chao Zhang2012-06-010.02Modified by Na Deng's comments