1. Object Oriented Analysis & Design Testing

2. Contents 2  Testing  Unit testing  CppTest  Regression testing  Equivalence testing  White & black box testing  Test coverage  Test plans  Test roles

3. Testing 3  Testing seeks to  Determine if the game meets requirements  If the game works as specified  If the game is bug-free  We test using  Unit tests  Tests by playing the game  Testing seeks to  Determine if the game meets requirements  If the game works as specified  If the game is bug-free  We test using  Unit tests  Tests by playing the game

4. Importance of Testing 4  Only high-quality games will sell  Players do not like bugs in games  It is difficult to get a program correct  Only testing can determine if a program is correct  Testing is an important part of producing a game!  Only high-quality games will sell  Players do not like bugs in games  It is difficult to get a program correct  Only testing can determine if a program is correct  Testing is an important part of producing a game!

5. Unit Testing 5  Unit Testing  Tests each class  Tests every method on every class  Unit testing shows that the classes work before they are put into a complete game  Unit tests are written as the classes are developed  Unit Testing  Tests each class  Tests every method on every class  Unit testing shows that the classes work before they are put into a complete game  Unit tests are written as the classes are developed

6. Test Harness 6  Unit testing involves  Writing code to perform the tests  Checking the results  Reporting the results  There are frameworks which can aid with  A template to write the tests  An easy way to check results  Automating the reporting of results  Unit testing involves  Writing code to perform the tests  Checking the results  Reporting the results  There are frameworks which can aid with  A template to write the tests  An easy way to check results  Automating the reporting of results

7. CppTest 7  CppTest is  A lightweight unit testing framework for C/C++ programs  It can be downloaded from  http://cpptest.sourceforge.net  CppTest is  A lightweight unit testing framework for C/C++ programs  It can be downloaded from  http://cpptest.sourceforge.net

8. A Test 8  A test is  A void method with no parameters  Contains code to calculate a result  Uses a test assertion to compare the calculated result with the expected result  If the result is incorrect, the assertion will report the bug  A test is  A void method with no parameters  Contains code to calculate a result  Uses a test assertion to compare the calculated result with the expected result  If the result is incorrect, the assertion will report the bug

9. Test Suites 9  A test suite is  A class derived from Test::Suite  Contains one or more test methods or other test suites  Adds each of the tests to the suite in the constructor  Can be run by calling the run() method which will run all tests in the suite  A test suite is  A class derived from Test::Suite  Contains one or more test methods or other test suites  Adds each of the tests to the suite in the constructor  Can be run by calling the run() method which will run all tests in the suite

10. Assertions 10  TEST_ASSERT(boolean_expr)  Boolean expression is true to pass test  TEST_ASSERT_MSG(bool_expr, msg)  If test fails, msg is printed  TEST_ASSERT_DELTA(a, b, delta)  Test fails if a is not equal to B +/- delta  TEST_ASSERT_DELTA_MSG(a, b, delta, msg)  If test fails, msg is printed  TEST_ASSERT(boolean_expr)  Boolean expression is true to pass test  TEST_ASSERT_MSG(bool_expr, msg)  If test fails, msg is printed  TEST_ASSERT_DELTA(a, b, delta)  Test fails if a is not equal to B +/- delta  TEST_ASSERT_DELTA_MSG(a, b, delta, msg)  If test fails, msg is printed

11. Sample Test Suite 11 #include class MathTestSuite: public Test::Suite { public: MathTestSuite() { TEST_ADD(MathTestSuite::addTest) } private: virtual void setup() { /* init code before first test */ } virtual void tear_down() { /* cleanup code after last test */ } void addTest() { TEST_ASSERT_MSG(2 + 2, 4, “this does not add up”); } }; #include class MathTestSuite: public Test::Suite { public: MathTestSuite() { TEST_ADD(MathTestSuite::addTest) } private: virtual void setup() { /* init code before first test */ } virtual void tear_down() { /* cleanup code after last test */ } void addTest() { TEST_ASSERT_MSG(2 + 2, 4, “this does not add up”); } };

12. Running a Test Suite 12 int main(int argc, char **argv) { MathTestSuite mts; Test::TextOutput output(Test::TextOutput::Verbose); mts.run(output, true);// true runs rest of tests after a // test fails } int main(int argc, char **argv) { MathTestSuite mts; Test::TextOutput output(Test::TextOutput::Verbose); mts.run(output, true);// true runs rest of tests after a // test fails }

13. Test Guidelines 13  Each test should test one simple operation  Each test should contain one assertion  Related tests should be in the same test suite  Related test suites can be combined into a test suite using the add(Test::Suite) method of the test suite  Each test should test one simple operation  Each test should contain one assertion  Related tests should be in the same test suite  Related test suites can be combined into a test suite using the add(Test::Suite) method of the test suite

14. Regression Testing 14  Once a test suite is created, it can be easily run again and again  This means we can run the test suite  After every change to the code  Before the changes are checked into the repository  Before you ship a release  Once a test suite is created, it can be easily run again and again  This means we can run the test suite  After every change to the code  Before the changes are checked into the repository  Before you ship a release

15. Selecting Test Cases 15  It is impossible to test all possible values  We identify groups of values which test the same thing  Then we test only one value from each equivalence group  It is impossible to test all possible values  We identify groups of values which test the same thing  Then we test only one value from each equivalence group

16. Equivalence Testing 16  Most bugs in programs occur at the boundaries of the data or on special values  Zero is a special value  MAXINT and MININT are boundaries  An empty string and the longest permisable string are boundaries  A NULL string is a special value  Most bugs in programs occur at the boundaries of the data or on special values  Zero is a special value  MAXINT and MININT are boundaries  An empty string and the longest permisable string are boundaries  A NULL string is a special value

17. Equivalence Testing 17  Test the boundary values  Test values adjacent to the boundary values  Test values just outside the boundaries to see if the code handles these correctly  Test a few routine values to make sure they work  Test values that should cause the code to fail  Test the boundary values  Test values adjacent to the boundary values  Test values just outside the boundaries to see if the code handles these correctly  Test a few routine values to make sure they work  Test values that should cause the code to fail

18. Sample Equivalence Tests 18  You need to test a function which takes a single integer parameter greater that zero  Use the following test data  Largest negative value  -100 random negative value  -1near boundary  0boundary  +1near boundary  8routine even value  9 routine odd value  Largest positive value -1  Largest positive value  You need to test a function which takes a single integer parameter greater that zero  Use the following test data  Largest negative value  -100 random negative value  -1near boundary  0boundary  +1near boundary  8routine even value  9 routine odd value  Largest positive value -1  Largest positive value

19. Black Box Testing 19  Treats the software as a black box which you do not know how it works  Tests that the documented features work correctly  The black box tests can be designed as soon as the requirements are available  Treats the software as a black box which you do not know how it works  Tests that the documented features work correctly  The black box tests can be designed as soon as the requirements are available

20. White Box Testing 20  Tests designed based on a knowledge of the internals of the game  Must be designed by a programmer  Creates tests which  Test the path through every if and switch  Test unusual error conditions  Test infrequently executed code  Invoke specific methods  Tests designed based on a knowledge of the internals of the game  Must be designed by a programmer  Creates tests which  Test the path through every if and switch  Test unusual error conditions  Test infrequently executed code  Invoke specific methods

21. Testing Coverage 21  Most testing leaves many parts of the code unexecuted  A coverage analyzer can tell you the number of times each statement was executed  This can identify untested code  You can design white box tests to test the untested code  Most testing leaves many parts of the code unexecuted  A coverage analyzer can tell you the number of times each statement was executed  This can identify untested code  You can design white box tests to test the untested code

22. Game Testing 22  When most people speak of game testing, they mean functionality testing  This involves  Performing every operation  Performing every combination of operations  Executing every menu item  Executing every level  Ensuring the results are as expected  When most people speak of game testing, they mean functionality testing  This involves  Performing every operation  Performing every combination of operations  Executing every menu item  Executing every level  Ensuring the results are as expected

23. Functionality Testing 23  Functionality testing is performed by  Creating a test plan which  Specifies how to set up each test  Specifies the steps in conducting each test  Specifies the expected results  The game tester is responsible for  Executing each of the tests  Reporting any bugs found  Functionality testing is performed by  Creating a test plan which  Specifies how to set up each test  Specifies the steps in conducting each test  Specifies the expected results  The game tester is responsible for  Executing each of the tests  Reporting any bugs found

24. Compliance Testing 24  Special agreements are imposed by  Console manufacturers  Software / hardware vendors  Governmental agencies  Entertainment Software Rating Board  Compliance testing involves checking that the game does not violate any of these agreements  Special agreements are imposed by  Console manufacturers  Software / hardware vendors  Governmental agencies  Entertainment Software Rating Board  Compliance testing involves checking that the game does not violate any of these agreements

25. Compatibility Testing 25  Games need to test for compatibility with  Different versions of the operating system  Different versions of graphics libraries  Different versions of third party libraries  Different input devices (mice & joysticks)  Different graphics cards  Processors with different numbers of cores  The minimal hardware the game requires  Games need to test for compatibility with  Different versions of the operating system  Different versions of graphics libraries  Different versions of third party libraries  Different input devices (mice & joysticks)  Different graphics cards  Processors with different numbers of cores  The minimal hardware the game requires

26. Localization Testing 26  Most games are sold in international markets and have to be correct in many languages  Check for  Is the locale automatically detected ?  Are all menus translated correctly ?  Is printed dialogue translated correctly ?  Is audio correct ?  Are number, date and moetary formats correct?  Are there offensive cultural references?  Most games are sold in international markets and have to be correct in many languages  Check for  Is the locale automatically detected ?  Are all menus translated correctly ?  Is printed dialogue translated correctly ?  Is audio correct ?  Are number, date and moetary formats correct?  Are there offensive cultural references?

27. Cultural Considerations 27  What is good in one culture is bad in another  Some numbers are bad luck in some cultures  Any hand gesture is obscene somewhere  Colours have different emotional connotations in different cultures  Cultures have different attitudes towards  Men and women  Religion  Authority  Short term versus long term thinking  What is good in one culture is bad in another  Some numbers are bad luck in some cultures  Any hand gesture is obscene somewhere  Colours have different emotional connotations in different cultures  Cultures have different attitudes towards  Men and women  Religion  Authority  Short term versus long term thinking

28. Soak Testing 28  This involves leaving the game running for long periods of time in different modes  Can also involve performing the same operation a large number of times  Can detect  Memory leaks  Buffer overflows  Race conditions  Huge output files  This involves leaving the game running for long periods of time in different modes  Can also involve performing the same operation a large number of times  Can detect  Memory leaks  Buffer overflows  Race conditions  Huge output files

29. Load Testing 29  This puts a high load on the system to ensure it will not fail  A large number of players on an MMO  A large number of enemy characters  A large number of threads  Fast input over extended time periods  Large amounts of scenery  This puts a high load on the system to ensure it will not fail  A large number of players on an MMO  A large number of enemy characters  A large number of threads  Fast input over extended time periods  Large amounts of scenery

30. Multiplayer Testing 30  Tests to see that the program works well with multiple players  Supports large numbers of players  Works over various network types  Has good performance under high network load  There is acceptable lag in updating the view of each player  Single player mode works as well as multiplayer  Tests to see that the program works well with multiple players  Supports large numbers of players  Works over various network types  Has good performance under high network load  There is acceptable lag in updating the view of each player  Single player mode works as well as multiplayer

31. Installation Testing 31  Does the program install correctly on all target systems ?  Are the correct resources installed for each locale ?  Is the installer in the right language ?  Does the documentation agree with the version of the game it shipped with ?  Can users understand the documentation ?  Does the program install correctly on all target systems ?  Are the correct resources installed for each locale ?  Is the installer in the right language ?  Does the documentation agree with the version of the game it shipped with ?  Can users understand the documentation ?

32. The Test Plan 32  The test plan  Provides details of all the test to conduct  For each test it specifies  How to prepare for the test  The steps in performing the test  How to determine if the test passed or failed  The test plan  Provides details of all the test to conduct  For each test it specifies  How to prepare for the test  The steps in performing the test  How to determine if the test passed or failed

33. The Test Report 33  Produced after conducting a test  States  The test conducted & time & date  Any special conditions for the test  The result(s) of the test  If the test failed  All available logs from the game when the test failed  Observations by the tester  Produced after conducting a test  States  The test conducted & time & date  Any special conditions for the test  The result(s) of the test  If the test failed  All available logs from the game when the test failed  Observations by the tester

34. Roles in Testing 34  Game producer  Sets testing deadlines in conjunction with marketing  Lead tester  Manages the testing process  Keeps records on the tests performed  Tracks bugs reported  Understands how to program  Game producer  Sets testing deadlines in conjunction with marketing  Lead tester  Manages the testing process  Keeps records on the tests performed  Tracks bugs reported  Understands how to program

35. Roles in Testing 35  Test Designer  Responsible for designing the tests  Usually a programmer who knows the type of things which go wrong  Tester  The person who actually conducts the tests  Does not require programming skills  Must be detail oriented  Must like repetitive tasks  Test Designer  Responsible for designing the tests  Usually a programmer who knows the type of things which go wrong  Tester  The person who actually conducts the tests  Does not require programming skills  Must be detail oriented  Must like repetitive tasks

36. Beta Testing 36  After testing within the company is complete, the game is released to select customers  They get the game early with the agreement that they will  Play the game a lot  Report any bugs they find  Send in logs produced at the time the bugs occurred  After testing within the company is complete, the game is released to select customers  They get the game early with the agreement that they will  Play the game a lot  Report any bugs they find  Send in logs produced at the time the bugs occurred