An Automated Testing Framework CPP Unit An Automated Testing Framework Copyright © 2016 Curt Hill

Introduction CPPUnit follows the more common and popular Junit The goal is to run a series of tests automatically Provide results that allow test failures to be converted into corrections The unit tests are usually maintained along with the code to be tested Any changes then provoke a full set of unit tests Copyright © 2016 Curt Hill

Aside on Languages The process is easier in some languages than others Java has several features that make it easier than in C++ A function calls are always dynamic Reflection allows a program to interrogate a module at run-time We also see coding that is legal but unusual for C++ Lots of scope resolution operators Anonymous new calls Copyright © 2016 Curt Hill

CppUnit Terms Tests Fixtures Suites Failures Errors A single method that checks one thing Fixtures One or more related test calls Generally tests the same method Suites A group of fixtures that is tested together Failures Tests that gave incorrect results Errors Tests that aborted Copyright © 2016 Curt Hill

Approach In building an automated test we create a test class This is a derivation of a fixture It uses the real class or classes that we wish to make unit tests for It has several void parameterless functions that do one or more tests We aggregate these tests into a suite We create a main program (console or GUI) that exercises the test case Copyright © 2016 Curt Hill

Legend We have at least two classes that we define The class that will be part of the production system This will be referred to as the target class The class that will be used to perform the unit tests on the target class This will be referred to as the test class If it is not clear which is being discussed: Ask! Copyright © 2016 Curt Hill

Example In this presentation we will see examples of actual code The target class will be CurtDate The test class will be CurtDateTest Copyright © 2016 Curt Hill

A test The basic call is to: CPP_ASSERT This is a function with a Boolean parameter If the Boolean is true this is a correct test If it is false this is an incorrect test If it aborts this is an error Errors are not the same as failures A sample call might look like this: CPPUNIT_ASSERT(d1->getMonth()==1); Copyright © 2016 Curt Hill

Preparation Before we can do the CPP_ASSERT there must be some preparation This usually involves creating variables and initializing them It may involve substantial execution prior to get values into a desired state The CPP_ASSERT then tests them against the expected values This usually requires a fixture Copyright © 2016 Curt Hill

A Fixture A fixture is a class derived from CppUnit::TestFixture It usually has private variables that are often pointers to the target class It has two methods that you may override setUp initialized the variables tearDown releases them It also has a series of methods that exercise one aspect of the target class Copyright © 2016 Curt Hill

Fixture Derivation The test class is a public derivation of a TestFixture: class CurtDateTest: public CppUnit::TestFixture No constructor is needed Some of the public methods are: void setUp(); void tearDown(); void defaultTest(); void threeIntegerTest(); Copyright © 2016 Curt Hill

Methods setUp prepares variables for use tearDown releases them In this example there are four pointers to CurtDate objects Named d1..d4 The code is next Copyright © 2016 Curt Hill

setup and teardown void CurtDateTest::setUp(){ d1 = new CurtDate(); d4 = new CurtDate(*d3); } void CurtDateTest::tearDown(){ delete d1; delete d2; delete d3; delete d4; Copyright © 2016 Curt Hill

Two Test Methods void CurtDateTest::defaultTest(){ CPPUNIT_ASSERT(d1->getMonth()==1); CPPUNIT_ASSERT(d1->getDay()==20); CPPUNIT_ASSERT(d1->getYear()==2016); } void CurtDateTest::threeIntegerTest(){ CPPUNIT_ASSERT(d2->getMonth()==1); CPPUNIT_ASSERT(d2->getDay()==2); CPPUNIT_ASSERT(d2->getYear()==2016); Copyright © 2016 Curt Hill

What is a test? Is a call to Answer: yes CPP_ASSERT a test? or defaultTest a test? Answer: yes The call to CPP_ASSERT tests one small portion of a larger test CurtDate is a good example We cannot declare success based on one CPP_ASSERT call Thus the entire method is counted as one test Copyright © 2016 Curt Hill

Who Calls? You might think you should be able to see the code that calls any of these four methods You would be wrong They will be called by a TestRunner But that will wait for some more explanation Copyright © 2016 Curt Hill

How many tests? The method defaultTest had three calls to CPP_ASSERT The method threeInteger had three How ever each method is considered a unit If they all pass it is one success Thus the final score that is seen is the summary of several tests Copyright © 2016 Curt Hill

A Suite A suite is a collection of tests Each test is a call to one of these test methods Now we must create a suite that is ready to do all of the method calls We will reference the methods but not call them Rather we will pass them to the suite for later calls Copyright © 2016 Curt Hill

Declaration In this case the suite will be a public static function If you do it this way the test methods do not have to be public It returns a Test pointer The declaration is: static CppUnit::Test * suite(); The implementation is what sets up the tests Copyright © 2016 Curt Hill

Implementation In the implementation we do two distinct things We create a TestSuite object The only parameter is a string to identify We add the methods to it that constitute the tests Parameter are an identifying string and the address of the method The object that is added is a template class TestCaller Copyright © 2016 Curt Hill

The Implementation CppUnit::Test * CurtDateTest::suite(){ CppUnit::TestSuite * suiteOfTests = new CppUnit::TestSuite("CurtDateTest"); suiteOfTests->addTest( new CppUnit::TestCaller<CurtDateTest>( "Default Constructor Test", &CurtDateTest::defaultTest)); suiteOfTests->addTest( new CppUnit::TestCaller<CurtDateTest>( "Three Integer Constructor Test", &CurtDateTest::threeIntegerTest); … return suiteOfTests; } Copyright © 2016 Curt Hill

Anonymous new usage Notice the new in the actual parameter In C++ we are most likely to use new to assign to a pointer This makes it easier to delete In Java the garbage collector finds unreferenced thing Thus this style is much more common Here the CppUnit code will delete the pointers given to it Copyright © 2016 Curt Hill

Calls? As mentioned earlier, who calls these methods? It is a TestRunner object The TestRunner object takes the Test pointer This result of suite method It has a run method that does all the work Copyright © 2016 Curt Hill

The main method int main(){ cout << "Testmain is starting\n"; CppUnit::TextUi::TestRunner runner; runner.addTest(CurtDateTest::suite()); runner.run(); cout << "\nAll done.\n"; return 0; } Copyright © 2016 Curt Hill

The output Testmain is starting ...F.E...E !!!FAILURES!!! Test Results: Run: 7 Failures: 1 Errors: 2 1) test: meant to fail Test (F) line: 58 CurtDateTest.cpp assertion failed - Expression: d2->getMonth()>1 2) test: First meant to abort Test (E) uncaught exception of unknown type 3) test: Second meant to abort Test (E) All done. Copyright © 2016 Curt Hill

Commentary This run had seven tests It had one failure Two errors One for each method added It had one failure A Boolean that was not true This is a result other than what was expected Two errors One was an abort in the test method The other an abort in the target class Copyright © 2016 Curt Hill

TestRunner Again There is no reason why only one set of tests is the limit The addTest method may be called repeatedly with calls from different suites Typically in a bottom up test scheme the lowest level object is tested Then more and more tests are added The top level class would be the last to be added Copyright © 2016 Curt Hill

Variations 1 In these examples only the private variables of the CurtDateTest class were used Those created and destroyed by setUp and tearDown These are just a convenience for when getting the initial values right is more complicated than this It is also possible to declare and initialize everything in the test methods Without any use of the private variables Copyright © 2016 Curt Hill

Variations 2 Similarly, the private variables of the test class do not have to be pointers As private variables the only code that can see them in the class code The unit test framework does not know about them Only the test class methods do They may also include any other supplementary variables that are needed Copyright © 2016 Curt Hill

Value of Pointers One of the virtues of pointers in this case is that the constructors are deferred until the time of new Thus, if a target default constructor aborts we will see that in setUp or in the method that declares Otherwise it may occur before CppUnit is ready to catch the error Copyright © 2016 Curt Hill

Output In this example the output went to the console This was from using the TextUi version of TestRunner: CppUnit::TextUi::TestRunner runner; Other possibilities exist Such as producing an XML file, which facilitates further automation We will not consider that Copyright © 2016 Curt Hill

Other Compilers CppUnit should compile on most other compilers In this example I used Visual C++ Previous versions of Visual Studio allowed integration of CppUnit Which allowed it to look somewhat like a debugger The include library needs to be available As well as cppunit.lib which contains the object code Copyright © 2016 Curt Hill

Installation We will use a simplified installation scheme The include files will be attached to the VC++ include directory A lib file will be added to the MakeFile This will be seen in the assignment Copyright © 2016 Curt Hill

Finally Let’s next look at the complete set of code After that we will be ready for an assignment or two Copyright © 2016 Curt Hill