Good Testing Practices John Tullis DePaul Instructor john.d.tullis@us.arthurandersen.com Copyrighted material John Tullis 1/18/2019

Good Testing Practices Testing Preparation All functions/methods should be coded so that -DDEBUG will turn on #ifdef DEBUG sections of the code, that will enable the output of trace statements to a debugging or testing log. All functions/methods should be coded so that the path of all regular and error logging files, all input files, and all initialization files; are set by environmental variables accessed by getenv(). (Note - often all paths are specified in the initialization file, and that file only is found by getenv(). Copyrighted material John Tullis 1/18/2019

Good Testing Practices Testing Preparation continued... All functions/methods should be unit tested by unit test drivers that contain the main() function. Unit test drivers should test all conditional code branches, including error handling branches not normally activated. Programs as opposed to functions or object methods should follow this same approach. However, they should be activated for testing by confirming a command line argument that informs them they are to execute testing conditions. Copyrighted material John Tullis 1/18/2019

Good Testing Practices Test Structure There should be a separate testing directory structure area, with subdirectories for: test documentation, test scripts, compiled test programs, log files, error files, expected results, actual results, and difference files (diffs between expected and actuals). There should be test management scripts that execute the test scripts. There should be initialization or cleanup scripts that set up a test for a given run by cleaning up the results of a previous run. This may include an optional archive capability to store the results of multiple runs. Copyrighted material John Tullis 1/18/2019

Good Testing Practices Clean Up Delete log files Delete tmp files Delete test data in database tables Load Test Data Load test data into database tables. Copyrighted material John Tullis 1/18/2019

Good Testing Practices Execute Test Execute shell script that sets environmental variables. Shell script executes test program. Test program calls function(s)/object method(s) tested. Test results sent to database or log files. Error conditions kicked to error log. (Note that errors are to be deliberately generated to test the error conditionals in the code!) Copyrighted material John Tullis 1/18/2019

Good Testing Practices Validate Results Execute shell script that compares actual results with expected results. (Note that when results are database changes they must be exported to the actual results directory to enable comparison with the previously exported expected results.) Comparisons may be done by “diff”, output redirected to a diff file. If that file is not empty, a problem was encountered. This can be determined by something like this: “tre=`wc -c tmp.dif | sed ‘s/tmp.dif//’` | if [ $tre -gt 0 ] then ech Results are not as expected in $testnumberwhatever. Copyrighted material John Tullis 1/18/2019