Testing Unit Testing In Evergreen Kevin Beswick Laurentian University / Project Conifer

This work is licensed under a Creative Commons Attribution-ShareAlike License.Creative Commons Attribution-ShareAlike

Overview Software Testing Unit Testing Unit Testing In OpenSRF/Evergreen Short intros to some of the tools used for unit testing in OpenSRF/Evergreen Test Driven Development Continuous Integration Challenges Ahead

Software Testing Software testing is the process that validates and verifies that a software application: 1. Meets business and technical requirements 2. Works as expected 3. Can be / is implemented with the same characteristics

Types of Software Testing Functional Testing Unit Testing Integration Testing System Testing System Integration Testing Smoke Testing Interface / Usability Testing Non-Functional Testing Performance / Load Testing Stress & Volume Testing Security / Penetration Testing Compatibility & Migration Testing Plus many more...

Current State of Software Testing in Evergreen / OpenSRF Most testing is done manually Constrictor is available to use for load testing A few collections of test cases are available on the Evergreen wiki for Functionality / User Interface testing Very few unit tests

What is Unit Testing? A method by which individual units of source code are tested to determine if they are fit for use Unit = smallest testable part of an application a function in procedural programming a method in object oriented programming

Why is Unit Testing Important? Facilitates change Simplifies integration Documentation Design

Unit Testing Basics Assertions Test Cases Test Fixtures Test Suites Test Execution

Current State of Unit Testing in OpenSRF / Evergreen There are very few existing unit tests in OpenSRF and Evergreen There is a continuous integration server in place, but it is not as useful without a good set of tests

What has been done about it so far? C, Perl, Javascript Unit Testing frameworks are in place C unit tests in OpenSRF Perl unit tests in OpenSRF Javascript unit tests in OpenSRF

C Unit Testing In Evergreen / OpenSRF Using the Check unit testing framework First result on a Google search for “C Unit Testing” Easy integration with automake Includes everything needed from a unit test framework, while being extremely simple

How To Run The C Tests The C tests are tied into the autotools, so running is just a matter of:./autogen.sh./configure [--options] make make check

How To Run The C Tests Running suite(s): osrf_message 100%: Checks: 6, Failures: 0, Errors: 0 PASS: check_osrf_mess age Running suite(s): osrf_json_object 100%: Checks: 23, Failures: 0, Errors: 0 PASS: check_osrf_json_ object Running suite(s): osrf_list 100%: Checks: 19, Failures: 0, Errors: 0 PASS: check_osrf_list Running suite(s): osrf_stack 100%: Checks: 1, Failures: 0, Errors: 0 PASS: check_osrf_stack Running suite(s): transport_client 100%: Checks: 9, Failures: 0, Errors: 0 PASS: check_transport_ client Running suite(s): transport_messag e 100%: Checks: 13, Failures: 0, Errors: 0 PASS: check_transport_ message =============== === All 6 tests passed =============== ===

Check Unit Testing Framework Basics Test Syntax: START_TEST(test_name)... END_TEST

Check Unit Testing Framework Basics Assertions: fail([message string]) fail_if(condition, [message string]) fail_unless(condition, [message string]) Optional message string will display the given message when the test fails – supports printf style format strings

Check Unit Testing Framework Basics Test Suites and Cases: Suite *my_suite(void) { Suite *s = suite_create(“My Suite”); Tcase *tc_core = tcase_create(“Core”); tcase_add_test(tc_core, my_cool_test); suite_add_tcase(s, tc_core); return s; }

Check Unit Testing Framework Basics Test Fixtures: Checked Fixtures  setup() and teardown() functions are run before and after each individual unit test in a test case  tcase_add_checked_fixture(tcase, setup, teardown); Unchecked Fixtures  setup() and teardown() functions are run before and after each test case  tcase_add_unchecked_fixture(tcase, setup, teardown);

Check Unit Testing Framework Basics Other features include: Test timeouts Determining Test Coverage Test Logging More info available at:

Perl Unit Testing Using the Test::More Perl module as a unit test framework This is what was being used in the past for the basic Perl module testing in OpenSRF and Evergreen One of the most popular Perl testing modules

How To Run The Perl Tests These are also tied into the autotools, so you would run them in the exact same way as the C tests./autogen.sh./configure [--options] make make check

How To Run The Perl Tests t/00-load.t /1 # Testing OpenSRF 2.00, Perl , /usr/bin/perl t/00-load.t ok t/01-Application.t ok t/02-AppSession.t ok t/03-DomainObject.t ok t/04-EX.t ok t/05-MultiSession.t ok t/06-System.t ok t/07-Transport.t ok t/08-Server.t ok t/09-Utils-Cache.t ok t/09-Utils-Config.t ok t/09-Utils-JSON.t ok t/09-Utils-Logger.t ok t/09-Utils-SettingsClient.t.. ok t/09-Utils-SettingsParser.t.. ok t/09-Utils.t ok t/pod-coverage.t ok t/pod.t ok All tests successful. Files=18, Tests=110, 5 wallclock secs ( 0.06 usr 0.02 sys cusr 0.24 csys = 3.31 CPU) Result: PASS

Test::More Basics use Test::More tests => 1; Basic Assertions: ok($got eq $expected, $test_name); is($got, $expected, $test_name); isnt($got, $expected, $test_name); is_deeply($got_complex_structure, $expected_complex_structure, $test_name);

JavaScript Unit Testing Dojo provides a JS unit testing framework called Dojo Objective Harness (DOH) Can test asynchronous functions such as XHR requests Cross browser compatibility Provides a command line interface, and browser based interface

How To Run the JS Tests Download a copy of Dojo to OpenSRF root wget tar xzf dojo-src.tar.gz Link your tests and custom OSRF dojo modules to your downloaded copy of Dojo ln -s src/javascript dojo-src/opensrf ln -s src/javascript/DojoSRF.js dojo- src/DojoSRF.js Run the tests cd dojo-src/util/doh sh runner.sh testModule=opensrf.tests.module

Or... Just download this shell script and run it from the OpenSRF root directory, and it will do everything for you. Disclaimer: Not guaranteed to work with your system configuration.

How To Run the JS Tests The Dojo Unit Test Harness, $Rev: $ Copyright (c) 2011, The Dojo Foundation, All Rights Reserved tests to run in 8 groups GROUP "osrfServerErrorTests" has 3 tests to run GROUP "JSONTests" has 25 tests to run GROUP "osrfMessageTests" has 6 tests to run | TEST SUMMARY: tests in 8 groups 0 errors 0 failures

Dojo Objective Harness (DOH) dojo.provide('some.dojo.module.name'); dojo.register(“testSuiteName”, [...testcases... ]); Test Cases are just normal functions Assertions doh.assertTrue(condition); doh.assertFalse(condition); doh.assertEqual(thing1, thing2); doh.assertNotEqual(thing1, thing2);

Dojo Objective Harness (DOH) Alternate Test Case Definition: { name: “testName”, setUp: setupFunction, runTest: testCaseFunction, tearDown: teardownFunction }

Dojo Objective Harness (DOH) Robot doh.robot adds an API to the Dojo Objective Harness framework that lets testers automate UI tests Cross platform Can be used for unit testing, and accessibility testing

Dojo Objective Harness (DOH) For more info on this, check out: a-aj-doh/index.html

Test Driven Development (TDD) Write the tests before you write the code Encourages simple designs, and inspires confidence according to Kent Beck Makes it easier to refactor code after the fact – passing tests ensures that the functionality remains the same when code is refactored

TDD Development Cycle 1. Write a test 2. Run all tests and see if the new one fails 3. Write some code 4. Run all tests and see them succeed 5. Refactor code 6. Repeat

Submitting a Patch To Fix a Bug Next time you submit a bug fix patch... Write the accompanying tests which fail before the bug fix is applied, and which pass after the bug fix is applied Doing this ensures that: Re-introducing the same bug in the future is far less likely Test coverage remains complete (as complete as it was prior to the patch)

Continuous Integration Recently, Dan Scott set up a continuous integration server Uses the buildbot Python project Builds code and runs tests after code is committed to the repositories Builds different versions of both OpenSRF and Evergreen on different platforms

Continuous Integration Evergreen continuous integration server Buildbot Project

Challenges Ahead There are still many tests to write Solutions need to be worked out for areas that require more complex testing Getting into the habit of submitting tests with patches

Call for Action There are currently unit testing frameworks in place for most of the languages used in Evergreen Write some tests for an area with little test coverage Submit tests with your patches

Questions ?

Thank you! kbeswick on IRC on Twitter