Test Automation Tools: QF-Test and Selenium by P. Kratzer / P.Sivera Software Engineer ESO

Introduction to QF-Test QF-Test is a GUI test tool for Java and web apps developed by German company Quality First Software GmbH (QFS) Vendor web site: http://www.qfs.de/en/ QF-Test pages: http://www.qfs.de/en/qftest/index.html Feature checklist: http://www.qfs.de/en/qftest/checklist.html

Introductions & Demos Demo screenshots: http://www.qfs.de/en/qftest/demos.html Demo video (length 8:40 min): http://www.qfs.de/en/qftest/demovideo.html Standard QF-Test library: http://www.qfs.de/en/qftest/stdlib.html

Typical workflow to create tests with QF-Test Create new test suite Define application (SUT) startup Run SUT (system under test) in QF-Test In recording mode: perform manual actions (includes component recognition) Check, adapt, parameterize recognized components (menus, windows, buttons, …) Use recorded actions as basis for new procedure(s), parameterize, … Define new test case, specify dependencies and call required procedures

ALMA-OT Test Implementation

Test Procedures

Test Procedure Detail

Test Documentation

Package Documentation

Short Demo Video to show a test set while it runs http://www.eso.org/~psivera/VideoOtAndQfTest/OtAndQfTestVideoHigherVolumeImproved.html

Test Results (Report)

Test Results (Run-Log)

Developers support Intervention on the code: call during the initialization phase to the method setName() for all GUI-elements event on a thread other than the AWT event dispatch thread (COMP-5141 and COMP-2905) (see http://en.wikipedia.org/wiki/Event_dispatching_thread) when important changes in the GUI are planned and are being developed. The sooner we are informed, the better.

Problems GUI test & QF-Test experience required Adaptations to client changes Component recognition issues, threads handling (cooperation with developers) QF-Test covers basic functionality but still requires a lot of manual work

Conclusions Once the manual work is done and the SUT is stable enough so that it does not require a lot of changes at every new release, the automated tests reveal to be very useful as regression tests. They are configurable, therefore can be run in whatever environment with whatever db and user (provided QFTest licence is available)

Selenium http://seleniumhq.org/ : Introduction, Documentation, Installation and Download, Tutorials… Example of a demo: http://vimeo.com/4671653 Products: Selenium IDE Selenium RC Selenium 2.0 ...

Selenium IDE Add-On for Firefox Record and Replay Tool Access to Selenium commands Save test cases in Selenese or many other programming languages

Selenium RC Selenium RC components are: The Selenium Server which launches and kills browsers, interprets and runs the Selenese commands passed from the test program, and acts as an HTTP proxy, intercepting and verifying HTTP messages passed between the browser and the AUT. Client libraries which provide the interface between each programming language and the Selenium RC Server.

Selenium and Zk ZK Web Applications can be a challenge with Selenium because ZK dynamically generates element IDs Workaround: not fully satisfactory QFTest: just starting with support for Zk Other products: Sahi: http://docs.zkoss.org/wiki/Making_ZK_Functional_Tests_With_Sahi