1. SciDAC SSS Quarterly Report Sandia Labs August 27, 2004 William McLendon Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000.

2. APItest - Overview Current Release : v0.2.0 Test Driver tool for testing software –Ordered tests –API Testing Available for download: –ftp://ftp.sandia.gov/outgoing/apitest/ftp://ftp.sandia.gov/outgoing/apitest/

3. APItest - Features Scripted Test Driver –XML based tests Object Oriented –Easy to define new test types. Conditional test ordering –Batch Scripts Can call other batches Built In Test Types –Shell –Script –SSS Can set environment variables for tests. SUID Test Capability –CAUTION! Timeouts User Interface Types –Web Browser –Command Line Test Results Export –Save Test results to an XML file. Open Source –LGPL Fully scripted in Python Portable

4. APItest - Requirements Packages required to run APItest: –Python 2.3 or greater –ElementTree (www.effbot.org)www.effbot.org –Twisted (www.twistedmatrix.com)www.twistedmatrix.com –ssslib (ftp://ftp.mcs.anl.gov/pub/sss/)ftp://ftp.mcs.anl.gov/pub/sss/ optional, for SSSlib tests.

5. APItest - Execution There are two ways to execute APItest Command Line Only –$> apitest -f test_file.[apt|apb] Web browser GUI (Interactive) –$> apitest httpd -i [--iroot] : root directory to look for tests -p [--port] : port number (default = 2112) -h [--host] : host (default = localhost) --help will display command line usage.

6. APItest - GUI (Top Page) Top level index page for APItest APItest recurses from the [testroot] to find tests. File Extensions: –TEST :.apt –BATCH :.apb Batch tests shown in bold Organized by directory

7. APItest - GUI (Test Run Execution) Quick status view of test execution. This screen loads when the EXECUTE button is selected. Color coded for easy viewing. Currently running test is indicated. Refreshes every 15 secs.

8. APItest - GUI (Test Run Summary) Summary of test results Color coding Batch Features –Cumulative summaries Total # of tests # that passed (green) # that failed (red) Can click the hyperlinks to see more details about a test or batch.

9. APItest - GUI (Batch Summary) Clicking a hyperlink into a batch test gives us a summary view. In this example, one test failed a dependency –FAILDEP in the status column indicates this. Clicking on the scriptTest2.apt link will show us more data about this test.

10. APItest - GUI (Test Details - FAILDEP) Detailed view of scriptTest2.apt from previous slide. Failed a dependency on cmdTest2.apt –Expected : FAIL –Actual : PASS

11. APItest - GUI (List of Test Runs) Clicking the View All Runs button gives this screen. Test Runs are sorted by date and time. Click links to view details of a particular run.

12. APItest - GUI (Test Details - PASSED) Example of a test that was executed and passed. Summary Data Dependency Data –Only shown if test was run as part of a batch. Output –Side-by-side comparison –Dependent on type of test.

13. SSS - Service Directory Startup Test PREPTEST SDOFF SSS STARTTEST SDON RM EMNGSDSTAT SSS STOPTEST SDOFF

14. APItest - Ongoing Work Develop more tests for SSS components! Validation via file matching. Configuration file. Test Developer GUI. Results browser. Session management and security (Cookies?) Additional native test types. –HTTP, TCP/IP, XMLRPC DB Connectivity. User Guide. Bug fixing.

16. APItest - Test Types CMD –Execute a command via the command line. SCRIPT –Execute a script. SSS –Execute a ssslib communication.

17. APItest - Example CMD Test Example test. Runs ls -lr from /tmp, expects 0 exit status and any input. ls -l -r.* 0

18. APItest - Example Script Test Timeout test for x in 1 2 3 4 5 6 7 8 9 10 do echo "timeout_test :${x}: `date`” sleep 1 done.*timeout.* 0

19. APItest - Example ENVVAR Test This script executes but adds some stuff to the environment. NOTE: setting envvar OVERRIDES any inherited environment variables. #!/bin/csh env foobar foo.*test_envvar_1=foobar\n.*test_envvar_2=foo\n.* 0

20. APItest - Example SUID Test #!/usr/bin/env python import os print os.getuid() print os.getgid() /tmp/testuser/ -l ^27030$\n^500$ 0

21. APItest - Example SSS Test Sample sss query of the service directory <get-location><location component='service-directory' host='*' port='*' protocol='*' schema_version='*' tier='*'/></get-location>.*