1. Slide 1 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. XQuery Unit Testing Mark Helmstetter, Senior Consultant NOVA MUG - June 29, 2010

2. Slide 2 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. A Story about Testing

3. Slide 3 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Agenda  Unit Testing Basics  Unit Testing XQuery via XQuery  Unit Testing XQuery via Java  End-to-End Testing using Selenium  Testing Tips

4. Slide 4 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Unit Tests  Test individual functions or modules (units)  A unit is the smallest testable part (function)  White-Box/Glass-Box – derived from internal structure  Test each unit separately, in a controlled environment  Repeatable with the same results  Facilitate automation  Facilitate change, code confidence, agility  Allows for “modular” development – build backend before UI

5. Slide 5 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Steps of a Unit Test 1.setup() 2.Call function(s) under test 3.Verify (assert) results from each function call 4.teardown()

6. Slide 6 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. TDD Basics  Add functionality in very small chunks  Write the test before we implement the unit 1.Decide how each unit will be tested 2.Write the unit test code 3.Implement (code) the unit 4.Run the test, ideally any related unit tests, or even all tests 5.Refactor to make unit cleaner, more efficient 6.Repeat

7. Slide 7 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. XQuery Unit Testing  Common approaches to XQuery Unit Testing  Write the test code in XQuery  Leverage full capability of XQuery within test code  Write the test code in some other language (Java)  Run tests directly from Eclipse  Leverage JUnit and related frameworks  Declarative test script using XML  Framework (XQuery, Java, etc.) reads XML, executes tests as described  Ideally test environment mimics execution environment as closely as possible

8. Slide 8 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. TDD Process http://en.wikipedia.org/wiki/Test-driven_development

9. Slide 9 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. XQuery Unit Testing – What’s different?  Very similar to testing other languages  Techniques similar to testing other databases  Connecting to a database, considered integration test (?)  Not trivial to mock MarkLogic Server or an XQuery environment  Better to use MarkLogic Server  Keep your datasets small

10. Slide 10 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Agenda  Unit Testing Basics  Unit Testing XQuery via XQuery  Unit Testing XQuery via Java  End-to-End Testing using Selenium  Testing Tips

11. Slide 11 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. xqunit – common functions for testing module namespace xq = "http://marklogic.com/xqunit"; declare function xq:assert-equal($name, $actual as item()*, $expected as item()*) { if (fn:deep-equal($expected, $actual)) then else {$expected} {$actual} };...

12. Slide 12 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 1: search-lib-test.xqy xquery version "1.0-ml"; import module namespace search="http://marklogic.com/search" at "search-lib.xqy"; import module namespace xq = "http://marklogic.com/xqunit" at "/lib/xqunit.xqy"; declare function local:test-get-query() { let $query as schema-element(cts:query)? := search:get-query("foobar") let $expected := document{cts:word-query("foobar")}/node() return xq:assert-equal("test-get-query", $query, $expected) }; {local:test-get-query()}

13. Slide 13 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 1: search-lib.xqy xquery version "1.0-ml"; module namespace search="http://marklogic.com/search"; declare function get-query($query-str as xs:string) as schema-element(cts:query)? { if ($query-str eq "") then () else document{cts:word-query($query-str)}/node() };

14. Slide 14 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Execute test via browser

15. Slide 15 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 1 (Continued)... declare function local:test-get-query-empty() { let $query as schema-element(cts:query)? := search:get-query(()) return xq:assert-equal("test-get-query-empty", $query, ()) }; declare function local:test-get-query-empty-string() { let $query as schema-element(cts:query)? := search:get-query("") return xq:assert-equal("test-get-query-empty-string", $query, ()) }; {local:test-get-query(), local:test-get-query-empty(), local:test-get-query-empty-string()}

16. Slide 16 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 1: Test PASSED!

17. Slide 17 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 1: Many more test cases to consider  test-get-query-stemmed()  test-get-query-case-sensitive()  test-get-query-phrased()  test-get-query-diacritic-sensitive()  test-get-query-boolean-and()  test-get-query-boolean-or()  …

18. Slide 18 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Loading Test Data  Previous test example did not require any database data  Many XQuery functions require some data  Unit tests should insert test data in set-up()  Framework invokes set-up() before each test function  Unit tests should remove all test data in tear-down()  Framework invokes tear-down() after each test function

19. Slide 19 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Test Environment Provisioning  One database per developer  Guarantee known state of database  In a shared server environment, need 1 unit test database per developer  Or ideally, each developer has a local MLS instance for testing  Use a separate database used exclusively for unit tests  Typically have a database with a larger set of data for ad-hoc application testing, keep that separate  Use MarkLogic Admin API  Script database configuration, make test setup easier  Ensure consistent configuration across environments

20. Slide 20 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Agenda  Unit Testing Basics  Unit Testing XQuery via XQuery  Unit Testing XQuery via Java  End-to-End Testing using Selenium  Testing Tips

21. Slide 21 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Unit Testing XQuery from Java  Why test from Java?  Many frameworks to help facilitate  JUnit, XMLUnit, HtmlUnit, XQueryUnit  Tests can be run directly from the IDE  Eclipse, IntelliJ, NetBeans  Tools have good support for JUnit  Ant, Maven, CruiseControl, Hudson, Anthill  Can combine Java unit tests with XQuery unit tests  Ideal for cases where the “front-end” is Java – mimic execution environment

22. Slide 22 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. XQueryUnit  JUnit extension classes and support classes to help facilitate testing XQuery  Developed by Mark Logic Professional Services, used by numerous customers  Supports both HTTP and XDBC interfaces  Includes a number of convenience utilities:  Inserting test data, clean up at tear down  Generates wrapper code for invoking library modules  XML diff utilities for assertions  Open Source, Apache 2 license  Available for download at http://xquery-unit.googlecode.com/http://xquery-unit.googlecode.com/

23. Slide 23 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Creating a simple XQueryUnit Test Case 1.Create a new class, subclass XQueryTestCase or XQueryXmlHttpTestCase 2.Override setUp() (optional) to insert any test data 3.Implement 1 or more test methods that invoke XQuery modules/functions under test

24. Slide 24 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Testing XQuery Modules  2 types of modules:  Main  Query body  Library  No query body  Variable and function declarations  XQueryTestCase provides support for testing main and library modules, invoked via XCC

25. Slide 25 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 1 – Library Module Test public class SearchLibTest extends XQueryTestCase { private String modulePath = "/example1/search-lib.xqy"; private String moduleNamespace = "http://marklogic.com/search"; public void testGetQuery() throws Exception { XdmValue[] params = new XdmValue[] { ValueFactory.newXSString("foobar") }; Document doc = executeLibraryModuleAsDocument(modulePath, moduleNamespace, "get-query", params); assertNotNull(doc); }

26. Slide 26 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 1: search-lib.xqy xquery version "1.0-ml"; module namespace search="http://marklogic.com/search"; declare function get-query($query-str as xs:string) as schema-element(cts:query)? { if ($query-str eq "") then () else document{cts:word-query($query-str)}/node() };

27. Slide 27 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 1: Run via Eclipse

28. Slide 28 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. HTTP POX Services  POX = “Plain Old XML” – Like POJO  POX is complementary to REST  REST refers to a communication pattern, while POX refers to an information format style.  POX Services are easy to test  Browser, wget, curl, xquery-unit, JUnit, HttpUnit  XQueryXmlHttpTestCase provides support for testing Http based services, including POX http://en.wikipedia.org/wiki/Plain_Old_XML

29. Slide 29 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example POX Service  Request: cardiac  Response:...

30. Slide 30 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 3 public class SearchTest extends XQueryXmlHttpTestCase { protected void setUp() throws Exception { this.setServicePath("example3/search.xqy"); super.setUp(); } }... public void testSearch() throws Exception { String req = " "; Document doc = executeQueryAsDocument(req); XPath xpath = XPath.newInstance("/response"); Element e = (Element) xpath.selectSingleNode(doc); assertNotNull(e); } public void testInvalidRequest() throws Exception { String req = " "; Document doc = executeQueryAsDocument(req); XPath xpath = XPath.newInstance("/error"); Element e = (Element) xpath.selectSingleNode(doc); assertNotNull(e); }

31. Slide 31 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 3: search.xqy xdmp:set-response-content-type("text/xml"), try { let $request := cm:get-request-element(()) let $action := $request/element() let $check := if ($action) then () else fn:error($NO-REQ, "No request...") let $result := typeswitch ($action) case element(search) return search:search($action) default return fn:error($INV-REQ, "Invalid request...") let $complete := cm:request-complete() return $result } catch($e) { (xdmp:log(xdmp:quote($e)), 400 Bad Request {$e} ) }

32. Slide 32 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 3: Test results

33. Slide 33 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Some problems to fix The test case passed, but we have some problems 1.search.xqy does not set the http response code when an error occurs 2.We probably want an exception would be thrown for the invalid request 3.Test does not handle exceptions

34. Slide 34 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 4: Fix SearchTest public class SearchTest extends XQueryXmlHttpTestCase {... public void testInvalidRequest() throws Exception { String req = " "; int statusCode = 0; try { Document doc = executeQueryAsDocument(req); } catch (HttpResponseException expected) { statusCode = expected.getStatusCode(); } assertEquals(400, statusCode); } * We could also use JUnit4, which uses annotations for expected exceptions

35. Slide 35 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 4: Exception handling fixed

36. Slide 36 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 4: Fix response code in search.xqy try { let $request := cm:get-request-element(()) let $action := $request/element() let $check := if ($action) then () else fn:error($NO-REQ, "No request...") let $result := typeswitch ($action) case element(search) return search:search($action) default return fn:error($INV-REQ, "Invalid request...") let $complete := cm:request-complete() return $result } catch($e) { (xdmp:set-response-code(400,"Bad Request"), xdmp:log(xdmp:quote($e)), 400 Bad Request {$e} ) }

37. Slide 37 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 4: Now the test passes legitimately

38. Slide 38 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 5: Inserting Test Data  Gather 1 or more test data files (XML, text, or binary)  Place the test data file(s) in a “resource” directory under the test package  Add a setUp() method to the TestCase  Call one of the insertContent() methods in setUp() protected void setUp() throws Exception { this.setServicePath("example5/search.xqy"); super.setUp(); insertTestContent("resource/medline1.xml", "/medline1.xml"); insertTestContent("resource/medline2.xml", "/medline2.xml"); }

39. Slide 39 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 5: Testing Search public void testSearch() throws Exception { // Build request and execute String req = " cardiac "; Document doc = this.executeQueryAsDocument(req); // Use XPath to retrieve the response body, assert not null XPath xpath = XPath.newInstance("/response/s:response"); xpath.addNamespace("s", "http://marklogic.com/appservices/search"); Element response = (Element) xpath.selectSingleNode(doc); assertNotNull(response); // Use JDOM to assert the total number of results int total = response.getAttribute("total").getIntValue(); assertEquals(1, total); // Execute more XPath to assert the highlighted keyword match xpath = XPath.newInstance("s:result/s:snippet/s:match/s:highlight/text()"); xpath.addNamespace("s", "http://marklogic.com/appservices/search"); Text highlightText = (Text)xpath.selectSingleNode(response); assertEquals("cardiac", highlightText.getText()); }

40. Slide 40 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 5: Test results

41. Slide 41 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 6: Testing a Main module xquery version "1.0-ml"; declare variable $uri as xs:string external; declare variable $new-title as xs:string external; let $doc := fn:doc($uri) let $title := $doc/MedlineCitation/Article/ArticleTitle return xdmp:node-replace($title, {$new-title} )

42. Slide 42 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 6: Unit test public class UpdateTest extends XQueryTestCase { private String docUri = "/medline1.xml"; protected void setUp() throws Exception { super.setUp(); insertTestContent("resource/medline1.xml", docUri); } …

43. Slide 43 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 6: Unit test (continued) public void testUpdate() throws Exception { String newTitle = "NEW_TITLE"; XdmVariable[] variables = new XdmVariable[] { ValueFactory.newVariable(new XName("uri"), ValueFactory.newXSString(docUri)), ValueFactory.newVariable(new XName("new-title"), ValueFactory.newXSString(newTitle))}; executeMainModule("example6/update.xqy", null, variables); // Verify update by running a query String q = "fn:doc('/medline1.xml')" + "/MedlineCitation/Article/ArticleTitle/text()"; ResultSequence rs = executeQuery(q, null, null); String updatedTitle = rs.asString(); assertEquals(newTitle, updatedTitle); }

44. Slide 44 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 6 Results

45. Slide 45 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Agenda  Unit Testing Basics  Unit Testing XQuery via XQuery  Unit Testing XQuery via Java  End-to-End Testing using Selenium  Testing Tips

46. Slide 46 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 7: Simple Web Application

47. Slide 47 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Example 7: Search Results

48. Slide 48 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Launching Selenium IDE

49. Slide 49 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Selenium assertText

50. Slide 50 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Selenium: Execute Suite

51. Slide 51 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Selenium: JUnit generated source

52. Slide 52 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Agenda  Unit Testing Basics  Unit Testing XQuery via XQuery  Unit Testing XQuery via Java  End-to-End Testing using Selenium  Testing Tips

53. Slide 53 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Testing Tips  “Whenever you are tempted to type something into a print statement or a debugger expression, write it as a test instead.” 1  When to write tests:  During Development - Write the tests first, you’re done when the test passes  During Debugging – When a defect is found, write a test for that defect, debug/fix until the test passes  Find bugs once 2  First time a human tester finds a bug, it should be the last time 1 http://junit.sourceforge.net/doc/testinfected/testing.htmhttp://junit.sourceforge.net/doc/testinfected/testing.htm 2 The Pragmatic Programmer, Andy Hunt, Dave Thomas

54. Slide 54 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Summary  Developer testing should be an integral part of the development process  Incremental steps of parallel development and testing lead to a quality end product  Testing tools and frameworks can help with test development, execution, and automation  Developer testing will result in fewer application defects, higher code confidence, greater agility  Examples from this presentation available for download at http://xquery-unit.googlecode.com/ http://xquery-unit.googlecode.com/

55. Slide 55 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Thank You Mark Helmstetter MarkLogic Corporation mark.helmstetter@marklogic.com

56. Slide 56 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Questions?

57. Slide 57 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. XQuery Unit Testing Tips  Use Small datasets whenever possible

58. Slide 58 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. References  Standard glossary of terms used in Software Testing http://www.istqb.org/downloads/glossary-current.pdf http://www.istqb.org/downloads/glossary-current.pdf  http://junit.sourceforge.net/doc/testinfected/testing.htm http://junit.sourceforge.net/doc/testinfected/testing.htm  http://en.wikipedia.org/wiki/Unit_testing http://en.wikipedia.org/wiki/Unit_testing  http://blog.stevensanderson.com/2009/11/04/selective-unit- testing-costs-and-benefits/ http://blog.stevensanderson.com/2009/11/04/selective-unit- testing-costs-and-benefits/

59. Slide 59 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Misc Links  http://www.fgeorges.org/xslt/xslt-unit/ http://www.fgeorges.org/xslt/xslt-unit/  http://thewallaceline.blogspot.com/2009/04/xquery-unit- tests.html http://thewallaceline.blogspot.com/2009/04/xquery-unit- tests.html  http://en.wikibooks.org/wiki/XQuery http://en.wikibooks.org/wiki/XQuery

60. Slide 60 Copyright © 2010 MarkLogic ® Corporation. All rights reserved. Testing Advice “Everyone agrees that more testing is needed, in the same way that everyone agrees you should eat your broccoli, stop smoking, get plenty of rest, and exercise regularly.” Andy Hunt & Dave Thomas, Pragmatic Unit Testing in Java with JUnit