1. “An automated tool designed to ease the pain of test creation and maintenance.” Nil Weerasinghe Bryan Robbins Mohamed Ibrahim

2. American University Presentation Copyright 2011 FINRA 1 About FINRA ￭ Financial Industry Regulatory Authority Largest independent regulator for all securities firms doing business in the U.S. ~4,500 brokerage firms ~163,500 branch offices ~634,400 registered securities representatives Arial Body Copy Providing independent, vigorous regulation Educating & informing investors Inviting active industry involvement & input Actively supporting firms’ compliance efforts Our Mission: Investor Protection. Market Integrity. Computerized certification and continued education. Series 7, 63 …etc.

3. American University Presentation Copyright 2011 FINRA 2 FINRA Open Source Projects ￭ Increase Community Involvement ￭ FINRA Open Source Projects http://finraos.github.io/ ￭ DataGenerator http://finraos.github.io/DataGenerator/ ￭ JTAF-ExtWebDriver http://finraos.github.io/JTAF- ExtWebDriver/http://finraos.github.io/JTAF- ExtWebDriver/

4. American University Presentation Copyright 2011 FINRA 3 How to get involved. ￭ Use it ￭ Extend it Fork it Discuss idea –Open ticket –Google group discussion –opensource@finra.orgopensource@finra.org Commit –DCO and ApacheV2 ￭ Report bugs ￭ Help document http://finraos.github.io/DataGenerator/ https://github.com/FINRAOS/DataGenerator

5. American University Presentation Copyright 2011 FINRA 4 Agenda What is the DataGenerator? Demo. –Dependency Modeling –Pairwise Data Generation. Current Limitations. Re-architecture plan. Questions

6. American University Presentation Copyright 2011 FINRA 5 Video http://www.youtube.com/watch?v=Wxa1T0gp56k http://finraos.github.io/DataGenerator/

7. American University Presentation Copyright 2011 FINRA 6 Current Approach ￭ Two ways to describe and generate datasets Equivalence Classes + Combinations Dependency Model + Graph Coverage ￭ Both use Apache Velocity to generate output from templates DataSpec Model Datasets Outputs

8. American University Presentation Copyright 2011 FINRA Demo ￭ Pairwise Combinations Uses equivalence classes from DataSpec to populate datasets ￭ All Paths Uses annotations from graphical model to populate datasets 7 DataSpec Model

9. American University Presentation Copyright 2011 FINRA Limitations of Current Approach ￭ Limited set of graph annotations Can only set variable values within model No support for logic, pos/neg equivalence classes in current version We need more powerful annotation ￭ Logic often split across spec, model, and templates Anything dynamic must be injected into Velocity template, as model and spec are both static We need more dynamic evaluation ￭ Performance considerations Breadth-first enumeration doesn’t scale well as domain becomes more complex We need more performant implementation 8

10. American University Presentation Copyright 2011 FINRA 9 Re-architecting Data Generator ￭ Replacing Visio with SCXML, an open standard to represent the state machine. <scxml xmlns="http://www.w3.org/2005/07/scxml" xmlns:cs="http://commons.apache.org/scxml" version="1.0" initial="start">.

11. American University Presentation Copyright 2011 FINRA 10 Re-architecting Data Generator ￭ SCXML Allows for complex modelling using embedded EL <transition event="OPTIONS_SYMBOLOGY_IDENTIFIER" target="OPTIONS_SYMBOLOGY_IDENTIFIER" cond="${var_out_PRODUCT_TYPE_CODE=='Derivatives-Options'}" /> <transition event="OPTIONAL_SECURITY_SYMBOL" target="OPTIONAL_SECURITY_SYMBOL" cond="${var_out_PRODUCT_TYPE_CODE!='Derivatives-Options'}" />.

12. American University Presentation Copyright 2011 FINRA 11 Re-architecting Data Generator ￭ SCXML Allows for complex modelling: A state can be written as a state machine itself ￭ We’re using apache commons-scxml in out POC

13. American University Presentation Copyright 2011 FINRA 12 Re-architecting Data Generator ￭ Overcoming memory issues by enhancing the all-paths algorithm, use DFS with minimal memory overhead

14. American University Presentation Copyright 2011 FINRA 13 Re-architecting Data Generator ￭ Short demo: http://www.w3.org/2005/07/scxmlhttp://commons.apache.org/scxml

15. American University Presentation Copyright 2011 FINRA 14 Re-architecting Data Generator ￭ Restructure the code to allow Hadoop Map Reduce and Giraph to operate on it. ￭ Data Generator won’t itself directly depend on Hadoop or Girpah, but will abstract the following: Input: Allow input from files Execution: Allow the execution from a middle state provided input variables Output: Allow outputs to different formats text files, several files, gz. The user will be able to extend the output to support: sequence files, redshift, hbase

16. American University Presentation Copyright 2011 FINRA 15 Re-architecting Data Generator