1. Model-Based Testing with Smartesting Jean-Pierre Schoch Sogetis Second Testing Academy 29 April 2009

2. Agenda Model-Based Testing and Smartesting A Technical Overview Benefits and Conclusion

3. Model-Based Testing: What and Why? The problems of software testing o Time is limited o Applications can be complex o Requirements are changing Limitations of manual testing o Time-Consuming o Can rarely go deep enough Limitations of test automation o Test scripts may be challenging to design o Maintenance can be difficult and costly Model-Based Testing (MBT) o Model-based testing is software testing in which test cases are derived in whole or in part from a model that describes some (usually functional) aspects of the system under test (SUT). [Wikipedia] ModelSystem Abstract tests Executable tests can be run against are derived from is a partial description of are abstract versions of

4. Model-Based Testing with Smartesting Smartesting provides: o A Model-Based Testing solution for functional testing o … For the automatic generation of test cases o … From a UML modeling of the requirements But who are we … ? o Independent Software Vendor, founded in 2003, privately held o Spin-off of a Computer Science Lab in France (CNRS / INRIA) o 35 people, R&D Center in Besançon, offices in Paris, Amsterdam and Bangalore o Several innovation awards o Supported by venture capital

5. The Smartesting Solution Modeling requirements to automatically generate test cases Functional testing Integrates with existing tools (requirement management, test management and execution, etc.) Eclipse-based Test Execution Test Management Smartesting Automated Test Design Business Needs Analysis

6. Agenda Model-Based Testing and Smartesting A Technical Overview Benefits and Conclusion

7. Smartesting Center Test Generation Process ITERATIVE PROCESS Manual Test Test Automation 7 Test Automation Engineer Testers Test Analyst Requirement Links Test scenarios Test scenarios Adaptation Layer (Keyword Library) Requirement Links GeneratedTests Business Needs Test Designer Business Analyst Testresults Test Model Repository Test Requirements Third party environment Test repository

8. Test Model and UML Diagrams Class diagrams o To represent the application to test (system under test) o To represent the business entities associated with the application State Machine diagrams o To model the dynamic behavior of the application Object (or instance) diagrams o To model test data (in a generic way) And also … o Graphical constructs are not enough o Business rules are captured using OCL (Object Constraint Lanquage)

9. Class and State Machine Diagrams: An Example 9 Class diagrams capture the definition of the system under test (not shown here) and the business entities of the application UML constraints (and class structures) capture business rules State machine diagrams capture the behavior of the system This constraint is used to evaluate the result of the submit transition (static) (dynamic)

10. The Reference Application

11. Modeling with Smartesting

12. Generating Test Cases

13. Publishing Test Cases to Test Management Environment HP Quality Center

14. Agenda Model-Based Testing and Smartesting A Technical Overview Benefits and Conclusion

15. Requirement Validation and Early Test Generation Test Model = Model of the requirements o Detailed level o Independent of code o Can (should?) exist before code is written o Test cases derived from the model Unreachable test cases may be an indication of requirement incompatibilities, errors, etc. Changes to the state of the system can be observed step by step Tool provides a simulation mode at the model level Benefits o Requirements validated before code is written o Facilitates communication with business experts o Can be used as an input to development

16. Requirement Traceability and Test Coverage Test Cases published to the Test Management tool with requirement links o Test model marked with requirement ID tags o Synchronization between generated test cases and requirements entirely automatic No manual, error-prone intervention o Publication will update the test status for modified test cases o Leverages the analysis and reporting capabilities in the test management tool Benefits o Reliable traceability links and test coverage information o Guarantee that the generated test cases are relevant

17. Improved Productivity and Maintenance Normal benefits from automation o Significant reduction on execution time In particular during regression testing o Effectiveness of automation vs. manual approach Separation of concerns o Business logic (captured in test model) separated from implementation technology (testing environment and adaptation layer) Impact of functional changes o Changes to the modeled requirements automatically applied to test cases The number of affected test cases has no consequence on schedule Impact of implementation changes o No effect on the test model For instance, the look and feel can evolve separately Significantly facilitates application migration

18. Smartesting and SOGETI Were here at the Software Testing Academy SOGETI NL o Proof of Concept (PoC) the Solution and Innovation Department Now looking for a first project Goal: To implement Smartesting as a standard tool for the STAAS test line o Smartesting initiative to document integration of Smartesting with TMap SOGETI France o Contribution of François Darphin, Testing National Practice Manager SOGETI France, to the book Industrialiser le test fonctionnel, by Bruno Legeard, Smartesting CTO o Business development in progress (identification of first project) o Participation of SOGETI France to a Smartesting customer event coming soon… And also with Capgemini o PoC realized for Capgemini France in Bordeaux o Business development in progress (identification of first project)

19. Practical Model-based Testing – December 2006 By Bruno Legeard, CTO Smartesting Smartesting Case Study – November 2008 Industrialiser le test fonctionnel – February 2009 By Bruno Legeard, CTO Smartesting Soon in English…

20. Sogetis 2 nd Testing Academy 29 April 2009 www.sogeti.com