1. Design Driven Cost Estimation via Design Driven Testing Doug Rosenberg Copyright © 2011 ICONIX

2. How can we count lines of code before coding? asked Alice Copyright © 2011 ICONIX

3. CoCoMo meets DDT Copyright © 2011 ICONIX 19872011

4. ICONIX was heavily involved with CoCoMo from the mid- 80s to the mid-90s We sold a CoCoMo tool for about 10 years What did we learn? A) Estimation is a lot easier when you know how many lines of code you will be building B) It isnt always easy to know how many LOC youll have up front Our CoCoPro tool calibrated estimations to a history file of past projects, to converge to better estimation results over time Copyright © 2011 ICONIX

5. We know how to count code after its written Copyright © 2011 ICONIX Q: how can we get a line of code count before coding? A: Count lines of unit test code, generated from model

6. Cost estimation via testing? Isnt that backwards??? In todays agile universe, tests are sometimes written before code (Test Driven Development) With Design Driven Testing we generate unit test code from UML/SysML models before code is written We can line-count this unit test code and use it as a basis for estimation Copyright © 2011 ICONIX

7. DDT flips agile test driven development around Thats good because driving design from unit tests is 180 degrees backwards Its better to drive testing from design Cost estimation can be driven in the same way

8. Copyright © 2011 ICONIX DDT addresses unit testing by developers and acceptance testing by QA folks Well focus on unit testing in this talk DDT automates generation of skeleton unit test code from UML models This unit test code can be line-counted and used as a basis for LOC estimates

9. Copyright © 2011 ICONIX Unit tests are auto-generated from sequence diagrams This is good because we can count lines of unit test code and use it as a basis for estimation, before code is written How does it work?

10. Copyright © 2011 ICONIX 1. Generate a test case for each message on a sequence diagram

11. Copyright © 2011 ICONIX 2. Define test scenarios for each test case Test with good data, test with bad data, etc. Define as many scenarios as are needed to fully exercise the operation being tested More complex operations will have more test scenarios

12. Copyright © 2011 ICONIX 3. Transform test cases to unit test classes A test method is created for each test scenario These test methods can be code generated

13. Copyright © 2011 ICONIX Transforms are instantaneous setUp and tearDown methods are also generated, in a addition to a method per test scenario Automation is key to the whole DDT approach because its too much work if its not automated

14. Copyright © 2011 ICONIX 4. Test class -> test code Unit test code is generated automatically The amount of code is proportional to the complexity of the operation being tested This unit test code can be line-counted and used as a basis for COCOMO estimation NOTE: Not a 1:1 ratio between unit test code and production code

15. Copyright © 2011 ICONIX We have an early-lifecycle line of code count (before coding begins) This line of code count is directly generated from the UML or SysML model, and is directly proportional to the complexity of the code that will be written, because more complex methods will have more test scenarios No handwaving

16. Copyright © 2011 ICONIX Oh, BTW…No gaps in test coverage Minor side benefits of DDT include better designs and better test coverage

17. Copyright © 2011 ICONIX What if we could start from an earlier lifecycle artifact than sequence diagrams? Sequence diagrams represent a very detailed level of design. We probably need to estimate earlier in the lifecycle. We can generate test code from conceptual designs, as soon as use cases are defined

18. Copyright © 2011 ICONIX Controllers are testable units of system behavior We identify logical software functions on robustness diagrams as a disambiguation device when writing use cases Each controller may expand to several messages on a sequence diagram Fewer tests are required Controller tests can be code generated just like sequence message tests

19. Copyright © 2011 ICONIX Breadth-first test coverage across a scenario And an earlier-lifecycle artifact to drive test code from This can provide a first guess estimate Re-estimation during detailed design will provide more accurate estimates

20. Copyright © 2011 ICONIX Add test scenarios for each test case as before The conceptual design will not include theinfrastructure orscaffolding code, but will reflect the core functionality that needs to be built This is OK for a first- pass estimate

21. Copyright © 2011 ICONIX Transform to classes and generate code Since robustness diagrams are drawn just after use cases are written, we have a line-countable code base very early in the project lifecycle The earlier we can estimate accurately, the better we can plan budgets and schedules

22. What about hardware/software systems? Does all this still work if were modeling an embedded system using SysML? Yep. Copyright © 2011 ICONIX

23. DDT/Systems extends DDT/Software for SysML DDT/Software is a great place to start system testing, because most systems are software intensive DDT/Software (DDT) includes both Developer Testing and QA/Acceptance Testing DDT Scenario Testing applies directly to system of systems testing Copyright © 2011 ICONIX

24. What is DDT/Systems? A comprehensive approach to validation and verification for systems and system-of-systems A systematic way to drive V&V from SysML models An extension of Design Driven Testing for software to include full HW/SW system V&V Copyright © 2011 ICONIX

25. DDT/Systems extends DDT for the full range of SysML models, including interface and state testing Copyright © 2011 ICONIX

26. SysML Example Model: Pothole detecting spacecraft Orbits the earth Takes pictures of city streets Downlinks images to ground station Detects potholes via image analysis Emails images with annotated potholes to subscribers (Thanks to Dr. Mike Sievers of NASA/JPL for his help with the POTHOLE example, and with defining DDT/Systems) Copyright © 2011 ICONIX

27. How do we V&V our spacecraft from this SysML model? Generate tests from requirements Generate interface tests from ports Generate scenario tests from use cases Generate state machine tests from triggers, do/entry/exit behaviors Generate unit tests from sequence messages Generate unit tests from class operations Generate parametric tests Copyright © 2011 ICONIX

28. Well have one sequence diagram per use case Copyright © 2011 ICONIX

29. Sequence diagrams show all system behavior within each use case Copyright © 2011 ICONIX

30. These test cases are transformed into unit test code and can be an input to CoCoMo cost models Copyright © 2011 ICONIX

31. State machines include triggers, and entry/exit/do behaviors Thanks to Dr. Mike Sievers of NASA/JPL for assistance Copyright © 2011 ICONIX

32. We can generate all of these behavior tests automatically from a SysML model Thanks to Dr. Mike Sievers of NASA/JPL for assistance Copyright © 2011 ICONIX

33. Interface tests are generated from Ports Thanks to Dr. Mike Sievers of NASA/JPL for assistance Copyright © 2011 ICONIX

34. Interface Tests for Non Volatile Memory include testing the Buffer Control Interface, Instrument Data Input, and Xband Output Thanks to Dr. Mike Sievers of NASA/JPL for assistance Copyright © 2011 ICONIX

35. Current Status DDT is described in Design Driven Testing (Apress 2010) DDT/Systems is described in ICONIX Process Roadmaps (Fingerpress 2011) DDT/Software currently automated in Sparx Enterprise Architect; DDT/Systems extensions not yet released MagicDraw plug-in currently under development, scheduled release 4Q 2011 Copyright © 2011 ICONIX

36. Summing up Early lifecycle line of code estimates are generally beneficial for cost estimation Design Driven Testing provides a basis for early LOC estimates from conceptual and detailed design views of a use case DDT/Systems extends this to cover state and interface testing from SysML models LOC estimates are generated directly from the design models, thus all handwaving is eliminated WAGs become substantially more scientific (SWAGs are better than WAGs)

37. Copyright © 2011 ICONIX Questions? doug@iconixsw.com umltraining@iconixsw.com sysmltraining@iconixsw.com

38. Copyright © 2011 ICONIX Thank You