1. Generating Software Documentation in Use Case Maps from Filtered Execution Traces Edna Braun, Daniel Amyot, Timothy Lethbridge University of Ottawa, Canada damyot@uottawa.ca SDL 2015, Berlin, October 14

2. SDL 2015 – p. 2 Motivation Maintenance and evolution of large legacy software User-friendly, automated software comprehension Focus on the elimination of utilities from long execution traces, with documentation and visualization based on Use Case Maps

3. SDL 2015 – p. 3 Program Comprehension Static Analysis –Based on code structure –Recovery of packages, classes, components, attributes, associations and other relationships… Dynamic Analysis –Based on traces and/or other behavioral information –Recovery of invocation sequences, scenarios, state machines… Hybrid Approaches –Combine (parts of) static and dynamic analyses

4. SDL 2015 – p. 4 Context For most software applications, dynamic analysis based on execution traces is difficult –Very, very lengthy traces, even for simple features Several approaches for component filtering and for trace compression exist –E.g., filtering out simple getters and setters in Java –But traces are still large –Need to use many approaches in practice Different visualizations exist –MSC / UML sequence diagrams, (business) processes…

5. SDL 2015 – p. 5 Trace Example (Extract)

6. SDL 2015 – p. 6 Problem and Approach Can we detect and filter out utilities in software in order to shorten traces to be visualized and used for program comprehension, while preserving the traces’ meaning? Exploration of hybrid approaches –Different algorithms for utility detection, based on static and dynamic information –Use of execution traces for dynamic analysis and recovery of high-level behaviour –Visualization with Use Case Maps –Experiments with OO programs in Java

7. SDL 2015 – p. 7 Process Overview Application + Source Code Generate Long Trace Filter Short Trace Visualize UCM Adjust Filter and Visualize Help Comprehend UCM

8. SDL 2015 – p. 8 Definitions A utility is any element of a program designed for the convenience of the designer and implementer and intended to be accessed from multiple places within a certain scope of the program Utilityhood is a metric defined as the extent to which a particular method can be considered a utility. Calculated with dynamic and static variables When the utilityhood is high, remove from the trace Non-utility methods help describe features and business logic

9. SDL 2015 – p. 9 TracetoUCM Tool

10. SDL 2015 – p. 10

11. SDL 2015 – p. 11 Static Metrics Collected with Metrics 1.3.6 –Method Lines of Code (MLOC) –Nested Block Depth (NBD) –McCabe Cyclomatic Complexity (McCabe)

12. SDL 2015 – p. 12 Dynamic Metrics Collected with Eclipse TPTP tracer and a Java profiler –UniqueFanout (array): Unique set of methods that this method called. The length of the array is the fan-out. –UniqueFanin (array), Fan-in, and Fan-out –NumberOfTimesCalledBy (array) –NumberOfTimesMethodsCalled (array) –TotalSegmentPresence –Depth –Method BaseTime, CumulativeCpuTime, CpuUsageAverage –…

13. SDL 2015 – p. 13

14. SDL 2015 – p. 14 Utilityhood Metrics Combine basic static/dynamic metrics collected from software code and execution traces Combinations based on preliminary experiments and on related work –Tested on 18 simple Java open source systems –Kept the most promising combinations for further experiments U(r): utilityhood for method r used in a trace RelativeMethodSize(r): size of a method r in relation to other methods in the system N: number of unique methods in the execution trace

15. SDL 2015 – p. 15

16. SDL 2015 – p. 16

17. SDL 2015 – p. 17 Use Case Maps Part of the User Requirements Notation (Rec. Z.151) Combine structure with scenario-oriented behaviour High level of abstraction, above that of messages

18. SDL 2015 – p. 18 Trace  UCM Mapping [Hamou-Lhadj et al., 2005]

19. SDL 2015 – p. 19 Example: UCEd Tool Use Case Editor, version 1.6.2 (Somé, 2007) 700 Java classes, 3284 methods, 33,004 lines of code Feature explored: “Open a project” Trace size: 19,543 method invocations

20. SDL 2015 – p. 20

21. SDL 2015 – p. 21 Example UCM Algorithm 4 40 top methods preserved Needs another iteration, with better parameters...

22. SDL 2015 – p. 22 Same initial trace Algorithm 4 20 top methods preserved Use of UCM stubs for hierarchical decomposition

23. SDL 2015 – p. 23 Validation with Experts SoftwareClassesMethodsLOC UCEd700328433,004 Umple430445779,459 jUCMNav851740693,919 TraceToUCM35963441

24. SDL 2015 – p. 24 Results of Utility Selection Algorithms “Are the methods selected by this algorithm correct?” Algorithms 2 (Hamou-Lhadj) and 4 (Braun) ahead of the pack

25. SDL 2015 – p. 25 “Assuming the correct methods had been selected, would you use UCM diagrams to document the behavior of a feature in your design documents?” Results of Visualization

26. SDL 2015 – p. 26 Documentation Experiment Traced our own TraceToUCM tool during the analysis of the HotDraw software application 12 grad students and 4 professional developers Showed UCMs of traces; some were very (too) short “Pick the top three UCM diagrams that you fell gives enough information to describe what the program does” Newer algorithms scored best, but…

27. SDL 2015 – p. 27 Related Work and Threats to Validity Tons of them! See paper and Braun’s thesis

28. SDL 2015 – p. 28 Conclusions TraceToUCM: an environment where different algorithms targeting the filtering of utilities (exploiting dynamic, static, and CPU information) can be implemented for comparison and evaluation Visualization of traces with UCM (and indirectly MSC) Comparison of 8 utilityhood functions, 4 based on existing work and 4 new ones –Usually better results with some new functions –But: still much room for improvement

29. SDL 2015 – p. 29 Future Work Re-run study with other traces, software, parameter configurations and OO languages, with more subjects Improve UCM generation with better auto-layout and multiple layers of stubs Investigate whether utilities change from one feature to the next Extend the study to handle multi-threaded applications