1. 1 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 GUI Testing Term paper in Scientific Programming

2. 2 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 2 Overview Motivation Introduction to software testing GUI representation Coverage criteria for GUI testing Optimizing the test suite Conclusion Future work

3. 3 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 3 Motivation Software testing is a proven method to increase the quality of software Software bugs can have severe consequences  Loss of money or material e.g. Ariane 5 launch  Loss of human life e.g. Toyota’s electronic throttle control system GUIs are an ubiquitous instrument for interacting with software

4. 4 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 4 Software Testing A process of executing a program to determine if it meets its specifications Categories of software testing  Correctness testing  Performance testing  Reliability testing  Security testing  Usability testing Whitebox or blackbox

5. 5 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 5 Challenges of GUI Testing Event based Sequence of events  Large number of combinations of events in the sequence  Possibly unbound

6. 6 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 6 Requirements for GUI Representation Conceptually at high level of abstraction  Independent of the platform for which the GUI is developed  Guarantees that generated testing information is portable across platforms Possible to represent a variaty of GUIs Low level details of GUIs must be captured Representation must be scalable Easy to develop and use

7. 7 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 7 GUI Representation Directed graphs Nodes represent events Edges represent transitions from one event to another

8. 8 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 8 Example GUI Xie and Memon, 2008

9. 9 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 9 Event Sequence Graph (ESG) Hierarchical structure of a GUI is contained in the graph Subgraphs are represented as a node of the graph Uses explicit entry and exit nodes [Xie and Memon, 2008] Belli et al., 2005

10. 10 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 10 Complete Event Sequence Graph (CESG) Belli et al., 2005

11. 11 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 11 Event Flow Graph (EFG) No hierarchical structure contained in the graph Uses integration tree to represent hierarchy Entry and exit nodes are implicit [Belli et al., 2011] Memon and Science 2001

12. 12 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 12 Integration Tree Gives a hierarchical overview of the GUI components of a software Each leaf invokes a GUI component or triggers an action Memon and Science, 2001

13. 13 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 13 Comparison of Graphs [Xie and Memon, 2008]

14. 14 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 14 Coverage Criteria Determine what to test  Straightforward answer: all possibilities  impractical Determine when to stop testing Set of rules that determine whether a software has been tested enough

15. 15 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 15 Intra-Component Coverage Focuses on events that are bound to one GUI component Event coverage  Each event has to be performed at least once  Checks if each event behaves as expected Event-interaction coverage  Checks the interaction between all possible pairs of events Length-n event sequence coverage  Behavior of events can change depending on previous events  Captures the contextual impact of events  There are an infinite amount of contexts  limit of the length of event sequences is defined

16. 16 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 16 Inter-Component Coverage Invocation coverage  Ensures that each component in the GUI can be invoked Invocation-termination coverage  Ensures that GUI components can be terminated Inter-component length-n event sequence coverage  Tests all event sequences that start in one component and end in another component Uses integration tree to identify interactions between components

17. 17 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 17 Metric of Coverage Intra-component coverage  Percentage of covered intra-component length-n event sequences Inter-component coverage  Percentage of covered inter-component length-n event sequences

18. 18 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 18 Why is Optimizing Necessary? Thoroughly testing can result in a large test suite Adopting the test suite to evolving software can result in redundant tests Test suites can contain many similar tests

19. 19 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 19 Efficient and Flexible GUI Test Execution via Test Merging Goal reducing the number of test Merging redundant or similar tests Constraint:  Preserves fault localization capabilities of a test suite Benefits:  Reduces number of failures from the same source  Reduces testing time

20. 20 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 20 Redundant Tests Tests are identical if they start in the same state and end in the same state A test is redundant if all entities exercised by a test case are also covered by another test case from the test suite

21. 21 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 21 Example of Similar Tests Comparing rating a book with buying a book

22. 22 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 22 Example for Similar Tests (Rating a book)

23. 23 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 23 Example for similar tests (Rating a book)

24. 24 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 24 Example for similar tests

25. 25 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 25 Example for similar tests (Merged test)

26. 26 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 26 Results of merging Empirical study with five web application Number of test steps were reduced by 29% Execution time was reduced by 39%

27. 27 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 27 Conclusion Software testing important to ensure high quality software Testing GUI is challenging because it uses events It is possible to use the ESG for the coverage criteria Using the methods described  GUIs can be modeled using one of the GUI representations  Tests can be created based on the coverage criteria  Test suites can be optimized

28. 28 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 28 Future Work

29. 29 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 29 Future Work

30. 30 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 30 Thank you for your attention. Do you have any questions?

31. 31 Visual Computing Institute | Prof. Dr. Torsten W. Kuhlen Virtual Reality & Immersive Visualization Till Petersen-Krauß | GUI Testing | 27.01.2014 31 References [Devaki et al., 2013] Devaki, P., Thummalapenta, S., Singhania, N., and Sinha, S. (2013). Efficient and flexible GUI test execution via test merrging. Proceedings of the 2013 International Symposium on Software Testing and Analysis - ISSTA 2013, page 34. [Memon et al., 2001] Memon, A. M., Soffa, M. L., and Pollack, M. E. (2001). Coverage criteria for GUI testing. ACM SIGSOFT Software Engineering Notes, 26(5):256. [Xie and Memon, 2008] Xie, Q. and Memon, A. M. (2008). Using a pilot study to derive a GUI model for automated testing. ACM Transactions on Software Engineering and Methodology, 18(2):1–35. [Belli et al., 2011] Belli, F., Beyaz, M., and Güler, N. (2011). Eventbased GUI Testing and Reliability Assessment Techniques An Experimental Insight and Preliminary Results. Software Testing, Verification and Validation Workshops (ICSTW), 2011 IEEE Fourth International Conference on, pages 212–221. [Belli et al., 2005] Belli, F., Nissanke, N., Budnik, C. J., and Mathur, A. (2005). Technical Reports and Working Papers Test Generation Using Event Sequence Graphs. (September). [Memon and Science, 2001] Memon, A. M. and Science, C. (2001). A COMPREHENSIVE FRAMEWORK FOR TESTING.