1. TTCN-3 Test Case Generation from arbitrary traces Capture & Replay Bogdan Stanca-Kaposta & Theofanis Vassiliou-Gioles (Testing Technologies)

2. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © Agenda The Motivation The Technical Challenges Capture & Replay for TTworkbench Demo Q & A 2

3. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © Motivation(s) TTCN-3 Users We have a TTCN-3 framework, where do the test cases come from? Issues Manual test case creation tedious task Error prone Domain Knowledge required Non-TTCN-3 Users We see a scenario and we would like to replay it later Issues What means replay w.r.t. testing? What is the test system? What is the system under test? 3 The desire to capture observed communication in test scripts suitable for test automation

4. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © The Idea 4

5. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © What is a Trace? 5

6. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © What is a Trace? The observed communication is captured (stored) in a trace (file) Trace contains Communication between two or more entities Message exchange One or more protocols 6

7. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © A Generic Communication Setup 7 A1 A2 B1 B2............

8. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © The Setup But Who is Who? A1 B1 A1 SUT B1 IOP1 IOP2 SIM1 SIM2

9. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © The Setup From Message to Templates A1 B1 INVITE sip:dbo@192.168.89.76 SIP/2.0 Call-ID: 6929@127.0.0.1 Contact: Content-Disposition: session Content-Length: 150 Content-Type: application/sdp CSeq: 2 INVITE From: "TESTER1" ;tag=1169 Max-Forwards: 70 To: Via: SIP/2.0/UDP 127.0.0.1:5061;branch=z9hG4bK5704... template INVITE_Request T1 := { requestLine := { method := “ INVITE”, requestUri := “ sip:dbo@192.168.89.76”, sipVersion := “ SIP/2.0” }, msgHeader := { callId := “ 6929@127.0.0.1”, contact := …, cSeq := “ 2 INVITE”, fromField := "TESTER1 …", toField :=, via := …., maxForwards := { fieldName := MAX_FORWARDS_E, forwards := 70 },.... allow := omit,...

10. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © The Technical Challenge It is somehow easy to generate TTCN-3 templates from each message, but the challenge is to obtain usable and maintainable test data and test scripts Variable information From run to run, perhaps unpredictable From message to message, perhaps constant perhaps dependend Variable but constant information Should be configurable before execution White noise Lot of information that is potentially irrelevant for a particular test scenario but inevitable for the test execution 10

11. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © The Technical Challenge It is somehow easy to generate TTCN-3 templates from each message, but the challenge is to obtain usable and maintainable test data and test scripts Variable information From run to run, perhaps unpredictable From message to message, perhaps constant perhaps dependend Variable but constant information Should be configurable before execution White noise Lot of information that is potentially irrelevant for a particular test scenario but inevitable for the test execution 11 Protocol and/or domain knowledge has to be included!

12. The Solution Automated TTCN-3 Test Generation and Execution 12

13. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © Example Scenario 13 SIP User A calls User B User B accept call

14. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © Example Scenario 14 SIP Use a common Monitor Tool to capture messages e.g. Wireshark Trace File

15. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © Example Scenario 15 Trace FileTTworkbench Execution Put the Trace File into TTworkbench …. Generate the captured Test Scenarios… Replay your previously captured Test Scenarios full automatically

16. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © Example Scenario 16 Trace FileTTworkbench Execution Put the Trace File into TTworkbench …. Generate the captured Test Scenarios… Replay your previously captured Test Scenarios full automatically

17. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © Know-How Integration Manual definitions in a configuration file, to Extract existing data and use the same values in the message flow (sequence numbers, call IDs, …) Define uninteresting fields for ignoring at matching time Automatically extract similar data from templates to create “better” templates and structures Reuse existing Type systems and codecs Test adapters and component definitions Test functions and Alt-Steps

18. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © DEMO! 18

19. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © Key Benefits Speed up functional and interoperability testing By reducing manual task Automate testing Generate your test, instead of implementing it manually Execute them full automatically Get your detailed test reports Quality Optimization Test case analyzer reduces errors in test cases Fast reusability and easy extension of already defined tests 19

20. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © Supported Capturing Sources & Requirements Wireshark The most popular network analysis and capturing tool Jpcap Service The remote capturing service provides capturing on remote machines Pcap Files Trace files created beforehand TTworkbench logs (TLZ) Analyze the logs without an SUT Capturing Console Input and Output Capturing of data via serial connections (RS232, USB) SSH, TELNET Local Console Windows WinPCAP Linux/Unix Libpcap at least version 1.0 Recommendation: Install Wireshark Includes the installation of WinPCAP and LibPCAP 20

21. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © Further information Testing Technologies testingtech.com facebook.com -> Testing-Technologies Capture & Replay testingtech.com/products/ttworkbench.php testingtech.com/download/datasheets/CaptureAndReplay.pd f 21

22. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © Contact 22 Bogdan Stanca-Kaposta Testing Technologies stanca@testingtech.com Theo Vassiliou Testing Technologies vassiliou@testingtech.com

23. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © The process 23

24. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © Features Designed to generate test cases from existing traces Start your own capture with the Capture & Replay tool Or import trace files created beforehand with other capturing tools, e.g. Wireshark Capture from multiple sources at the same time 24

25. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © Benefits Fast and easy test authoring No programming skills required Support of multiple capturing sources Post-hoc analysis of real world situations via simulation mode Flexible creation of test configurations out of selected scenarios Decide which communication peers shall be tested Decide which ones shall be simulated 25

26. Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com. © On the second thought... User: Wait! How about the variable data? The Manager: Sell consultancy $$$ Technical guy: Let‘s integrate the protocol knowledge Extract existing data and use the same values in the message flow (e.g. sequence numbers, call IDs) Ignore the uninteresting fields for matching Extract similar data from templates User: I would like to reuse my Test Structure The Manager: Reuse existing Adaptation, Codecs, Type systems, Components, Functions, Alt Steps 26