1. Keynote SIGOS GmbH Klingenhofstrasse 50d 90411 Nuremberg Germany www.keynote-sigos.com SIGOS Integrated Test Environment SITE 2.3 Advanced User Training June, 2010

2. ©2010 Keynote SIGOS GmbH Testing is our Competence Basic User Training You should have passed the Basic User Training SITE GUI Usage of test editor Basic understanding of control structures Scheduling Different test scenarios (IP, Voice, Messaging) 2

3. ©2010 Keynote SIGOS GmbH Testing is our Competence Repetition 3 Brief Repetition of Base User Training

4. ©2010 Keynote SIGOS GmbH Testing is our Competence SITE Concept Permanent Supervision of the Services  All operator services  Convenient handling  Quick results  With precise measurements  High reliability  Always  Over all interfaces  Everywhere 4 SITE System SITE solution Real end-to-end testing, simulating the mobile subscriber's behaviour

5. ©2010 Keynote SIGOS GmbH Testing is our Competence SITE Concept 5 All services GSM A S I T E T E S T S Y S T E M Video Tel. WAP Streaming Browsing IP (CS/PS) Video Quality BlackBerry MMS SMS IVR Voice Quality Basic Call Suppl. Services Push-to-Talk VoIP 3G Layer 3 Tracing WLAN Handset Simulation Handset Simulation Further Technologies GSM Gb IN/CAMEL SS7 Protocol IuCS / IuPS Core Netw. ISUP MAP (HLR/VLR) Value Added Services (VAS) Revenue Assur. Inbound Roaming Outbound Roam. GlobalRoamer DATA Messaging SPEECH Network Services SYNERGIES Prepaid Recharging Drive Service Test DSL iphone Digital Rights Mgmt

6. ©2010 Keynote SIGOS GmbH Testing is our Competence SITE Concept  Graphical User Interface  GUI enables user friendly system operation and administration.  The WEB Interface provides an easy access to SITE via internet.  XML interface as integration interface for SITE, e.g., XML Push URL  Test Case Setup  Large libraries of test cases  Tree-oriented definition and scheduling of test jobs  convenient drag and drop functionality to arrange, copy and edit test elements 6 Convenient handling

7. ©2010 Keynote SIGOS GmbH Testing is our Competence SITE Concept 7 Flexible charts and diagrams can be immediately generated from the test results. A highly flexible service alarm generator is available. Online Reporting Measurement Strategy Test Execution Test execution on different locations Post Reporting e-mail dispatch real time reporting alarming Traces KPIs Quick results

8. ©2010 Keynote SIGOS GmbH Testing is our Competence SITE Concept 8 M2M1 Location Update Duration KPI Database Precise measurements, high reliability Reporting

9. ©2010 Keynote SIGOS GmbH Testing is our Competence SITE Concept  Automatically testing  Day and night, weekends,…  Avoids time consuming and work intensive manual tests  Scheduling  Administration of test orders and execution schedules  Intelligent test job management for optimum use of resources  Setting of priority levels 9 Always

10. ©2010 Keynote SIGOS GmbH Testing is our Competence SITE Concept 10 Interfaces: UMTS, HSDPA/HSUPA, WLAN, VoIP, GSM/GPRS/EDGE, SS7, ISDN, PSTN, ISUP, INAP, TCAP*, CAP*, MAP*, SCCP* * These test interfaces are implemented in SITE but are offered as customized solutions. Over all interfaces

11. ©2010 Keynote SIGOS GmbH Testing is our Competence SITE Concept 11 LAN/WAN SIM Multiplexer Central Unit Central Unit Extension Clients Mobile LUs Wireless LUs unlimited number of Local Units can be placed on geographically different locations Local Units (LU)‏ Compact LU all LUs with different UMTS, GSM/GPRS, ISDN, PSTN test probes and GSM A, ISUP interfaces Data Bases Everywhere

12. ©2010 Keynote SIGOS GmbH Testing is our Competence SITE Concept 12 SITE architecture Clients Central Unit Central Unit Extension SIM Multiplexer Local Unit (LU)‏ Compact LU Wireless LU Mobile LU Data Bases User Interface XML O&M Manager Scheduler Resource Manager Test Execution Manager Central Unit Data Base Execution Data Base KPI Data Base Reporting Data Base Selfmgmt Data Base Remote SIM Multiplexing Test ExecutionTest Interfaces

13. ©2010 Keynote SIGOS GmbH Testing is our Competence SITE Concept 13 SITE SIM Multiplexing supporting thousands of SIM cards serving U m, U u - and SS7 Interfaces (A,...)‏ LAN/WAN Central Unit SIM Multiplexer Local Units UMTS Probe GSM/GPRS Probe Compact Local Units SIM card Selection 1. Virtual SIM Card Transfer 2. Test Case Resource SIM Location 2. A Interface selection

14. ©2010 Keynote SIGOS GmbH Testing is our Competence Software Overview 14

15. ©2010 Keynote SIGOS GmbH Testing is our Competence 15 Graphical User Interface WebUI Login Dialog window is started via: http://sitecu/site/gui http://sitecu/site/gui and a click on “Start”  test definitions, test executions and test analysis are completely controlled by the WebUI

16. ©2010 Keynote SIGOS GmbH Testing is our Competence 16 Graphical User Interface Navigation Area Overview Area Details Area Menu Bar Status Bar

17. ©2010 Keynote SIGOS GmbH Testing is our Competence 17 Task Based GUI: Test Test Analysis The first view is to analyze the test results. What happened during test execution? What tests are running PASS, FAIL or INCONCLUSIVE? Which orders are running and how looks the trace like? The Test Analysis view provides the test definition tree in the Navigation Area, the Test Case Status in the Overview. Two tabs are in the Details Area: the trace and the Orders of the selected test case.

18. ©2010 Keynote SIGOS GmbH Testing is our Competence 18 Task Based GUI: Test Test Administration In the view Test Administration the tests can be parameterized and scheduled. Sequences and Control Structures can be filled with test cases. In the Navigation Area the Toolbox for Application provides the test cases for the Test Editor in the Overview. The Test Editor itself is divided into the structure and the parameter section. The Details Area is showing the TestcaseStatus and the Orders for the selected test case.

19. ©2010 Keynote SIGOS GmbH Testing is our Competence 19 Task Based GUI: Test Test Design The third view is to design tests. Just with drag and drop of nodes into the structure tree specialized tests can be build. The Toolbox for Editing Test Cases on the left side and the Test Case Editor showing the structure tree in the Overview Area. The Details Area is showing the parameter section of the TestcaseEditor.

20. ©2010 Keynote SIGOS GmbH Testing is our Competence 20 Task Based GUI: Reporting and GIS Reports The Reports view is for analyzing. Each report can be viewed and opened in an extra window. The drill down or derive features can be accessed here. The Navigation Area contains two trees: The Report Definition tree and the Alarm Status tree. If a Report is selected, the Overview is showing the report. Is an Alarm selected, the Alarm History Report is shown. The Details Area is showing the AlarmStatus for the Reports.

21. ©2010 Keynote SIGOS GmbH Testing is our Competence 21 Task Based GUI: Reporting and GIS Report Administration In the Report Administration view the reports are build. For more usability the current selected report is shown also. So every saved change can be verified directly. On the left the Toolbox for Reports is shown. The Overview Area contains the structure tree of the reports and the parameter section. The Details Area is showing the current report. The Report Status and the Alarm Status can be accessed in two additional tabs in the Details Area.

22. ©2010 Keynote SIGOS GmbH Testing is our Competence 22 Task Based GUI: Reporting and GIS GIS The GIS view is showing all data for the Drive Tests. The single tests can be accessed via tooltip and Details Area. In the Navigation Area a special tree for GIS Maps is located, where the user can select the layers to be shown on the map. The small map is for orientation. In the Overview Area the zoomed sector of the map is shown with the layers the user has selected. The Parameter Area is showing the Testcase Status and the trace of the selected test case in two tabs.

23. ©2010 Keynote SIGOS GmbH Testing is our Competence 23 Task Based GUI: System related System Health With the System Health view a quick overview about the Hardware of the CU and SITE is introduced. Additional to the Selfmanagement Reports, the current workload is monitored. The maintenance of the Units and the Users can be accessed here. The tree of System Units or the tree of users are shown in the Navigation Area. In the Overview Area some meters are giving the overall status of the system. Beneath the meters the Units table respective the User Profiles are shown in the Details Area.

24. ©2010 Keynote SIGOS GmbH Testing is our Competence 24 Task Based GUI: System related Databases Select the database table in the Navigation Area. The table is shown in the Overview Area. Due to the different number of columns, in the Database view is no Details Area but the table entry is shown in an extra dialog. An additional tab with another table can be opened and closed via the well known buttons. Otherwise the table will be opened in the same tab.

25. ©2010 Keynote SIGOS GmbH Testing is our Competence Advanced Test Administration 25 Advanced Usage of Control Structures

26. ©2010 Keynote SIGOS GmbH Testing is our Competence Control Structures FOREACH Intelligent test definitions and test setup CSV as configuration file Easy to maintain / modify One test defintion  Hundreds of different test scenarios 26

27. ©2010 Keynote SIGOS GmbH Testing is our Competence FOREACH and CSV 27

28. ©2010 Keynote SIGOS GmbH Testing is our Competence FOREACH and CSV 28 Configuration within the CSV file Copy & Paste the CSV file into the CSV_Text parameter

29. ©2010 Keynote SIGOS GmbH Testing is our Competence Case Study Starting Point SIMs from 3 different operators Services to test: Voice, SMS, MMS, FTP, WAP 10 different locations Voice, SMS and MMS from each location to each other location Inter operational (also between operators 1, 2 and 3) Without FOREACH: 3 (Operators) * 2 (FTP, WAP) * 10 (Locations) = 60 test definitions PLUS 3 (Operators) * 3 (Voice, SMS, MMS) * 10 * 10 (Locations) = 900 test definitions  960 test definitions (intra operational) 900 * 3 + 60 = 2760 test definitions (between all operators ) 29 With FOREACH: Only 2 test definitions One for Voice, SMS and MMS, One for FTP and WAP

30. ©2010 Keynote SIGOS GmbH Testing is our Competence Approach 30

31. ©2010 Keynote SIGOS GmbH Testing is our Competence Approach 31 Definition of VARIABLES and VALUES within FOREACH (‚CSV_Text ‚parameter) Usage of VARIABLES within Sequence and Test Definition

32. ©2010 Keynote SIGOS GmbH Testing is our Competence VARIABLES in Sequence Level 32 A-Side and B-Side have to be configured (Voice, SMS, MMS) VARIABLES for Location, SIM and Roaming to (if needed, also for Number) VARIABLE for Type can also be used (allows switching between 2G and 3G tests)

33. ©2010 Keynote SIGOS GmbH Testing is our Competence Definition of VARIABLES Definition of VARIABLE and corresponding VALUES in each column E.g. Column A: Name of VARIABLE=Technology_A, VALUES=GSM_Um 33 VARIABLE Names VARIABLE Values

34. ©2010 Keynote SIGOS GmbH Testing is our Competence Usage of VARIABLES Once the VARIABLES are defined, they can be referenced within the corresponding test case parameters 34

35. ©2010 Keynote SIGOS GmbH Testing is our Competence In the same way, all VARIABLES for the B-Side have to be defined and referenced 35 Usage of VARIABLES

36. ©2010 Keynote SIGOS GmbH Testing is our Competence Usage of CSV REMARK: Usage of a nested FOREACH structure CSV file usage for A-Side in parent FOREACH CSV file usage for B-Side in nested FOREACH 36

37. ©2010 Keynote SIGOS GmbH Testing is our Competence Usage of CSV 37 FOREACH VALUE in Parent with EACH VALUE in nested structure Location_A=Austria* - Location_B=Austria* Location_A=Austria* - Location_B=Belgium* Location_A=Austria* - Location_B=Croatia* ….. Location_A=Belgium* - Location_B=Austria* … 10 * 10 = 100 Relations (Operator 1) Location_A=Austria* - Location_B=Austria* Location_A=Austria* - Location_B=Belgium* Location_A=Austria* - Location_B=Croatia* ….. Location_A=Belgium* - Location_B=Austria* … 10 * 10 = 100 Relations (Operator 1)

38. ©2010 Keynote SIGOS GmbH Testing is our Competence VARIABLES in Test Definition Level Wherever an individual test parameter is needed  Usage of VARIABLES E.g.: SMSC Number for each operator / MMS settings (not needed for VoiceQ) 38

39. ©2010 Keynote SIGOS GmbH Testing is our Competence Extension of CSV file SMS / MMS VARIABLES can be defined in parent OR nested FOREACH structure VARIABLES for A-Side  A-Side CSV VARIABLES for B-Side  B-Side CSV 39 VARIABLES in Test Definition Level

40. ©2010 Keynote SIGOS GmbH Testing is our Competence Reference to VARIABLES within test definition (Synthax = [VALUE VarName]) 40 VARIABLES in Test Definition Level

41. ©2010 Keynote SIGOS GmbH Testing is our Competence All Operators CSV – Side A Configuration for first operator only Adding of information for operator 2 and 3 within CSV 41 Operator 1 Operator 2 Operator 3

42. ©2010 Keynote SIGOS GmbH Testing is our Competence 42 All Operators CSV – Side B

43. ©2010 Keynote SIGOS GmbH Testing is our Competence Final Step – Copy & Paste CSV Copy complete CSV configuration for A-Side and B-side 43 Mark complete CSV and copy

44. ©2010 Keynote SIGOS GmbH Testing is our Competence Final Step – Copy & Paste CSV Paste complete CSV configuration for A-Side and B-side into the parameter CSV_Text 44

45. ©2010 Keynote SIGOS GmbH Testing is our Competence Result A-Side 45 VALUES separated by Tab Stop

46. ©2010 Keynote SIGOS GmbH Testing is our Competence 46 Result B-Side VALUES separated by Tab Stop

47. ©2010 Keynote SIGOS GmbH Testing is our Competence Scheduling 47 900 Sequences are being started at once = 2700 Test Cases 900 Sequences are being started at once = 2700 Test Cases

48. ©2010 Keynote SIGOS GmbH Testing is our Competence Intra Operational Test Scenarios Tests between 1 operator only  3 Test Definitions One CSV file for each operator and side 48

49. ©2010 Keynote SIGOS GmbH Testing is our Competence CSV File Configuration for IP Tests FTP and WAP test cases  A-Side only 3 operators, 10 locations  60 test scenarios / test runs 49

50. ©2010 Keynote SIGOS GmbH Testing is our Competence VARIABLES for sequence Level 50 CSV File Configuration for IP Tests

51. ©2010 Keynote SIGOS GmbH Testing is our Competence 51 CSV File Configuration for IP Tests VARIABLES for test case level

52. ©2010 Keynote SIGOS GmbH Testing is our Competence 52 CSV File Configuration for IP Tests VARIABLES for test case level

53. ©2010 Keynote SIGOS GmbH Testing is our Competence Copy & Paste CSV to FOREACH parameter CSV_Text 53 CSV File Configuration for IP Tests

54. ©2010 Keynote SIGOS GmbH Testing is our Competence 54 VALUES separated by Tab Stop CSV File Configuration for IP Tests

55. ©2010 Keynote SIGOS GmbH Testing is our Competence 55 Scheduling 30 Sequences are being started at once = 60 Test Cases 30 Sequences are being started at once = 60 Test Cases

56. ©2010 Keynote SIGOS GmbH Testing is our Competence FOREACH Options 56 If only 1 or 2 VARIABLES are needed No need for CSV creation and configuration

57. ©2010 Keynote SIGOS GmbH Testing is our Competence FOREACH Options 57 If only 1 or 2 VARIABLES are needed No need for CSV creation and configuration Free Text Usage: [VALUE Number]

58. ©2010 Keynote SIGOS GmbH Testing is our Competence 58 FOREACH Options List of Patterns to include values for the selection list List of Patterns to ignore values for the selection list If set to ‚Yes‘ all matching values from the selection list will be taken If empty, all iterations are done as specified (DEFAULT) If not empty, FOREACH values are taken in random order If > 0, the SampleSize iterations are done If = 0, the number of iterations is as spedified by the FOREACH values If empty, all iterations are done as specified (DEFAULT) If not empty, FOREACH values are taken in random order If > 0, the SampleSize iterations are done If = 0, the number of iterations is as spedified by the FOREACH values

59. ©2010 Keynote SIGOS GmbH Testing is our Competence SET Control Structure Execution of a test case set with the same parameters E.g. a set of IP test cases Definition of VARIABLES  Assign individual VALUES Usage of the VALUES within test case parameters Modification of VALUES only within SET structure necessary 59

60. ©2010 Keynote SIGOS GmbH Testing is our Competence SET Control Structure Definition of all needed VARIABLES within SET 60

61. ©2010 Keynote SIGOS GmbH Testing is our Competence SET Control Structure Usage of [VALUE VarName] within test case parameters External Number should be changed  Modification of VARIABLE ExtNumber within SET only! 61

62. ©2010 Keynote SIGOS GmbH Testing is our Competence Advanced IVR Testing 62 IVR Test Scenarios

63. ©2010 Keynote SIGOS GmbH Testing is our Competence IVR Test Scenarios IVR  Interactive Voice Response Testing and verifying of IVR systems (voicemail) Fully supported by SITE System Generic Voicmal test case  VMS 63

64. ©2010 Keynote SIGOS GmbH Testing is our Competence IVR Test Scenarios Creation of a reference prompt Usage of IVR_RecordPrompt test case (IVR group) 64 Dial short code 1212 DTMF Sequence, if necessary Record 15 seconds

65. ©2010 Keynote SIGOS GmbH Testing is our Competence Open trace and follow AudioFile link 65 IVR Test Scenarios

66. ©2010 Keynote SIGOS GmbH Testing is our Competence Save recorded audio file as ALAW file Transfer audio file to CU (CU:/tmp/Advanced) 66 IVR Test Scenarios Save Audio File as ALAW (.al)

67. ©2010 Keynote SIGOS GmbH Testing is our Competence Configuration of OM_PR_RegisterPrompt_LU test case 67 IVR Test Scenarios CLU, where prompt should be registered Source Host of Audio File Path & Audio File Name on Source Host Name of new PromptPool Description for PromptPool Name of new Prompt

68. ©2010 Keynote SIGOS GmbH Testing is our Competence Test Run - Verdict After successful registration of prompt  IVR_RecognizePrompt test case can be used 68

69. ©2010 Keynote SIGOS GmbH Testing is our Competence IVR Test Case Configuration of IVR_RecognizePrompt test case 69 Short Code Expected Prompt Settings

70. ©2010 Keynote SIGOS GmbH Testing is our Competence Prompt Recognition Parameter 70 Host on which Prompt Recognizer is running Port of Prompt Recognizer (50044 DEFAULT) Start offset of AudioFile for recognition DEFAULT: Perform one Recognition tillEnd: Search for a Prompt within given AudioFile DEFAULT: Perform one Recognition tillEnd: Search for a Prompt within given AudioFile Threshold between sure & unsure Recognition Main directory of ReferencePrompt VERDICT for unsure Recognition

71. ©2010 Keynote SIGOS GmbH Testing is our Competence Prompt Recognition Result 71 Trace of successful Prompt Recognition Test Case

72. ©2010 Keynote SIGOS GmbH Testing is our Competence Advanced Scheduling 72 Intelligent Scheduling

73. ©2010 Keynote SIGOS GmbH Testing is our Competence Intelligent Scheduling Scheduling can be adapted to specific needs 73

74. ©2010 Keynote SIGOS GmbH Testing is our Competence All test senarios can be covered Very flexible scheduling 74 Abort After: Abort Test Case after waiting 4h Only on specific days Only between a specific period of a day Time to wait after starting of next test case execution within the order Intelligent Scheduling

75. ©2010 Keynote SIGOS GmbH Testing is our Competence Resource Filter For an intelligent scheduling, different resource filter can be used 75

76. ©2010 Keynote SIGOS GmbH Testing is our Competence Resource Filter Used by the resource management in order to select SIMs Value can be any SQL expression using attributes from the SIM table 76

77. ©2010 Keynote SIGOS GmbH Testing is our Competence Resource Filter Used by the resource management for selection of mobiles or other interfaces Value can be any SQL expression using attributes from the Interfaces table 77

78. ©2010 Keynote SIGOS GmbH Testing is our Competence Preferred PLMNs Used to set the preferred PLMN field on the SIM Especially useful while using RoamingTo option ‚99999‘ Parameter limited to 30 PLMNs 78

79. ©2010 Keynote SIGOS GmbH Testing is our Competence Preferred PLMNs Used to set the forbidden PLMN field on the SIM Parameter limited to 4 PLMNs 79

80. ©2010 Keynote SIGOS GmbH Testing is our Competence 80 Thank you.