1. HUAWEI TECHNOLOGIES CO., LTD. :32-35pt : R153 G0 B0 : FrutigerNext LT Medium : Arial :30-32pt : R153 G0 B0 : :20-22pt (2-5 ) :18pt : FrutigerNext LT Regular : Arial :18-20pt (2-5 ):18pt :. Employing TTCN-3 in a structured way for large-scale testing of telecom protocol components Balakrishna S Pol Prasanna Venkatesh Huawei Technologies India Private Ltd

2. HUAWEI TECHNOLOGIES CO., LTD.Huawei Confidential Page 2 Slide title :32-35pt Color: R153 G0 B0 Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Slide text :20-22pt Bullets level 2-5: 18pt Color:Black Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Top right corner for field-mark, customer or partner logotypes. ---------------- The following nine groups of colors are an example of how our design colors can be used, please take note that you should only use one design color group per slide. For specific usage details, refer to the Typesetting Standard. Employing TTCN-3 in a structured way for large scale Protocol testing Number of Protocols Supported Lines of Code Maintained Network Elements (NEs) deployment Of history with feature growth of almost 15% per Year

3. HUAWEI TECHNOLOGIES CO., LTD.Huawei Confidential Page 3 Slide title :32-35pt Color: R153 G0 B0 Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Slide text :20-22pt Bullets level 2-5: 18pt Color:Black Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Top right corner for field-mark, customer or partner logotypes. ---------------- The following nine groups of colors are an example of how our design colors can be used, please take note that you should only use one design color group per slide. For specific usage details, refer to the Typesetting Standard. If your protocol testing business is this large then Else you will experience Escalating Cost, Resource difficulty, Quality issues, T2M issues, Maintenance issues …

4. HUAWEI TECHNOLOGIES CO., LTD.Huawei Confidential Page 4 Slide title :32-35pt Color: R153 G0 B0 Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Slide text :20-22pt Bullets level 2-5: 18pt Color:Black Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Top right corner for field-mark, customer or partner logotypes. ---------------- The following nine groups of colors are an example of how our design colors can be used, please take note that you should only use one design color group per slide. For specific usage details, refer to the Typesetting Standard. Challenges in Large scale protocol testing Maintainability, re-usability and scalability of huge number of test suites Various types of testing to ensure the coverage and test quality Significant growth in terms of number, type & features of protocols

5. HUAWEI TECHNOLOGIES CO., LTD.Huawei Confidential Page 5 Slide title :32-35pt Color: R153 G0 B0 Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Slide text :20-22pt Bullets level 2-5: 18pt Color:Black Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Top right corner for field-mark, customer or partner logotypes. ---------------- The following nine groups of colors are an example of how our design colors can be used, please take note that you should only use one design color group per slide. For specific usage details, refer to the Typesetting Standard. Challenge 1: Evolution of protocols There are different kind of communication protocols being (by their representation and encoding) developed. Typically there seems to be 3 kinds: Text, XML based, Binary (ASN representation or vanilla TLV representation) For e.g. there are XML based telephony protocols like VXML/CCXML Different Kinds of protocols Varied Requirements Huge number and variety of NE deployment with protocols leading to very different kind of requirements For e.g. SIP is used in terminals, core telecom network, enterprise UC applications SIP RFCs per year Increase in protocol features Protocol Standardization has been continuously on the increase For e.g. SIP which is a decade old protocol, had 20 new standards added in 2011 So we need a test methodology and framework which can scale for different representations, different requirements from the network and different standardization requirements

6. HUAWEI TECHNOLOGIES CO., LTD.Huawei Confidential Page 6 Slide title :32-35pt Color: R153 G0 B0 Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Slide text :20-22pt Bullets level 2-5: 18pt Color:Black Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Top right corner for field-mark, customer or partner logotypes. ---------------- The following nine groups of colors are an example of how our design colors can be used, please take note that you should only use one design color group per slide. For specific usage details, refer to the Typesetting Standard. Approach: Test System for Protocol Testing Standalone protocol layer testing and not at product/solution level to helpverifying the protocols across devices irrespective of their nature Embedded –portable test stub for seamless communication with the protocol layer to avoid device related issues Generic framework to cater to different types of protocols. TTCN-3 can seamlessly take care of ASN.1 and Text. For XML based protocols a codec plug-in is used Systematic design of TTCN-3 test suite to accommodate varied requirement from customers and enhancement in standards Separating out the protocol layer testing, a systematic-reusable TTCN-3 suite and separating out of codecs to handle different encodings will help to meet the protocol evolution challenge

7. HUAWEI TECHNOLOGIES CO., LTD.Huawei Confidential Page 7 Slide title :32-35pt Color: R153 G0 B0 Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Slide text :20-22pt Bullets level 2-5: 18pt Color:Black Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Top right corner for field-mark, customer or partner logotypes. ---------------- The following nine groups of colors are an example of how our design colors can be used, please take note that you should only use one design color group per slide. For specific usage details, refer to the Typesetting Standard. Case Study: VoiceXML (VXML) testing TTCN3 test case handling multiple responses from the VXML interpreter. 1 2 3 VXML test system where VXML component interprets the vxml file and interacts with other systems through different interfaces asynchronously VXML is the W3C's standard XML format for specifying interactive voice dialogues between a human and a computer. This enables user to interact with voice applications over the telephone. Do not try to use too many languages or frameworks. For all practical purpose TTCN-3 is good enough for any protocol testing. But the framework should see if it can handle the complexity of the protocols

8. HUAWEI TECHNOLOGIES CO., LTD.Huawei Confidential Page 8 Slide title :32-35pt Color: R153 G0 B0 Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Slide text :20-22pt Bullets level 2-5: 18pt Color:Black Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Top right corner for field-mark, customer or partner logotypes. ---------------- The following nine groups of colors are an example of how our design colors can be used, please take note that you should only use one design color group per slide. For specific usage details, refer to the Typesetting Standard. Challenge 2: Coverage by testing types Conformance testing of standards API/interface testing Product scenario testing Functional Testing Verification of thread model by concurrently executing different/same test scenarios in multiple threads Concurrency Testing Distributed architecture testing of protocol stacks across physical nodes 1+1 or 1+N backup functionality of protocol stack Distributed & Backup Validation of performance requirements & benchmarking System failure and long run testing to verify robustness Performance & Stress So we need a test methodology and framework which enables us to carry out different types of testing

9. HUAWEI TECHNOLOGIES CO., LTD.Huawei Confidential Page 9 Slide title :32-35pt Color: R153 G0 B0 Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Slide text :20-22pt Bullets level 2-5: 18pt Color:Black Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Top right corner for field-mark, customer or partner logotypes. ---------------- The following nine groups of colors are an example of how our design colors can be used, please take note that you should only use one design color group per slide. For specific usage details, refer to the Typesetting Standard. Approach: Test Configurations for Protocol Testing 12 Multiple parallel test components (PTC) created under one main test component with required number of ports. And each PTC will be used to executed one test scenario so the concurrency test can be achieved Standalone protocol testing involve both message communication and API invocation Every SUT instance communicate with TTCN-3 suite with bidirectional message ports for both upper & lower level. Separate ports are created for invoking APIs of the SUT Such multiple SUT instances created to achieve the distributed and backup testing 1 2

10. HUAWEI TECHNOLOGIES CO., LTD.Huawei Confidential Page 10 Slide title :32-35pt Color: R153 G0 B0 Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Slide text :20-22pt Bullets level 2-5: 18pt Color:Black Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Top right corner for field-mark, customer or partner logotypes. ---------------- The following nine groups of colors are an example of how our design colors can be used, please take note that you should only use one design color group per slide. For specific usage details, refer to the Typesetting Standard. Case Study: Concurrency testing 1 2 Test code at SUT side which creates required multithreading framework by creating thread pools and queues. Concurrent test frameworks built using the Parallel Test Components (PTC) architecture of the TTCN3 Each PTC runs under MTC and have message and API invocation ports Multiple TCs runs in parallel using multiple PTCs achieving concurrency testing Sample Test Case 1 2 Through an adaptive usage of dynamic test configurations (PTC/MTC), TTCN-3 can be used for multiple types of testing, typically functional, distribution, reliability and concurrency testing.

11. HUAWEI TECHNOLOGIES CO., LTD.Huawei Confidential Page 11 Slide title :32-35pt Color: R153 G0 B0 Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Slide text :20-22pt Bullets level 2-5: 18pt Color:Black Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Top right corner for field-mark, customer or partner logotypes. ---------------- The following nine groups of colors are an example of how our design colors can be used, please take note that you should only use one design color group per slide. For specific usage details, refer to the Typesetting Standard. Challenge 3 : Long Time maintenance Test cases growth is a fact of life. We have experienced about 10% Y-o-Y growth in our test suites. If we did not have a maintainable, reusable and scalable test automation framework, the cost would have been severe and TCs would have grown much more steeply 60,000+ TCs 68,000+ TCs 1,16,000+ TCs 1,09,000+ TCs 93,000+ TCs 84,000+ TCs 76,000+ TCs 1,01,000+ TCs

12. HUAWEI TECHNOLOGIES CO., LTD.Huawei Confidential Page 12 Slide title :32-35pt Color: R153 G0 B0 Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Slide text :20-22pt Bullets level 2-5: 18pt Color:Black Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Top right corner for field-mark, customer or partner logotypes. ---------------- The following nine groups of colors are an example of how our design colors can be used, please take note that you should only use one design color group per slide. For specific usage details, refer to the Typesetting Standard. Approach: TTNC-3 Test Suite Design The bottom up approach (from Level-0 to Level-3) is used to design the components of the test suite Each layer focuses on the specific objective like Layer 0 for message structure definition, Layer 1 & 2 for common functions and Layer 3 for test scripts The design allows smooth addition or modification of test scripts in current & future releases and to reuse test suite seamlessly. Finally to achieve maintainability, reusability and scalability aspects.

13. HUAWEI TECHNOLOGIES CO., LTD.Huawei Confidential Page 13 Slide title :32-35pt Color: R153 G0 B0 Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Slide text :20-22pt Bullets level 2-5: 18pt Color:Black Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Top right corner for field-mark, customer or partner logotypes. ---------------- The following nine groups of colors are an example of how our design colors can be used, please take note that you should only use one design color group per slide. For specific usage details, refer to the Typesetting Standard. Case Study : Diameter Test Suite Design ASN.1 Definitions for Diameter messages, Diameter API parameters etc. TTCN-3 templates for Diameter message, Diameter API parameters etc. TTCN-3 Functions for SUT APIs, Sending/Receiving of Protocol message and common utilities TTCN-3 Functions for test scenario Pre-conditions TTCN-3 test cases for Diameter functionality for all kinds of test scenarios The scalable, reusable and maintainable framework is required to handle future needs. This requires the systematic approach toward the design of test automation framework.

14. HUAWEI TECHNOLOGIES CO., LTD.Huawei Confidential Page 14 Slide title :32-35pt Color: R153 G0 B0 Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Slide text :20-22pt Bullets level 2-5: 18pt Color:Black Corporate Font : FrutigerNext LT Medium Font to be used by customers and partners : Arial Top right corner for field-mark, customer or partner logotypes. ---------------- The following nine groups of colors are an example of how our design colors can be used, please take note that you should only use one design color group per slide. For specific usage details, refer to the Typesetting Standard. Consider the longevity of the test suite/framework before designing the framework. The longer the test suite is going to be maintained the more compact the test suites will become in the long run. Large scale protocol testing has the following attributes: Varied protocols Same Protocols used for a variety of purposes and devices Growth of protocol in one format or other over a very long time frame TTCN-3 architecture enables different types of protocol testing. There is almost no limit to the kind of testing you can do. Using different TTCN-3 concepts we can adapt to different circumstances Finally, if the scale of testing is large and long, initial investment into framework will have significant dividends. Take Away

15. Thank you www.huawei.com